haoheliu / ssr_eval

Evaluation and Benchmarking of Speech Super-resolution Methods

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

metrics inconsistent with paper

l6chen opened this issue · comments

Hi, when I evaluated the example model NVSR like this:

import soundfile as sf
device="cuda"
print("device", device)
for test_name in ["NVSRPostProcTestee"]:
    testee = eval(test_name)(device=device)
    helper = SSR_Eval_Helper(
        testee,
        test_name=test_name,
        input_sr=16000,
        output_sr=16000,
        evaluation_sr=16000,
        setting_fft={
            "cutoff_freq": [4000],
        },
        save_processed_result=False,
    )
    helper.evaluate(limit_test_nums=-1, limit_test_speaker=-1)

I got the final "lsd": 1.0318345543848155, which is not consistent with the result you provide in your paper, could you help me review my test code above?
FYI: I am using the following conda environment:
librosa 0.10.1
lightning-utilities 0.8.0
llvmlite 0.39.1
matplotlib 3.5.1
matplotlib-inline 0.1.6
numba 0.56.4
numpy 1.21.6
opencv-python 4.1.2.30
opencv-python-headless 4.5.4.60
packaging 21.3
pandas 1.1.5
Pillow 9.2.0
pip 23.0.1
protobuf 3.20.1
psutil 5.8.0
pyparsing 3.0.9
pytest-runner 5.3.0
pytorch-lightning 1.9.5
PyWavelets 1.3.0
PyYAML 6.0
requests 2.27.1
scikit-image 0.19.3
scikit-learn 0.22.1
scipy 1.5.2
semantic-version 2.8.5
setuptools 65.3.0
six 1.16.0
sklearn 0.0
soundfile 0.12.1
soxr 0.3.6
ssr-eval 0.0.7
tensorboard 2.11.2
tensorboard-data-server 0.6.1
tensorboard-plugin-wit 1.8.1
tensorboardX 2.0
terminaltables 3.1.10
text-unidecode 1.3
thop 0.1.1.post2207130030
tifffile 2021.11.2
torch 1.10.2
torchaudio 0.10.2
torchlibrosa 0.0.7
torchlightning 0.0.0
torchmetrics 0.11.1
torchvision 0.11.3
tornado 6.2
tqdm 4.64.1
voicefixer 0.0.17
Wave 0.0.2
wheel 0.37.1
yarl 1.8.1
zipp 3.8.1

Never mind. I misunderstand the sample rate and cutoff freq used in the code.