Giters

clovaai / aasist

Official PyTorch implementation of "AASIST: Audio Anti-Spoofing using Integrated Spectro-Temporal Graph Attention Networks"

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

What is the reason behind not being able to reproduce results unless we get same environment as you have?

hashim19 opened this issue · comments

commented

I trained the AASIST-L model with the following environment,

GPU: 1 NVIDIA RTX A2000 12 GB
GPU Driver: 535.104.12
Cuda Version: 12.2

I got an EER of 1.47% and a min-tDCF of 0.0465

Does this mean I cannot reproduce results without an NVIDIA TESLA V100?

commented

Hi. I don't think it means that you need a V100 GPU in order to reproduce.

In my opinion, what you're observing is the same thing happening with below paper, results being too fluctuant.

  • X. Wang and J. Yamagishi, "A Comparative Study on Recent Neural Spoofing Countermeasures for Synthetic Speech Detection," in Proc. Interspeech, 2021.

At this moment, the practical advice I can give is to try with different environments (e.g., torch version, cudnn version, ...). We've seen people who were able to reproduce results as well as similar cases to yours.
It's a pity to see people reporting this even though we reported best & average performance with three random seeds.