Giters

RF5 / simple-asgan

Training code and trained checkpoints for ASGAN.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Some questions about the reproduction of this paper: from newcomer

wangxuanji opened this issue · comments

commented

Hello, I am a novice in the field of speech, and I don't understand some things. I hope you don't take the trouble to answer my questions. Thank you again.

After training the code, I received a bunch of checkpoints. How can I obtain the model evaluation indicators mentioned in your paper? For example, IS, mIS, FID, this indicator, should I calculate it myself? I don't seem to have found any calculation in the code.

The question may seem childish to you, but they are really important to me. Could you please answer them and express my gratitude to you again.

commented

Hi @wangxuanji , thanks for your questions, I'll try answer them as best I can:

  • How can I obtain the model evaluation indicators mentioned in your paper?

    You can do this by generating predictions for the SC09 test set that I use and then computing the same metrics as in the paper. I realize that I haven't provided the exact test splits, so I have added the SC09 test and validation splits as csv files in the releases tab, which should help.

  • For example, IS, mIS, FID, this indicator, should I calculate it myself? I don't seem to have found any calculation in the code.

    Yep you'd need to compute them yourself if you want to reproduce the same results. I ultimately didn't include the metric code in this repo because it is largely copied from existing repositories, where there is relatively original work in the evaluation code. For example, FID score is adapted from here, ndb score adapted from here, inception score is adapted from here. My evaluation script cobbled metrics from these places and ran the output from our model and the baselines through them. My eval script is quite dirty and involves lots of local paths, but to make your life easier, I uploaded a cleaned version of the main metrics in one script to the repo, here -- this should be useful in your efforts.

I hope that helps! Good luck with your reproduction!

commented

Closing for now, feel free to open again if you have other questions