How to use only several randomly selected pages?
janvainer opened this issue · comments
I need to evaluate 4 different speech synthesis systems.
I have 100 sentences and I have synthesized audio of each sentence from each TTS system.
The users should evaluate a sentence generated by all systems with mushra method, but I do not want a single user to have to go through 100 tests. Is it possible to randomly select eg. 10 pages that would be presented to the user?
@lordofluck I think the easiest way to implement this is to generate and serve the config files dynamically using e.g. python flask so that users will get a random experiment presented. Doing this externally also gives you more control over how to save the results (e.g. in a centralized db instead of plain files)