[Multiple correct outputs] Modify generated XML so that results are always uploaded
canihavesomecoffee opened this issue · comments
https://github.com/CCExtractor/sample-platform/blob/master/mod_ci/controllers.py#L266-L284 needs modification to always insert a fixed filename to be compared against if the original file would need to be compared. This should result in the test suite always uploading the resulting produced file, so we can compare it on the platform rather than during the test.
Hey @canihavesomecoffee !
Can I work on this issue?
Yes you can, but it'll only be for getting to know the platform. There's an open PR (#489) that'll close all these Multiple Correct Outputs at once once it's merged (and it's close to being merged)
@canihavesomecoffee I don't think #489 is solving this issue. I believe, @adityaofficial10 you can start working on the issue. You can ask @kvshravan for help as he has worked on the related issues.
@thealphadollar We have decided not to upload results every time, instead we decided to split the check between the platform and the testsuite as then we no longer need to check the obtained result file against original files. The test suite will do its part comparing the original files, even if there are multiple original files. The test suite does this part efficiently by checking if the result file exists in the directory. If we were to implement the original file check on platform it wont be as efficient as the testsuite cause the result file needs to be compared against each of those original files ( assume the CEA-708 case where there can be multiple original files). So we decided whenever the got is not None we know the result is not original file or not one of the original files and thus the got value can be compared with the variants on the platform. So this issue is no longer needed IMO.
Oh, cool. Thanks for helping me understand.
@canihavesomecoffee shall we close this issue?