About MaxWell_val.csv
JennyVanessa opened this issue · comments
Could you please provide the MaxWell_val.csv file? Thank you very much :)
and when i used the train_single_maxwell.py to reproduce your result, the val srcc on overall_quality was only 0.471. i checked if this was a zero-shot clip version of your experiment, but seems like it is the final version( ctx+mlp+fastvqa features).
Let me check this. Before that, may I know how do you get the result without the MaxWell_val?
Let me check this. Before that, may I know how do you get the result without the MaxWell_val?
Thank you for your reply! i used the test_labels in 'examplar_data_labels/MaxWell/test_labels', i thought it was the overall quality MOS. and calculated all the 16 factors with ovarall quality MOS
This is strange. Let me check with our local file. I think the test_labels.txt (the one you used) should be correct, and we did not experience so strong over-fitting during our training.
Will return to you later.
Hi, we have released the MaxWell_val.csv, and it will be very good if you can retest on this.
Evaluating O: high quality<->low quality 0.8172193511084216
Evaluating A-1: good content<->bad content 0.6900310476046603
Evaluating A-2: organized composition<->chaotic composition 0.7053497215560715
Evaluating A-3: vibrant color<->faded color 0.7532806612508306
Evaluating A-4: contrastive lighting<->gloomy lighting 0.7490894818502545
Evaluating A-5: consistent trajectory<->incoherent trajectory 0.712315335449577
Evaluating A-all: good aesthetics<->bad aesthetics 0.7785322652954051
Evaluating T-1: sharp<->fuzzy 0.8294349839678878
Evaluating T-2: in-focus<->out-of-focus 0.7513342345710007
Evaluating T-3: noiseless<->noisy 0.7789598457907729
Evaluating T-4: clear-motion<->blurry-motion 0.759417238964974
Evaluating T-5: stable<->shaky 0.7788389080847824
Evaluating T-6: well-exposed<->poorly-exposed 0.7462807758603156
Evaluating T-7: original<->compressed 0.7636470678737879
Evaluating T-8: fluent<->choppy 0.686984365317068
Evaluating T-all: clear<->severely degraded 0.8262245231247884
Here is the result we tested from the official weights provided. You may see whether your runs can match this.