Cannot reproduce the PVQ numbers in test_PVQ_on_new_datasets.ipynb
jonfryd opened this issue · comments
Hi,
Very interesting work!
So for fun I tried following every step in test_PVQ_on_new_datasets.ipynb (the only difference being that I'm running a bunch of smaller Python programs instead a Jupyter notebook), but I'm getting different values on LIVE_VQC_tiny_all:
PVQ 0.704,0.605
Instead of 0.548,0.515 as stated in test_PVQ_on_new_datasets.ipynb. Are the numbers in the Jupyter notebook outdated perhaps?
I verified every intermediate step and everything seems to be OK in terms of tensor sizes and so on.
I'm curious if anyone has been able to reproduce the stated PVQ numbers successfully from the Jupyter notebook.
I tried running everything in a Jupyter notebook today to check if I had made a mistake maybe while breaking up the functionality of the full notebook into smaller Python scripts. Well, it's still the same issue as shown above. The SRCC and LCC values are off by quite a bit compared to the expected values.