Downloading scores from okpy.org doesn't always work
adnanhemani opened this issue · comments
When downloading scores for single assignments from the okpy.org website, sometimes some scores aren't included in the downloaded csv file that are in okpy. Unsure how to reproduce, but seems like the tendency is for downloading big files - and sometimes switching computers helps to get the complete file (but even this is not always the case).
It's a pretty big pain to be able to validate that all the rows of scores are in a particular download as well.
Will try to look into this. Can you provide an example of when this occurred (even if inconsistently)?
We would download grades for Data 8 last semester. Almost every other time we'd download grades for an assignment with lots of records, we'd have at least 5 missing entries. Again, no idea how to reproduce or help you reproduce this issue other than to just try it out on our class. But again, due to student data privacy issues, I'm not sure if I can add you to the class to help test it out? Anything else you'd deem helpful from me?
@adnanhemani Could you give @ja5087 a sense of scale so we can try to replicate. How many students? How many submissions? How many scores?
Sure, it was ~1200 students, about as many submissions (at least the ones that would get graded), and twice the number of scores (we would have one autograded score and one written score that we would import from Gradescope). Total amount of rows that should be in the export would be anywhere between 1700 - 2600. Thank you all for looking into this!! Really, really appreciate it!
Update: I think I've managed to reproduce this. It starts to happen if someone else is concurrently downloading scores for another assignment (or perhaps any other heavy database load?). Will look into ways we can perhaps queue these tasks up or make them effectively atomic.
@ja5087 that might be it...I definitely used to download multiple assignments at a time. Thank you for your work on this!
I get the feeling this might be related to our liveness checks on Kubernetes. Downloading is a long-running task that causes the web app to block completely (i.e. won't respond to /healthz).