Temporary file deleted by Codalab
tjj0502 opened this issue · comments
Hi all, we encountered an issue with the submission files. We plan to launch a remote server instance once a week and it is expected to evaluate all the submissions in the queue. However, we noticed that Codalab deletes the submission files after a certain time. Resulting in the following error in worker.py:
"Program bundle is not available."
How can we resolve this issue?
Hello,
It is not a bug but a feature: the compute workers (the server you setup with this doc, having compute_worker.py) are made only for computing submissions. They do not serve the purpose of storing the submissions or the results. This is why everything is removed after each job.
The instance of the platform, in the other hand, stores everything inside a dedicated storage. As a competition organizer, you can typically download submissions files and results from the "Submissions" panel.
Is there any problem with this approach regarding your project?
Hello,
Thank you for your reply.
We did realize that we can rerun the submission and the bundle is indeed stored in the platform and we could rerun the evaluation. However, we have not configured out the cause of the error yet, see the message below:
Traceback (most recent call last):
File "/worker/worker.py", line 364, in run
raise Exception("Program bundle is not available.")
Exception: Program bundle is not available.
The submission was done on Friday and we launched the server this morning, and it detected the task in the worker queue, however, it failed. Do you know the reason for it and how can we fix it?
It is the first time I see this error. I think more logs are needed to understand and debug the problem.
Also, while this should not make a difference, you can try to make a submission while the worker is already launched.