Zenodo auth valid but then fails when trying to cache
broulston opened this issue · comments
Hi there,
I'm trying to build a project with intermediate results cached in the Zenodo Sandbox. It appears the Sandbox was created successfully (I can see it in the browser) and when I run showyourwork build the line User authentication for 10.5072/zenodo.1181485 is valid. appears after setting up the workflow line.
The build continues, showyourwork finds that it needs to make the data products, and then builds the data files. Once the files are made, it shows Caching output file on remote: src/data/mass_needed_all.pkl.... However, after a minute or so, it errors out with the following Zenodo Sandbox authentication failed. Unable to upload cache for rule make_data_compar_dM_vs_CtoO. and fails to cache the data files.
Any idea what is happening? I've attached the showyourwork.log as well
I include the full output below for reference:
git:(main) ✗ showyourwork build
Fetching Overleaf repo...
Pulling changes from Overleaf: src/tex/ms.tex src/tex/bib.bib src/tex/aasjournal.bst src/tex/aassymbols.tex src/tex/aastex631.cls
No Overleaf changes to commit to the repo.
Setting up the internal conda environment...
Setting up the workflow...
User authentication for 10.5072/zenodo.1181485 is valid.
Building the workflow graph...
Generating figure output: src/tex/figures/needed_mass_vs_CtoO_compare_FeH.pdf...
Generating figure output: src/tex/figures/needed_mass_vs_CtoO_compare_Misalski2013.pdf...
Running user rule make_data_compar_dM_vs_CtoO...
Searching remote file cache: src/data/mass_needed_all.pkl...
Required version of file not found in cache: src/data/mass_needed_all.pkl.
Running rule from scratch...
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:10<00:00, 1.06s/it]
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:10<00:00, 1.03s/it]
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:10<00:00, 1.05s/it]
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:10<00:00, 1.01s/it]
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:10<00:00, 1.02s/it]
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:10<00:00, 1.03s/it]
[showyourwork.log](https://github.com/showyourwork/showyourwork/files/11174881/showyourwork.log)
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 6/6 [01:02<00:00, 10.34s/it]
Caching output file on remote: src/data/mass_needed_all.pkl...
Zenodo Sandbox authentication failed. Unable to upload cache for rule make_data_compar_dM_vs_CtoO.
Generating figure output: src/tex/figures/dC_mass_needed_vs_CtoO_FeH-0.pdf...
/Users/benjaminroulston/Dropbox/Research/Manuscripts/dwarf-carbon-star-formation-paper/src/scripts/make_compare_dM_vs_CtoO.py:78: UserWarning: Tight layout not applied. tight_layout cannot make axes width small enough to accommodate all axes decorations
plt.tight_layout()
Running user rule make_large_data_compar_dM_vs_CtoO...
Searching remote file cache: src/data/mass_needed_all_large.pkl...
Required version of file not found in cache: src/data/mass_needed_all_large.pkl.
Running rule from scratch...
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [01:42<00:00, 1.03s/it]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [01:45<00:00, 1.05s/it]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [01:43<00:00, 1.04s/it]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [01:42<00:00, 1.03s/it]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [01:42<00:00, 1.02s/it]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [01:42<00:00, 1.03s/it]
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 6/6 [10:19<00:00, 103.28s/it]
Caching output file on remote: src/data/mass_needed_all_large.pkl...
Zenodo Sandbox authentication failed. Unable to upload cache for rule make_large_data_compar_dM_vs_CtoO.
Generating figure output: src/tex/figures/dC_median_mass_needed_vs_Mstar_FeH-0.pdf...
Setting up the internal conda environment...
Rendering the article DAG...
Generating the article PDF...
Fetching Overleaf repo...
Pushing changes to Overleaf: src/tex/figures src/tex/output
Done!
Hi - this may be related to the problem I reported in #336 - I've been in contact with Zenodo support but they haven't found the problem. The Zenodo twitter account says they were carrying out some upgrade on March 28, and I think that's when the cached functionality stopped working.
No idea how to fix/mitigate this.....
@broulston — Interesting question! It's definitely not obvious to me what's going on here. I don't think it's related to the issue that @mkenworthy brought up here. For this one, I'm not sure what the best solution is. Perhaps you could try deleting the cache with
showyourwork cache deleteand then trying to regenerate it. If that doesn't work, I'm happy to keep brainstorming!
Running showyourwork cache delete results in the following
Deleting Zenodo Sandbox deposit with concept DOI 10.5072/zenodo.1181485...
The provided `id` 1181485 does not seem to be a valid Zenodo version or concept id.
Checking the Zenodo Sandbox site in the browser still show's the deposit so it indeed did not delete.
Can you confirm that it's a "draft" upload on Zenodo Sandbox, and then perhaps send a screenshot of the deposit listing, since I can't see it without your credentials?
On the upload page, it does appear to be a draft. I tried a showyourwork clean and then a showyourwork cache delete and the same result as above.
@broulston — Very strange! It might be worth iterating on this out of thread and then reporting back a solution if we can find it. Can you email me (should be easy to find my email...) so that we can follow up there? Thanks!
To report back on my issue, zenodo restored all my sandbox records and it started working again. No explanation for where all the records went to for two weeks......