Cleanup of tar files which were not uploaded/deleted
nemobis opened this issue · comments
We have some tar files which don't get cleaned up. I'm not sure how to check what is in the redis queue.
$ ls -lht /data/scratch/BUB_downloads/*tar*
-rw-r--r-- 1 tools.bub tools.bub 28M Feb 14 15:03 /data/scratch/BUB_downloads/bub_gal_ark_12148_bpt6k80993x_images.tar
-rw-r--r-- 1 tools.bub tools.bub 8.8M Feb 14 14:54 /data/scratch/BUB_downloads/bub_gal_ark_12148_bpt6k375846s_images.tar
-rw-r--r-- 1 tools.bub tools.bub 417M Feb 14 14:31 /data/scratch/BUB_downloads/bub_gb_c9SIbcxGToMC_images.tar
-rw-r--r-- 1 tools.bub tools.bub 128M Feb 14 14:14 /data/scratch/BUB_downloads/bub_gb_c9r_bNd_C_4C_images.tar
-rw-r--r-- 1 tools.bub tools.bub 5.0M Feb 14 11:05 /data/scratch/BUB_downloads/bub_gb_C9Pgt1EPWbwC_images.tar
-rw-r--r-- 1 tools.bub tools.bub 76M Feb 10 01:57 /data/scratch/BUB_downloads/bub_gb_Nu8pAAAAYAAJ_images.tar
-rw-r--r-- 1 tools.bub tools.bub 105M Feb 9 01:03 /data/scratch/BUB_downloads/bub_gb_bM25rYaSBxYC_images.tar
-rw-r--r-- 1 tools.bub tools.bub 200M Feb 8 08:53 /data/scratch/BUB_downloads/bub_gb_BjMKAwDKwAMC_images.tar
-rw-r--r-- 1 tools.bub tools.bub 131M Feb 7 22:48 /data/scratch/BUB_downloads/bub_gb_PokUAAAAYAAJ_images.tar
-rw-r--r-- 1 tools.bub tools.bub 160M Feb 7 22:45 /data/scratch/BUB_downloads/bub_gb_bIC6hga9NnUC_images.tar
-rw-r--r-- 1 tools.bub tools.bub 62M Feb 6 16:27 /data/scratch/BUB_downloads/bub_gb_B7svnAknY3gC_images.tar
-rw-r--r-- 1 tools.bub tools.bub 53M Feb 6 16:26 /data/scratch/BUB_downloads/bub_gb_b7q3O2VIy98C_images.tar
-rw-r--r-- 1 tools.bub tools.bub 7.9M Feb 6 16:23 /data/scratch/BUB_downloads/bub_gb_b7nF8S6BmosC_images.tar
-rw-r--r-- 1 tools.bub tools.bub 182M Feb 5 08:16 /data/scratch/BUB_downloads/bub_gb_aXNa4u56zBgC_images.tar
-rw-r--r-- 1 tools.bub tools.bub 457M Feb 4 23:22 /data/scratch/BUB_downloads/bub_gb_u6pEAAAAcAAJ_images.tar
-rw-r--r-- 1 tools.bub tools.bub 154M Feb 3 02:06 /data/scratch/BUB_downloads/bub_gb_AQnLJHyv4c8C_images.tar
-rw-r--r-- 1 tools.bub tools.bub 14M Feb 2 18:13 /data/scratch/BUB_downloads/bub_gb_trJBAAAAcAAJ_images.tar
-rw-r--r-- 1 tools.bub tools.bub 86M Feb 2 18:09 /data/scratch/BUB_downloads/bub_gb_FtFAAQAAMAAJ_images.tar
-rw-r--r-- 1 tools.bub tools.bub 103M Feb 2 18:05 /data/scratch/BUB_downloads/bub_gb_aPmkoYrG6WQC_images.tar
-rw-r--r-- 1 tools.bub tools.bub 286M Feb 2 18:02 /data/scratch/BUB_downloads/bub_gb_trIpAQAAMAAJ_images.tar
-rw-r--r-- 1 tools.bub tools.bub 129M Feb 2 12:24 /data/scratch/BUB_downloads/bub_gb_aOvO8o1vxyIC_images.tar
-rw-r--r-- 1 tools.bub tools.bub 3.5M Feb 2 12:18 /data/scratch/BUB_downloads/bub_gb_aOVc1tWFatgC_images.tar
-rw-r--r-- 1 tools.bub tools.bub 201M Feb 2 12:18 /data/scratch/BUB_downloads/bub_gb_tpYTAAAAQAAJ_images.tar
-rw-r--r-- 1 tools.bub tools.bub 18M Feb 2 12:17 /data/scratch/BUB_downloads/bub_gb_AOuTrLs-eS4C_images.tar
-rw-r--r-- 1 tools.bub tools.bub 152M Feb 1 19:28 /data/scratch/BUB_downloads/bub_gb_am_GLeB7lXMC_images.tar
-rw-r--r-- 1 tools.bub tools.bub 19M Feb 1 19:26 /data/scratch/BUB_downloads/bub_gb_tjYyAQAAMAAJ_images.tar
-rw-r--r-- 1 tools.bub tools.bub 138M Feb 1 19:25 /data/scratch/BUB_downloads/bub_gb_tjYsAAAAYAAJ_images.tar
-rw-r--r-- 1 tools.bub tools.bub 5.4M Feb 1 19:23 /data/scratch/BUB_downloads/bub_gb_AmfxS0CFIyAC_images.tar
-rw-r--r-- 1 tools.bub tools.bub 146M Feb 1 19:22 /data/scratch/BUB_downloads/bub_gb_AMFVomPaOlsC_images.tar
-rw-r--r-- 1 tools.bub tools.bub 73M Feb 1 19:20 /data/scratch/BUB_downloads/bub_gb_tjYYAAAAYAAJ_images.tar
-rw-r--r-- 1 tools.bub tools.bub 141M Feb 1 19:17 /data/scratch/BUB_downloads/bub_gb_tjVMAQAAIAAJ_images.tar
-rw-r--r-- 1 tools.bub tools.bub 180M Feb 1 19:16 /data/scratch/BUB_downloads/bub_gb_AMfpVKxANl8C_images.tar
-rw-r--r-- 1 tools.bub tools.bub 155M Feb 1 15:21 /data/scratch/BUB_downloads/bub_gb_Alxu4AhT0lcC_images.tar
-rw-r--r-- 1 tools.bub tools.bub 412M Jan 29 23:35 /data/scratch/BUB_downloads/bub_gb_acUprmtsMGcC_images.tar
-rw-r--r-- 1 tools.bub tools.bub 278M Jan 29 15:53 /data/scratch/BUB_downloads/bub_gb_bF8-AQAAMAAJ_images.tar
-rw-r--r-- 1 tools.bub tools.bub 114M Jan 27 21:35 /data/scratch/BUB_downloads/bub_gb_t70UAAAAYAAJ_images.tar
-rw-r--r-- 1 tools.bub tools.bub 439M Jan 27 21:35 /data/scratch/BUB_downloads/bub_gb_A1lCoy8JHkUC_images.tar
-rw-r--r-- 1 tools.bub tools.bub 165M Jan 26 08:37 /data/scratch/BUB_downloads/bub_gb_9S4qXE7GU-UC_images.tar
-rw-r--r-- 1 tools.bub tools.bub 540M Jan 25 22:11 /data/scratch/BUB_downloads/bub_gb_9PecqfzyzVMC_images.tar
-rw-r--r-- 1 tools.bub tools.bub 88M Jan 24 23:05 /data/scratch/BUB_downloads/bub_gb_9Htph5DDY58C_images.tar
-rw-r--r-- 1 tools.bub tools.bub 536M Jan 24 21:58 /data/scratch/BUB_downloads/bub_gb_-9gOuMLrSFMC_images.tar
-rw-r--r-- 1 tools.bub tools.bub 27M Jan 24 11:15 /data/scratch/BUB_downloads/bub_gb_97lJnk6si_wC_images.tar
-rw-r--r-- 1 tools.bub tools.bub 187M Jan 23 02:53 /data/scratch/BUB_downloads/bub_gb_8kDERUFC4o4C_images.tar
-rw-r--r-- 1 tools.bub tools.bub 621M Jan 22 21:10 /data/scratch/BUB_downloads/bub_gb_8In_BgYkO0MC_images.tar
-rw-r--r-- 1 tools.bub tools.bub 90M Jan 22 19:44 /data/scratch/BUB_downloads/bub_gb_8icca7yTZwIC_images.tar
-rw-r--r-- 1 tools.bub tools.bub 23M Jan 22 19:37 /data/scratch/BUB_downloads/bub_gb_8ibOuCI-yusC_images.tar
I checked two books of those:
- one gives 404 with the API http://googleapis.com/books/v1/volumes/tpYTAAAAQAAJ
- the other is available and PD https://www.googleapis.com/books/v1/volumes/am_GLeB7lXMC but was already uploaded in 2014 https://archive.org/details/bub_gb_am_GLeB7lXMC , weird that it got queued in the first place.
So I don't have any reason to think there is something (seriously) going wrong yet.