ValueError: Event 1001 does not exist in the file containing events 1000 - 1999!
pdeperio opened this issue · comments
From @pdeperio on February 11, 2017 21:53
Running on Midway:
paxer --config XENON1T --input /project2/lgrandi/xenon1t/raw/161023_1902 --event 1001 --output test
some runs return the following error:
ReadZipped MainProcess L102 INFO InputFromFolder: Selecting file /project2/lgrandi/xenon1t/raw/161023_1902/XENON1T-3995-000001000-000001999-000001000.zip (number 2/32 in folder) for reading
Traceback (most recent call last):
File "/project/lgrandi/anaconda3/envs/pax_head/bin/paxer", line 4, in <module>
__import__('pkg_resources').run_script('pax==6.3.1', 'paxer')
File "/project/lgrandi/anaconda3/envs/pax_head/lib/python3.4/site-packages/setuptools-20.3-py3.4.egg/pkg_resources/__init__.py", line 726, in run_script
File "/project/lgrandi/anaconda3/envs/pax_head/lib/python3.4/site-packages/setuptools-20.3-py3.4.egg/pkg_resources/__init__.py", line 1484, in run_script
File "/project/lgrandi/anaconda3/envs/pax_head/lib/python3.4/site-packages/pax-6.3.1-py3.4.egg/EGG-INFO/scripts/paxer", line 292, in <module>
main()
File "/project/lgrandi/anaconda3/envs/pax_head/lib/python3.4/site-packages/pax-6.3.1-py3.4.egg/EGG-INFO/scripts/paxer", line 116, in main
parallel.maybe_multiprocess(args, **config_kwargs)
File "/project/lgrandi/anaconda3/envs/pax_head/lib/python3.4/site-packages/pax-6.3.1-py3.4.egg/pax/parallel.py", line 166, in maybe_multiprocess
pax_instance.run()
File "/project/lgrandi/anaconda3/envs/pax_head/lib/python3.4/site-packages/pax-6.3.1-py3.4.egg/pax/core.py", line 310, in run
total=self.number_of_events)):
File "/project/lgrandi/anaconda3/envs/pax_head/lib/python3.4/site-packages/tqdm/_tqdm.py", line 459, in __iter__
for obj in iterable:
File "/project/lgrandi/anaconda3/envs/pax_head/lib/python3.4/site-packages/pax-6.3.1-py3.4.egg/pax/core.py", line 160, in get_events
yield self.input_plugin.get_single_event(event_number)
File "/project/lgrandi/anaconda3/envs/pax_head/lib/python3.4/site-packages/pax-6.3.1-py3.4.egg/pax/FolderIO.py", line 144, in get_single_event
self.event_numbers_in_current_file))
ValueError: Event 1001 does not exist in the file containing events 1000 - 1999!
Event numbers which do exist in file: [0, 1, ..., 999]
Copied from original issue: XENON1T/pax#494
From @JelleAalbers on February 23, 2017 22:4
It appears something went wrong transferring the data for this run (either on datamanager or somewhere in between), all the files have the identically same size and contain only event 0-999.
Indeed, problem with transfer to Midway though I cannot diagnose as the logs have been purged.
I will manually re-transfer from PDC, which seems to be good.
Corrupt raw data on Midway and Stash (login) now replaced, and old processed data deleted.