your_viewer.py fails
devanshkv opened this issue · comments
your_viewer.py -f /gbo/20m/57889/Skynet_57889_psr_b1822_09_28142_28500_0002.fits
Leads to
Traceback (most recent call last):
File "/home/dagarwal/soft/conda/envs/your_env/bin/your_viewer.py", line 4, in <module>
__import__('pkg_resources').run_script('your==0.5.6', 'your_viewer.py')
File "/home/dagarwal/soft/conda/envs/your_env/lib/python3.8/site-packages/pkg_resources/__init__.py", line 665, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "/home/dagarwal/soft/conda/envs/your_env/lib/python3.8/site-packages/pkg_resources/__init__.py", line 1463, in run_script
exec(code, namespace, namespace)
File "/home/dagarwal/soft/conda/envs/your_env/lib/python3.8/site-packages/your-0.5.6-py3.8.egg/EGG-INFO/scripts/your_viewer.py", line 295, in <module>
app.load_file(values.files, values.start, values.gulp, values.chan_std) # load file with user params
File "/home/dagarwal/soft/conda/envs/your_env/lib/python3.8/site-packages/your-0.5.6-py3.8.egg/EGG-INFO/scripts/your_viewer.py", line 125, in load_file
self.read_data()
File "/home/dagarwal/soft/conda/envs/your_env/lib/python3.8/site-packages/your-0.5.6-py3.8.egg/EGG-INFO/scripts/your_viewer.py", line 228, in read_data
self.data = self.your_obj.get_data(self.start_samp, self.gulp_size).T
File "/home/dagarwal/soft/conda/envs/your_env/lib/python3.8/site-packages/your-0.5.6-py3.8.egg/your/your.py", line 204, in get_data
data = self.formatclass.get_data(self, nstart, nsamp, pol=pol)[:, 0, :]
File "/home/dagarwal/soft/conda/envs/your_env/lib/python3.8/site-packages/your-0.5.6-py3.8.egg/your/formats/psrfits.py", line 384, in get_data
data.append(self.read_subint(fsub, pol=pol))
File "/home/dagarwal/soft/conda/envs/your_env/lib/python3.8/site-packages/your-0.5.6-py3.8.egg/your/formats/psrfits.py", line 259, in read_subint
data = data.reshape((self.nsamp_per_subint, self.nchan)).astype(np.float32)
ValueError: cannot reshape array of size 262144 into shape (32,1024)
Can you run it in debug mode and share the logs. I don't think it is an issue with your_viewer, but logs should help.
2020-10-07 11:12:40,303 - load_file - root - INFO - Reading file ['/gbo/20m/57889/Skynet_57889_psr_b1822_09_28142_28500_0002.fits'].
2020-10-07 11:12:40,304 - __init__ - your.your - DEBUG - Reading the file(s): ['/gbo/20m/57889/Skynet_57889_psr_b1822_09_28142_28500_0002.fits']
2020-10-07 11:12:40,459 - __init__ - your.your - DEBUG - Generating unified header for file Skynet_57889_psr_b1822_09_28142_28500_0002
2020-10-07 11:12:40,482 - __init__ - your.your - DEBUG - Successfully generated unified header for file /gbo/20m/57889/Skynet_57889_psr_b1822_09_28142_28500_0002.fits
2020-10-07 11:12:40,482 - load_file - root - INFO - Printing Header parameters
filelist: ['/gbo/20m/57889/Skynet_57889_psr_b1822_09_28142_28500_0002.fits']
filename: /gbo/20m/57889/Skynet_57889_psr_b1822_09_28142_28500_0002.fits
basename: Skynet_57889_psr_b1822_09_28142_28500_0002
source_name: psr_b1822_09
ra_deg: 276.403375
dec_deg: -9.589677777777778
bw: -500.0
center_freq: 1550.0
nbits: 16
dtype: <class 'numpy.uint16'>
format: fits
time_decimation_factor: 1
frequency_decimation_factor: 1
native_tsamp: 0.004194304
native_foff: -0.48828125
native_nchans: 1024
native_nspectra: 167264
fch1: 1800.0
npol: 4
tstart: 57889.20090869939
gl: -158.53899954118444
gb: 1.3013420298292275
tstart_utc: 2017-05-16T04:49:18.512
tsamp: 0.004194304
nchans: 1024
foff: -0.48828125
nspectra: 167264
2020-10-07 11:12:40,483 - get_data - your.your - DEBUG - Reading 4096 samples from sample 0
2020-10-07 11:12:40,483 - get_data - your.your - DEBUG - time_decimation_factor: 1
2020-10-07 11:12:40,483 - get_data - your.your - DEBUG - frequency_decimation_factor: 1
2020-10-07 11:12:40,483 - get_data - your.formats.psrfits - DEBUG - Number of spectra to skip from start: 0
2020-10-07 11:12:40,483 - get_data - your.formats.psrfits - DEBUG - Number of spectra to truncate from end: 0
2020-10-07 11:12:40,483 - get_data - your.formats.psrfits - DEBUG - Startsub 0, endsub 127
2020-10-07 11:12:40,483 - get_data - your.formats.psrfits - DEBUG - isub is 0
2020-10-07 11:12:40,483 - get_data - your.formats.psrfits - DEBUG - file id is 0
2020-10-07 11:12:40,483 - get_data - your.formats.psrfits - DEBUG - Using: [<astropy.io.fits.hdu.image.PrimaryHDU object at 0x7ff812ebb3d0>, <astropy.io.fits.hdu.table.BinTableHDU object at 0x7ff811ba7f70>]
2020-10-07 11:12:40,484 - get_data - your.formats.psrfits - DEBUG - Reading subint 0 in file /gbo/20m/57889/Skynet_57889_psr_b1822_09_28142_28500_0002.fits
Traceback (most recent call last):
File "/home/dagarwal/soft/conda/envs/your_env/bin/your_viewer.py", line 4, in <module>
__import__('pkg_resources').run_script('your==0.5.6', 'your_viewer.py')
File "/home/dagarwal/soft/conda/envs/your_env/lib/python3.8/site-packages/pkg_resources/__init__.py", line 665, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "/home/dagarwal/soft/conda/envs/your_env/lib/python3.8/site-packages/pkg_resources/__init__.py", line 1463, in run_script
exec(code, namespace, namespace)
File "/home/dagarwal/soft/conda/envs/your_env/lib/python3.8/site-packages/your-0.5.6-py3.8.egg/EGG-INFO/scripts/your_viewer.py", line 295, in <module>
app.load_file(values.files, values.start, values.gulp, values.chan_std) # load file with user params
File "/home/dagarwal/soft/conda/envs/your_env/lib/python3.8/site-packages/your-0.5.6-py3.8.egg/EGG-INFO/scripts/your_viewer.py", line 125, in load_file
self.read_data()
File "/home/dagarwal/soft/conda/envs/your_env/lib/python3.8/site-packages/your-0.5.6-py3.8.egg/EGG-INFO/scripts/your_viewer.py", line 228, in read_data
self.data = self.your_obj.get_data(self.start_samp, self.gulp_size).T
File "/home/dagarwal/soft/conda/envs/your_env/lib/python3.8/site-packages/your-0.5.6-py3.8.egg/your/your.py", line 204, in get_data
data = self.formatclass.get_data(self, nstart, nsamp, pol=pol)[:, 0, :]
File "/home/dagarwal/soft/conda/envs/your_env/lib/python3.8/site-packages/your-0.5.6-py3.8.egg/your/formats/psrfits.py", line 384, in get_data
data.append(self.read_subint(fsub, pol=pol))
File "/home/dagarwal/soft/conda/envs/your_env/lib/python3.8/site-packages/your-0.5.6-py3.8.egg/your/formats/psrfits.py", line 259, in read_subint
data = data.reshape((self.nsamp_per_subint, self.nchan)).astype(np.float32)
ValueError: cannot reshape array of size 262144 into shape (32,1024)Yup. Something is wrong in get_data.
I guess that's get_data in psrfits.py?
PS: This is 16 bit and 4 pol data.
shape of self.fits['SUBINT'].data[isub]['DATA'] is (32, 4, 1024, 2)
the code assumes that that last axis would be 1. I think this is why this error is happening.
y.fits['SUBINT'].header['TDIM17'] gives (2,1024,4,32)'. There it says Dimensions (NBITS or NBIN,NCHAN,NPOL,NSBLK). But this is not a typical thing for 16 bit data.
Also, both the arrays for that 4th axis have some data. Still not sure what that axis is for.
For completeness sake, here is what's happening here. The data is unsigned 16 bit which has been split into two 8 bit chunks and saved in 0 and 1 index of the data array returned by astropy fits. Here 0th index returns the lower 8 bits and 1 index returns the higher 8 bits. This is probably because there is no uint16 type in fits formats (here).
So we can add in the condition to check the length of this 4th axis, and if it is 2, then interpret the data accordingly.
Also, these are supposed to be ints and not uints.
Fixed with 1c70986.