thepetabyteproject / your

Your Unified Reader

Home Page:https://thepetabyteproject.github.io/your/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Fits to Filterbank conversion using your_writer has issue

sahilatri126 opened this issue · comments

I am processing AO327 data which is in a 4-bit data format. I used your_writer to convert those fits file to filterbank format so that I can later decimate it using sigproc. But the filterbank files produced by your_writer.py, when run on sigproc for decimation is showing " ERROR: input data to decimate is not in the filterbank format" and as a result I am not able to decimate those files.

To cross check I converted those files from fits to filterbank format using psrfits2fil.py tool of PRESTO. The filterbank files generated using this are accepted by sigproc for decimation and not showing any error.

On looking at the details of the filterbank file produced by psrfits2fil and your_writer.py using readfile command of PRESTO. I found that many values of frequency and other things are mentioned wrong in the filterbank file produced by your_writer.py

To check if the issue is only with the 4-bit data file or not. I did the same process on 16-bit data files and I am getting the same error.

To Reproduce
your can simply go inside /hyrule/data/ao327/4bitdata and run it on any file.
for some 16 bit files you can find it in /hyrule/data/users/satri/16_bit_test/ in the bowser

Expected behavior
your_writer.py must convert the files from fits to filterbank format properly so that it show all the details of frequencies correct and the filterbank format is proper and Accepted by sigproc for decimation.

Screenshots
error
sigproc_fil
your_writer_fil
16-bit_test

Versions (please provide the versions of the the following packages):

  • your -0.6.6

Additional context
Let me know if any other information is required from my side.

@sahilatri126 please run the above again with the --verbose flag and send us the outputs. Please also post the outputs of your_header.py on both of these files.

file1
file2
verbose

I did not see abything after using --verbose option I am attaching the your_writer log file.
your_writer_2022_01_22_16_13_12_702063.log

These don't make any sense. I don't understand what's this converted file is, how it was converted and what should it actually look like. Please provide:

  1. your_header.py output of the original psrfits file.
  2. your_header.py output of the file converted filterbank file using your.

And kindly label the screenshots so that we understand what image corresponds to what.

Ok, This converted file is the filterbank file generated after I run your_writer.py on a file 4bit_puppi_58888_Strip25_0068.0001.fits.
It was converted simple using " your_writer.py -f filename -t fil"
I am just getting error that it is not in filterbank format when I use this file for decimation in sifproc, files converted using psrfits2fil are not giving such error, so I consider the details of the data shown by that file is correct and if I cross check it with your_writer converted file I see that lowest frequency mentioned there is not matching with the psrfits2fil converted fil file.
I am not sure what is actually wrong with this fil format file generated but I just don't want to get error when I use sigproc on it for decimation. It shows the file is not a filterbank.

This is the original fits file header.
Screenshot from 2022-01-22 23-02-26

This is the generated filterbank file header
Screenshot from 2022-01-22 23-21-57

error of running sigproc on this converted file for decimation
Screenshot from 2022-01-22 23-25-03

@sahilatri126 We have made a fix and issued a new release. Install version 0.6.7 and confirm it from your end.