Reading nbits = 1, 2 or 4 in fil_reader.py
evanocathain opened this issue · comments
Describe the bug
It doesn't seem that blimpy's fil_reader.py can read sigproc filterbank files unless they are 8, 16, 32 bits.
I believe it is due to this line:
self._n_bytes = int(self.header['nbits'] / 8) #number of bytes per digit.
Oddly enough I think without the int cast some downstream issues might work out ok, but it still wouldn't be right. So I suppose this a request that the standard 1, 2 and 4-bit reading methods be added.
@evanocathain This is addressed in issue #151 and PR #146 (but not fixed). I did not update the title of issue #151 but this is mentioned in the PR. Comments/suggestions are always welcome. Once the PR is merged, at least, the restriction of nbits to [8, 16, 32] will be formalized and there will be a specific diagnostic message. Unfortunately, the timing for support of other values of nbits is currently undefined.
Hmm, we did have basic support for 1/2/4 bits in earlier code, e.g.
blimpy/blimpy/deprecated/filterbank.py
Line 247 in b323993
There is some blimpy code already for reading low bitwidth data, however it seemed to be about twice as slow as sigpyproc (in a poorly controlled test).
I would, however, quite like to refactor the code to use numpy.memmap, and have better astropy integration; memmap doesn't support 1/2/4 bits so that's not going to make pulsar folks happy.
One course of action is to port sigpyproc's 1/2/4 bit reading in C using pybind11. Will look into this
Here's a simple 1/2/4 bit to/from 8 bit numpy array package: https://github.com/telegraphic/numbits
I think this is better as a separate package from blimpy as it does a very specific thing. Integration will take a while as it'll be a bit tricky to make sure strides are right and plotting works (I just wanted to do the fun bit tonight!)
Agree with last remark from @telegraphic