Support for volume-reduced data (event-wise pixel selection)
moralejo opened this issue · comments
We need to read data in which only pixels which are likely to contain a useful signal are written out.
The written out pixels are indicated by bits 0 - 1 of the pixel status. If both are set to 0 the pixel waveform won't be present in the file.
Problem: in the original raw data (and in the gain-reduced version) all pixels present, but status bits 0 & 1 are set to 0. How can we know that a certain file is reduced and, hence the two bits should be considered when reading the events?
Also: for interleaved events (for which in the reduced files we keep both gains for all pixels), what should be the DVR status bits? (as of now they are not modified, so they should be 00)
The best thing would be to write DL0 objects, but I think that would loose all lst specific information.
So we probably can't do that.
The next best thing I thinknwould be to add a header keyword indicating that dvr has been applied, which can be then checked by the source.
As to the calib events, I'd enable the bits for them.
I can enable the bits for interleaveds. But I am not sure how to do the header thing. The original non-reduced files must be readable, so we cannot simply add a field, right?
On the other hand I wonder if going back to using the gain status bits ("channel info") to indicate what is saved for the pixel, is an acceptable solution. That already works with the current source, and I am not sure it has any downside other than being a slight "misuse" of the bits' meaning - and losing the info on whether a non-saved pixel had a saturated high gain or not. But the answer to the latter is always "no" - a saturated high gain means >~100 pe so such a signal would never be dropped.
The original non-reduced files must be readable, so we cannot simply add a field, right?
Adding a new header keyword to the gain reduced files won't create issues with the existing files.
You can check for the existence of a header keyword and assume it is false if it doesn't exist:
dvr_applied = f.Events.header.get("LSTDVR", False)