M10 and file cleaning
roryos1 opened this issue · comments
Hi @ghammad,
Just a couple of questions about the M10 and file cleaning.
I am aware that the binarized M10 data shows the average proportion of time spent active per hour (during the M10 hours), however I was wondering what the non-binarized M10 data might indicate? Is there somewhere I can access the formula you used to calculate the variable?
On another note, I have been playing around with the following code used to set start times and periods of data collection:
raw_cropped = pyActigraphy.io.read_raw_awd(
fpath+'example_01.AWD',
start_time='1918-01-24 08:00:00',
period='9 days')
I have found that when data collection runs from one month into the next (as in the example above, 24th Jan + 9 day period = 1st Feb), I receive a keyerror highlighting last day of data collection (KeyError: Timestamp('1918-02-01 12:00:00'), for example above). This has happened across a couple of data files. Thought this might be worth bringing to your attention, unless I'm somehow using this incorrectly.
Hi,
After some serious exploring I managed to figure out how the M10 is calculated, for anybody else who stumbles upon this.
What happens is that the activity counts for each epoch (epochs making up most active 10 hours of the day) are averaged out against each other, this means that the M10 is the average number of activity counts per epoch.
For example, let's say you only have 10 epochs (equalling 1 hour each) in your M10.
Epoch 1 - 140 counts
Epoch 2 - 280 counts
Epoch 3 - 260 counts
Epoch 4 - 790 counts
Epoch 5 - 390 counts
Epoch 6 - 270 counts
Epoch 7 - 980 counts
Epoch 8 - 720 counts
Epoch 9 - 100 counts
Epoch 10 - 360 counts
(140+280+260+790+390+270+980+720+100+360)/10 = 429
Therefore M10 = 429, or 429 average activity counts per hour (epoch).
Hello @roryos1
thanks for reporting your observations. May I just ask you to do so in 2 separated issues next times if there are two issues to discuss? Easier to answer, maintain and index for the future.
Concerning the M10, all the info (I hope) are present in the online doc: https://ghammad.github.io/pyActigraphy/_autosummary/pyActigraphy.metrics.MetricsMixin.M10.html#pyActigraphy.metrics.MetricsMixin.M10
It is indeed the mean activity level during the most active 10h period. When you binarize the data, you can then interpret in number of active epochs within your period (60 epochs of 1min in 1h for example).
Concerning the issue about the month cross-over, could you provide me with an example(+version you use) so that I can reproduce the issue?
When I use the following code:
raw = pyActigraphy.io.read_raw_awd(
fpath+'example_01.AWD',
start_time='1918-01-24 08:00:00',
period='9 days'
)I do not get any error.
Best regards,
Grégory
Hi @ghammad,
Sorry about that, noted. I believe I didn't read correctly into 'The M10 variable is calculated as the mean, per acquisition period, of the average daily activities during the 10 most active hours.' I see now that the 'acquisition period' refers to each epoch.
Further apologies as I did not send the right code over last time, the actual code I've been having difficulty with is specific to csv files:
raw_cropped = pyActigraphy.io.read_raw_rpx( fpath+'act1.csv', start_time='2020-08-26 00:00:00', period='6 days')
I have sent the data that I've been using with this code to you via email, hopefully this will help you recreate the error. I'm operating on Python 3.8.5.
Best regards,
Rory
Update on this. I've been playing around with the code and found no error when I used the following:
raw = pyActigraphy.io.read_raw_rpx(fpath+'act1.csv', language='ENG_UK', start_time=’2020-08-26 00:00:00’ , period=’6 days’ )
It seems that entering the 'start time' and 'period' in the same command that initially identifies the actigraphy file circumvents the error. No idea why but it works.
Hello @roryos1
Thanks for sending me your file. In fact, the problem is not related to month cross-over but to the date time format.
The data time reported in the Respironics file format depends on your local computer settings (as well as the language used in the .rpx file). In North America, the format MM-DD-YYYY is used while in other part of the worlds, the format DD-MM-YYYY seems preferred (although I'm sure there are plenty of exceptions I am not aware of...). Since it is hard to infer programmatically which format is used (01-01-2022 for example is valid in both formats), it is to the user to decide which one to use.
Therefore, for files extracted from a Respironics' device with a "UK" computer, one needs to set
language='ENG_UK'in the reader function. Just like you did in your example.
More info about the available "languages" in the online documentation:
https://ghammad.github.io/pyActigraphy/_autosummary/pyActigraphy.io.read_raw_rpx.html#pyActigraphy.io.read_raw_rpx
Hope that helps.
Grégory
Brilliant, thank you very much for the help.