Comparing raw data analyzed with PyActigraphy versus data through native program

[gauerin]

Dear Grégory,

I found myself testing how comparable are the outputs generated by two different ways: raw data analyzed with your package and the native Condor program for reading the data of ActTrust accelerometers. But I bumped into a wall, the data don't match. Both the outputs are totally different. I tried to change some parameters using the functions related to IS and IV, even so I couldn't manage to achieve the results from original Condor program. Do you know how to solve this problem? What changes or functions should I run to parse the raw data?

Thank you!

[Grégory Hammad]

Hello @gauerin

Thank you for the feedback.

May I ask you to provide me with more info, so that I can reproduce the issue:

• test data if possible (anonymised)
• results for IS, IV from Condor
• the snippet of code you used in pyActigraphy to get the results

It is always good to cross-check the results obtained by various toolboxes. Actually, I compared the results from pyActigraphy with the ones obtained with MotionWare (CamNtech, Cambridge) and they are identical (see Table 1 in the pyActigraphy paper: https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1009514#sec013)

Greg

[gauerin]

First of all, thank you for your availability.

Here is the ActTrust2 document. rawdatatesting.txt

I will also attach the full report from Condor software.
condorreportrebeca.pdf

And here is the code used:

import pyActigraphy
import plotly.graph_objects as go
import os
fpath = os.path.join(os.path.dirname(r'C:\\Users\\Avell\\Desktop\\'))
raw = pyActigraphy.io.read_raw_atr(fpath+'\\rawdatatesting.txt')
raw.name
raw.IS()
raw.IV()

[Grégory Hammad]

Hello @gauerin

Sorry for the late reply. I tried to crack open that nut but...

Let me summarise.

First, like many recordings, yours contains a fair amount of "non-wear periods" which will bias almost every measure/metric you can look at. See the plot of your recording:

Then, I thought that Condor was potentially masking these periods in an automatic fashion. I thus created two time series:

1. raw: original data (12 days)
2. raw_7D: data restricted to the first 7 days.

import pyActigraphy
file = 'rawdatatesting.txt'
raw = pyActigraphy.io.read_raw_atr(file, mode='PIM')
raw_7D = pyActigraphy.io.read_raw_atr(file, mode='PIM', period='7D')

NB: I deduced from the pdf output, part "NPCRA", that the mode "PIM" was used and therefore used it as well.

Now, with these two objects, I calculated:

1. IS with the raw data, settings identical to the original publication for IS (freq='1h', binarize=True, threshold=4)
2. IS with the raw data, modified settings (freq='1h', binarize=False)
3. IS with the raw data, modified settings (freq='1h', binarize=True, threshold=250)
4. ISm with the raw data, modified settings (freq='1h',binarize=True, threshold=250)

Resultats are:

raw.IS(freq='1h', binarize=True, threshold=4)
>>>0.18156261795743403
raw.IS(freq='1H', binarize=False)
>>>0.19199251354512323
raw.IS(freq='1h', binarize=True, threshold=250)
>>> 0.20095315421465557
raw.ISm(binarize=True, threshold=250)
>>>0.18836962306563804

Values are close to the ones obtained in your report (IS 60: 0.2053) if you keep the binarization but increase the threshold to ... 250!!!??? but not equal... I have to admit I don't get what kind of settings are used by default by Condor. Concerning the other variable, IS MÉD. CASE 2, I thought it was the ISm from Gonçalvès et al. (2014) but apparently, it is not...

This is worrying from the reproducibility point of view and, actually, a good case in favour of open-source softwares, like pyActigraphy.
Nonetheless, by tweaking the settings, one can get quite close (although it might be just pure luck here!).
Actually, what is even more worrying is the agreement for IV:

raw.IV(freq='1h', binarize=True, threshold=4)
>>>0.2761297659456433
raw.IV(freq='1h', binarize=True, threshold=250)
>>>0.3523333185942571
raw.IVm(binarize=True, threshold=250)
>>>0.32643619072479424

while your report states: IV MÉD. CASE 2 = 0.6424 and IV 60 CASE 2 = 0.6907. The only way to get near these values is to exclude the non-wear periods by restricting your recording to 7 days:

raw_7D.IV(freq='1h', binarize=True, threshold=250)
>>>0.6419380278664567

Of course, I only explored a few configurations and it might again pure luck for IV...

From my perspective, I am not worried about a potential bug in pyActigraphy as, from the mathematical point of view, the IS and IV formula are quite simple to implement. Besides, as documented in the pyActigraphy paper published in Plos Computational Biology, I validated the code against the MotionWare software.

Here I suspect that there are some additional settings applied (threshold, resampling frequency, etc) that I am not aware of... Maybe you have more clues?

Hope that helps.

Grégory

[Grégory Hammad]

Hello @gauerin

Any update on this issue?

Thank you.

[Grégory Hammad]

Hello @gauerin

Closing this issue now. Feel free to re-open it if needed.

Greg