Support for raw pandas data?
raphaelvallat opened this issue · comments
Bonjour Gregory!
I just stumbled upon your package and it looks absolutely fantastic -- congrats on all the great work! The only thing preventing me from using it right now is that my actigraphy data is not in any of the supported format and is instead typically processed into a pandas.DataFrame with a DatetimeIndex, e.g:
| Datetime | Activity | Light | Non-wear |
|---|---|---|---|
| 01-01-2020 22:00:00 | 0 | 100 | 0 |
| 01-01-2020 22:00:30 | 10 | 120 | 0 |
| 01-01-2020 22:01:00 | 400 | 122 | 0 |
| 01-01-2020 22:01:30 | 0 | 130 | 0 |
I wonder if it would be possible and desirable to add a function to create a pyActigraphy.io.BaseRaw object directly from a raw dataframe like the one above? The frequency and start time attributes could be directly extracted from the DatetimeIndex. Other required attributes would have to be manually specified by the user as input parameters of the function.
Let me know what you think,
Thanks!
Raphael
Bonjour Raphaël.
Thank you. Such message is always appreciated.
Your suggestion is very welcome and makes sense. Under the hood, the BaseRaw class uses pandas. So it would be pretty straightforward to implement it. I am about to release v1.0 for pyActigraphy and submit the associated paper. So I am not sure if this feature will make it in time but I'll try.
In the meantime, so that you can give the package a try, I have set up a notebook that illustrates how to read data from a pandas.DataFrame with the current BaseRaw class. I will put it in the "Tutorials" section of the online doc. But I need to finish the massive update I started first.
I will let you know when I am done. Hopefully, tomorrow.
Cheers,
Grégory
PS: I just love Pingouin!
Oh, glad you like Pingouin! The Python ecosystem is a small world :-)
That sounds great, but no rush at all I'm not working with actigraphy data right now. Congrats on the v1.0 and the paper, looking forward to read it!
Cheers,
Raphael
@raphaelvallat : the v1.0rc1 is out. And I also drafted a small tutorial on how to use the present "BaseRaw" class to read data from pandas.DataFrame: https://ghammad.github.io/pyActigraphy/pyActigraphy-Pandas.html
Let me know if it suits your need.
If a full connector for DataFrame does not make it to the v1.0, it will for sure be included in the v1.1.
So no excuse not to work with actigraphy data right now.
Cheers,
G.
Amazing, thank you @ghammad !! Looks very clean, I'll try that in the upcoming days/weeks. Thanks again!
Works like a charm @ghammad 👏 !
Most of the associated methods (e.g. IS, IV) worked perfectly too. However, I'm getting an error when running raw.sleep_bouts():
Also, I think it may be useful to add some "safety checks" to make sure that data, start_time, period and freq are correctly specified. For instance, at first I was getting some obscure error message when trying to run some of the methods and I then realized that data.index.freq was set to None, which I solved with:
# Make sure that freq is not None
freq = pd.infer_freq(data.index)
data = data.asfreq(freq)Anyway, congrats again for this great package -- just saw the batch support, amazing!
Thanks,
Raphael
Was just about to come here to ask this exact question, thank you for the solution! P.s batch functions are life saving