Roadmap for future implementations
GitHunter0 opened this issue · comments
Hey @xiaodaigh ,
Do you have a roadmap for future implementations like support for .rds files (since they are way smaller than CSV files, support more column types like lists / nested data, and are easier to work with for R users)?
Thanks for the fantastic package, keep the great work
It's already covered in here #127
For now I need to rewrite the NSE system and push test coverage to as high as possible before attempting new features. unless that feature is absolutely necessary.
Nice to know, thanks for the feedback @xiaodaigh . The most important thing actually, that I forgot to mention, is the sort methods, which are essential for time series and window functions, but I'm sure it is in your future plans too.
Sort is already implemented in arrange
@xiaodaigh , I saw arrange but it only sorts within each chunk, right? Is there a method to sort the data frame as a whole?
I mean hard_arrange
Cool! That's what I was looking for, weird that I did not find it before, thanks again @xiaodaigh
Hard arrange i s very expensive. What's ur use case?
@xiaodaigh , I'm aware it's computationally expensive, and I try to avoid it as much as possible, but some times I need to use it to apply rolling means and similar window functions
I see. Thanks for clarifying. I am thinking about introducing a feature where you can store data in separate folders and each folder is a date so it's already "sorted" that way.
That would be awesome! And ideally allow a subfolder structure, since longitudinal/panel data is also very common. In a 'panel' structure, there are two key variables (columns), the 'id' variable and the 'date/time' variable, Having both sorted in folders-subfolders schema would be perfect. Thanks for the feedback @xiaodaigh
Follow-up here: #335