How to organize datastores

[Something Something]

Hi, really loving your project so far.

I have a question about organizing datastore files, this is my current workflow:

1. read websocket data, flush to file every 1000 rows
2. when file hits 100 thousand rows (this is just a limit I use, can be changed), start with a new file.
3. buffer the incoming data when switching to the new file
4. start writing to the new file.
5. put the old filename into a database for example mongodb, along with timestamp of starting and ending time.

When later on I want to find data within a time frame, I will search the mongodb to get the correct datastore file.

I'm wondering if this is the best way to do this?
Thank you!