Reduce processed dataset size
gflegar opened this issue · comments
To avoid loading unnecessarily large datasets, another JSONata script could be added to preprocess each file before it is loaded. This way, a single huge JSON object is never formed in memory.
It is probably best to implement this issue once #1 is complete.