Specify Field Selection
sabrehagen opened this issue · comments
Hi,
Mongoose supports the ability to select specific fields when querying the database via the query.select('document fields to select') syntax. This helps when serialization becomes a bottleneck. For example, when datasets get large and only a small fraction of each document is required (e.g. 1/10th) from the database, reading the entire document can be an unnecessary cost.
I have not looked into the source of linvodb. Would it be an optimization for for linvodb to implement a select option for queries, or would this provide no performance improvement? Currently I return all documents matching a query, then map over them to extract the fields I need before returning the result. The select option would save this extra map stage, and potentially save extracting a large amount of data not needed by the user.
Thoughts?
the way linvodb3 works is that it takes the objects out of any underlying storage and then JSON.parse-es them
this means that even if you want a subset of fields the whole object would still have to be retrieved and then parsed, which defeats the performance benefits
I suggest you utilize live queries and aggregation, that way the objects will be really retrieved only once at the loading of the app and afterwards the results will only be hot-updated when necessary
Also if you are running into performance issues you should try changing your back-end store. What are you using at the moment?
Currently leveldown holding 10,000 documents ~250 chars each.