Everything works well but super slow
davidlatwe opened this issue · comments
Just got a chance to dump 26.5K documents from mongodb to montydb and saved with flatfile storage, then kick it into production environment.
And the query is super slow, haha.
Should be related to #12.
Turns out it's not related to #12, and it wasn't the BSON encode/decode, the main reason was because I use SON() to store documents in memory (not recall why I did that, just giving it a go at the time being I guess).
montydb/montydb/storage/flatfile.py
Line 56 in 27deb4a
After I change to use OrderedDict, the speed was 50x faster at storage engine startup. That's a big difference !
But it's not enough, with 20K documents on hand, each query took about 0.8 sec to get results, this should be improved.