PSA: Migrating to Postgresql
EricTheMagician opened this issue · comments
For anyone actively using DriveFS, it will be migrated to postgres from mongodb in the coming weeks.
Interesting. Why so?
Is it for performance?
Thanks a lot for all the effort you are putting into this Project.
It was originally motivated by performance and memory issues.
MongoDB on my system was a huge memory hog, and that was something I wanted to tackle by switching databases.
The other was to reduce overall memory usage, especially for very large amounts files/folders by not having all the required information in memory, and falling back to the database, when needed.
As for performance, I did expect a small loss compared to having the data stored in memory, but it's quite significant. I primarily use it through rclone and I have noticed it hanging from time to time with ls. By hanging, it would last a few seconds, sometimes alot more, on the order of a minute / did I code this incorrectly moments. fixed, performs reasonably well.
Would this work with stateless container if everything is in database now and not in memory anymore? As in if I run multiple times the program connecting to the same database it would work?
I am running it in a cluster to mount the drive in multiple places and currently have multiple databases so would be nice to cut on that :)
It's still not stateless as some temporary states are still being stored. I still have some occasional crashes, so I want to fix those before pushing it to the master branch.
I'm currently traveling for work, and so I won't have too much time to work on this until august.
So stay tuned.
Hi, any updates on that one? :)