This project aims to be very Postgres centered, so that we can run several indexing processes at the same time. Just keep track of which process indexed what, and this should be lightning fast.
- PostgreSQL (>= 9.5) π
- NodeJS π
- Something to index π
- Patience π£
- Turn off
synchronous_commit
if you can. - Increase your usual settings like
shared_buffers
andwork_mem
. - One instance of Markov runs one Postgresql connection. Either run multiple instances or switch to node pg pooling.
- You can remove the fulltext index while inserting your data.