how to move the last modified tuple/rows from mongodb to elasticsearch
monojitbarua opened this issue · comments
new inserted rows are coming as mentioned in your rb file( :_id => {:$gt => last_id_object}). But is there any way to get the rows/tuples from the mongodb collection which are updated after insertion.
From the README:
This was designed for parsing logs that were written into mongodb. This means that it may not re-parse db entries that were changed and already parsed.
...which, for me, makes it useless.
@monojitbarua I was stuck in the similar problem as none of the available plugins/tools fully support elasticsearch >7.0 and lacks basic features. I end up writing simple python script to ingest data from mongo to elasticsearch. Feel free to use it https://github.com/hardiksondagar/mongodb-to-elasticsearch