hpgrahsl / kafka-connect-mongodb

**Unofficial / Community** Kafka Connect MongoDB Sink Connector -> integrated 2019 into the official MongoDB Kafka Connector here: https://www.mongodb.com/kafka-connector

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Use two WriteModel Strategies in one config?

kush1509 opened this issue · comments

Hi,

I wanted to ask if it is possible to use two WriteModel Strategies in one config. My use-case is that I need to use the ReplaceOneBusinessKeyStrategy to check for my business key and I want to add timestamps as well so I need the UpdateOneTimestampsStrategy. I am not entirely sure if my understanding of these is correct, but I would like to understand why (if so) we cannot chain such strategies.

Hi,

Thx for reaching out. The original design to allow for only one WriteModelStrategy has been decided on purpose since in general combining/chaining several such strategies often doesn't make sense. But you are right, that for the case you describe it would be reasonable to have both these strategies applied when writing to the DB. That being said, the idea here for now is to implement your own custom strategy and make it usable within the connector. The only inconvenience in this regard is that the configuration doesn't allow you to specify your own one directly - as you could do e.g. for the IdStrategy - which means until this becomes available you would need to fork the repo and adapt it in this regard. But it's rather straight forward and the "merged" strategy you are talking about results from copying together the code from the other two and giving it a creative name :-)

Also you might want to take a look at the official MongoDB connector which is available here: https://github.com/mongodb/mongo-kafka If you want raise a feature request for this there.

@kush1509 wanted to check back if my reply was in any way helpful for you? just let me know if there is anything unclear. I'm happy to answer :)

@hpgrahsl yes thanks for the answer. I did find a strategy that fit my purpose - UpdateOneBusinessKeyTimestampStrategy that does the two things but you're right, I can also write a custom strategy for more complicated use cases. Thanks!

THX @kush1509 for reporting back. Glad to hear your found a combined strategy serving your needs. I'm closing this then.

UpdateOneBusinessKeyTimestampStrategy do not do upsert, do we have an alternative in this moment ?