optiopay / kafka

Go driver for Kafka

Home Page:https://godoc.org/github.com/optiopay/kafka

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Abstract away connection error handling

husio opened this issue · comments

Right now, all top level broker/producer/consumer functions are handling network errors on it's own, but almost the same way. This has several negative consequences and code duplication is one of the obvious, but another issue is that all of them call muCloseDeadConnection which in addition to removing socket from cache, calls refreshMetadata.
If many consumers/producers are communicating to single node, they use the same connection. If this connection fails, they most likely call one after another muCloseDeadConnection, closing connection for each other and refreshing cluster metadata every time.

Finally, testing broken connections is hard. New code must be written the way that any network issue can be simulated.