Giters

honeycombio / helm-charts

Helm repository and charts for Honeycomb

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[refinery] Headless service for refinery

gjacquet opened this issue · comments

commented

Is your feature request related to a problem? Please describe.
We have noticed that all our otel collectors tend to send spans to only a single refinery instance. This typically occurs after a refinery rolling restart where otel collector tends to establish connection to a single pod, usually one of the first one that was created during the restart.

This describes the issue and the solution pretty well: https://kubernetes.io/blog/2018/11/07/grpc-load-balancing-on-kubernetes-without-tears/

Describe the solution you'd like
When using grpc to send spans to refinery, it is usually better to use a headless service for refinery and do client side load balancing in otel collector.