* collected metric "kubernetes_events" was collected before with the same name and label values
one1zero1one opened this issue · comments
Daniel Radu commented
Is this a BUG REPORT or FEATURE REQUEST?:
Uncomment only one, leave it on its own line:
/kind bug
/kind feature
What happened:
Error returned to prometheus scraping target: server returned HTTP status 500 Internal Server Error
An error has occurred while serving metrics:
2 error(s) occurred:
* collected metric "kubernetes_events" { label:<name:"event_kind" value:"Ingress" > label:<name:"event_message" value:"Ingress xxx/docker-registry-cache-docker-registry-caching-proxy" > label:<name:"event_name" value:"docker-registry-cache-docker-registry-caching-proxy" > label:<name:"event_namespace" value:"xxx" > label:<name:"event_reason" value:"UPDATE" > label:<name:"event_source" value:"/nginx-ingress-controller" > label:<name:"event_subobject" value:"" > label:<name:"event_type" value:"Normal" > gauge:<value:1 > } was collected before with the same name and label values
* collected metric "kubernetes_events" { label:<name:"event_kind" value:"Ingress" > label:<name:"event_message" value:"Ingress xxx/docker-registry-cache-docker-registry-caching-proxy" > label:<name:"event_name" value:"docker-registry-cache-docker-registry-caching-proxy" > label:<name:"event_namespace" value:"xxx" > label:<name:"event_reason" value:"UPDATE" > label:<name:"event_source" value:"/nginx-ingress-controller" > label:<name:"event_subobject" value:"" > label:<name:"event_type" value:"Normal" > gauge:<value:1 > } was collected before with the same name and label values
What you expected to happen:
An healthy endpoint with the healthy metrics and the error in the logs.
How to reproduce it (as minimally and precisely as possible):
Specific to one of our production cluster, unfortunately could not reproduce.
Anything else we need to know?:
Checking the events with kubectl
we do see three events created at the same time with the same TYPE/REASON/OBJECT