Help getting pulling stats from exporter.

[Nathan Stratton]

Running a kubernetes cluster and I am able to curl http://nvidia-gpu-exporter-1705447598.dev:9835/metrics and get data from all my nodes. I am struggling with how to get that data into grafana. I tried going to grafana and adding a new Permetheus connection, but I am not able to see any metrics.

[Utku Özdemir]

Hi, you cannot connect this exporter directly to Grafana. You need to run Prometheus in-between, which will scrape the endpoint you shared above. And Grafana will be connected to the Prometheus.

[Utku Özdemir]

Since it's a Kubernetes cluster, I can recommend looking into the kube-prometheus-stack helm chart. Setting it up and configuring is not trivial, but at the end you'll have everything you need.

[Nathan Stratton]

Added the following to flow-grafana-agent config map at it works! Thanks.

	discovery.kubernetes \"nvidiaExporterPods\" {
	  selectors {
	    label = \"app.kubernetes.io/name=nvidia-gpu-exporter\"
	    role = \"pod\"
	  }
	  role = \"pod\"
	}
	
	prometheus.scrape \"nvidiaExporterScrape\" {
	  forward_to = [
	    prometheus.remote_write.mimir.receiver,
	  ]
	  targets = discovery.kubernetes.nvidiaExporterPods.targets
	  metrics_path = \"/metrics\"
	  scrape_interval = \"10s\"
	}