prometheus / node_exporter

Exporter for machine metrics

Home Page:https://prometheus.io/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Node Exporter confusing 2 partitions

hermann-osi opened this issue · comments

Host operating system: output of uname -a

Linux amberlyst 5.10.0-26-amd64 #1 SMP Debian 5.10.197-1 (2023-09-29) x86_64 GNU/Linux

node_exporter version: output of node_exporter --version

Docker, latest, here is my configuration :

  node-exporter:
    image: quay.io/prometheus/node-exporter:latest
    volumes:
      - /proc:/host/proc:ro
      - /sys:/host/sys:ro
      - /:/rootfs:ro
      - /data:/data:ro
      - /data2:/data2:ro
      - /:/data3:ro
    command:
      - '--path.procfs=/host/proc'
      - '--path.sysfs=/host/sys'
      - '--collector.filesystem.ignored-mount-points=^/(sys|proc|dev|host|etc|home/ftpuser/video_folder)($$|/)'
    networks:
      - monitor-net
    ports:
      - '127.0.0.1:9100:9100'
    restart: always

node_exporter command line flags

node_exporter log output

Nothing weird in the logs but here are they :

grafana-node-exporter-1  | ts=2024-01-06T22:28:50.572Z caller=node_exporter.go:192 level=info msg="Starting node_exporter" version="(version=1.7.0, branch=HEAD, revision=7333465abf9efba81876303bb57e6fadb946041b)"
grafana-node-exporter-1  | ts=2024-01-06T22:28:50.572Z caller=node_exporter.go:193 level=info msg="Build context" build_context="(go=go1.21.4, platform=linux/amd64, user=root@35918982f6d8, date=20231112-23:53:35, tags=netgo osusergo static_build)"
grafana-node-exporter-1  | ts=2024-01-06T22:28:50.573Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$
grafana-node-exporter-1  | ts=2024-01-06T22:28:50.573Z caller=diskstats_linux.go:265 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data
grafana-node-exporter-1  | ts=2024-01-06T22:28:50.573Z caller=filesystem_common.go:94 level=warn collector=filesystem msg="--collector.filesystem.ignored-mount-points is DEPRECATED and will be removed in 2.0.0, use --collector.filesystem.mount-points-exclude"
grafana-node-exporter-1  | ts=2024-01-06T22:28:50.573Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(sys|proc|dev|host|etc|home/ftpuser/video_folder)($|/)
grafana-node-exporter-1  | ts=2024-01-06T22:28:50.574Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$

Are you running node_exporter in Docker?

Yes

What did you do that produced an error?

I added the following request in grafana to obtain the disk usage of / :
1 - (sum(node_filesystem_avail_bytes{mountpoint=~"/"}) / sum(node_filesystem_size_bytes{mountpoint=~"/"}))
1 - (sum(node_filesystem_avail_bytes{device=~"/dev/sdb3"}) / sum(node_filesystem_size_bytes{device=~"/dev/sdb3"}))
Same result with both : 77% of usage shown (exactly the same as /dev/sda1 and exactly the same maximum size)
df -h returns 30% of usage.

What did you expect to see?

30% of usage and 10 time less maximum size.

What did you see instead?

The exact same data as another partition in another disk of 1TB.

I know docker is not advised for node-exporter but it's strange to see that being off. Everything else is working fine, my other 2 partitions are correctly detected with their correct data. It's only the main partition that seem to be confused with /dev/sda1.
Does anyone know what could cause this behaviour ?

Thank you in advance for looking at my post.

Please read the docker documentation in the README.

https://github.com/prometheus/node_exporter#docker

For questions/help/support please use our community channels. There are more people available to potentially respond to your request and the whole community can benefit from the answers provided.