podman_container_net_{input|output}_total always gives 0

[Tanki]

Describe the bug
It seems the counters of podman_container_net_*_total are broken and always give 0 while podman pod stats gives correct informations regarding network I/O totals

To Reproduce

# curl http://localhost:9882/metrics | grep -E "podman_container_net_.*_total" : 
# HELP podman_container_net_input_total Container network input in bytes.
# TYPE podman_container_net_input_total counter
podman_container_net_input_total{id="072c7faa6476",pod_id="ff470ef09cf9",pod_name="metrics"} 0
podman_container_net_input_total{id="171cd6691abb",pod_id="ff470ef09cf9",pod_name="metrics"} 0
<ABRIDGED>
# HELP podman_container_net_output_total Container network output in bytes.
# TYPE podman_container_net_output_total counter
podman_container_net_output_total{id="072c7faa6476",pod_id="ff470ef09cf9",pod_name="metrics"} 0
podman_container_net_output_total{id="171cd6691abb",pod_id="ff470ef09cf9",pod_name="metrics"} 0
<ABRIDGED>

Expected behavior
Well, obviously I think it should provide correct network measurement

Screenshots
N/A

Desktop (please complete the following information):

• Debian 12 Bookworm

# podman version
Client:       Podman Engine
Version:      4.3.1
API Version:  4.3.1
Go Version:   go1.19.8
Built:        Thu Jan  1 01:00:00 1970
OS/Arch:      linux/arm64

Additional context
latest prometheus-podman-exporter runs in a container, results were the same with previous versions

[Navid Yaghoobi]

Hi @t4nki

Is it a rootless environment ?

podman documents:

Rootless environments that use CGroups V2 are not able to report statistics about their networking usage.

In the pr it's mentioned podman pod stats is giving correct information, can you also try podman container stats ?

[Tanki]

Hi

The container runs as root in a pod

Podman container stats only prints the pod total I/O and not a total per container as you can see in the screenshot attached

I'll be glad to provide you more information if you need

Edit:
I also noticed that I have been mistaken about the readings of podman pod stats or podman stats as they were both giving the same information as podman container stats IE only pod metrics and not individual ones

[Navid Yaghoobi]

Thanks for the information @t4nki

Can u also attach the podman system info ?

[Tanki]

sure !

There you go:

host:
  arch: arm64
  buildahVersion: 1.28.2
  cgroupControllers:
  - cpuset
  - cpu
  - io
  - memory
  - hugetlb
  - pids
  - rdma
  - misc
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon_2.1.6+ds1-1_arm64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.6, commit: unknown'
  cpuUtilization:
    idlePercent: 93.92
    systemPercent: 2.46
    userPercent: 3.63
  cpus: 4
  distribution:
    codename: bookworm
    distribution: debian
    version: "12"
  eventLogger: journald
  hostname: mirage
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 6.1.0-21-arm64
  linkmode: dynamic
  logDriver: journald
  memFree: 1349550080
  memTotal: 8236691456
  networkBackend: netavark
  ociRuntime:
    name: crun
    package: crun_1.8.1-1+deb12u1_arm64
    path: /usr/bin/crun
    version: |-
      crun version 1.8.1
      commit: f8a096be060b22ccd3d5f3ebe44108517fbf6c30
      rundir: /run/user/0/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL
  os: linux
  remoteSocket:
    exists: true
    path: /run/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: ""
    package: ""
    version: ""
  swapFree: 0
  swapTotal: 0
  uptime: 636h 27m 33.00s (Approximately 26.50 days)
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  volume:
  - local
registries: {}
store:
  configFile: /usr/share/containers/storage.conf
  containerStore:
    number: 19
    paused: 0
    running: 19
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /var/lib/containers/storage
  graphRootAllocated: 251583135744
  graphRootUsed: 21703700480
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 18
  runRoot: /run/containers/storage
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 4.3.1
  Built: 0
  BuiltTime: Thu Jan  1 01:00:00 1970
  GitCommit: ""
  GoVersion: go1.19.8
  Os: linux
  OsArch: linux/arm64
  Version: 4.3.1

[Tanki]

HI,

I've been thinking, wouldn't it be more convenient to have metrics like podman_pod_net_input_total and podman_pod_net_output_total available?

That's just an idea, as it looks like podman does not compute container's network I/O when it belongs to a pod

[Navid Yaghoobi]

Hi @t4nki

Its a good idea , I will look into it.

Regards

[Navid Yaghoobi]

Hi @t4nki
unfortunately for podman v4.x.y you need you can use exporter version <= 1.10.X.

I've updated the README with the information.

Regards