bug: --collector.store_labels causing empty collection
db-wally007 opened this issue · comments
Describe the bug
Upgraded to 1.80 version (on redhat 9.3) and when I launch exporter with the new flag --collector.store_labels - there arent any metrics exported.
● prometheus-podman-exporter.service - Prometheus exporter for podman (v4) machine
Loaded: loaded (/etc/systemd/system/prometheus-podman-exporter.service; enabled; preset: disabled)
Active: active (running) since Tue 2024-02-27 13:37:26 UTC; 14s ago
Main PID: 139424 (prometheus-podm)
Tasks: 14 (limit: 48830)
Memory: 21.9M
CPU: 76ms
CGroup: /system.slice/prometheus-podman-exporter.service
└─139424 /usr/bin/prometheus-podman-exporter --collector.store_labels --collector.enable-all --web.listen-address 127.0.0.1:9882
Feb 27 13:37:26 vlado1 prometheus-podman-exporter[139424]: ts=2024-02-27T13:37:26.309Z caller=handler.go:93 level=info msg="enabled collectors"
Feb 27 13:37:26 vlado1 prometheus-podman-exporter[139424]: ts=2024-02-27T13:37:26.309Z caller=handler.go:104 level=info collector=container
Feb 27 13:37:26 vlado1 prometheus-podman-exporter[139424]: ts=2024-02-27T13:37:26.309Z caller=handler.go:104 level=info collector=image
Feb 27 13:37:26 vlado1 prometheus-podman-exporter[139424]: ts=2024-02-27T13:37:26.309Z caller=handler.go:104 level=info collector=network
Feb 27 13:37:26 vlado1 prometheus-podman-exporter[139424]: ts=2024-02-27T13:37:26.309Z caller=handler.go:104 level=info collector=pod
Feb 27 13:37:26 vlado1 prometheus-podman-exporter[139424]: ts=2024-02-27T13:37:26.309Z caller=handler.go:104 level=info collector=system
Feb 27 13:37:26 vlado1 prometheus-podman-exporter[139424]: ts=2024-02-27T13:37:26.309Z caller=handler.go:104 level=info collector=volume
Feb 27 13:37:26 vlado1 prometheus-podman-exporter[139424]: ts=2024-02-27T13:37:26.328Z caller=exporter.go:82 level=info msg="Listening on" address=127.0.0.1:9882
Feb 27 13:37:26 vlado1 prometheus-podman-exporter[139424]: ts=2024-02-27T13:37:26.328Z caller=tls_config.go:313 level=info msg="Listening on" address=127.0.0.1:9882
Feb 27 13:37:26 vlado1 prometheus-podman-exporter[139424]: ts=2024-02-27T13:37:26.329Z caller=tls_config.go:316 level=info msg="TLS is disabled." http2=false address=127.0.0.1:9882
results in
root@vlado1:/>curl http://localhost:9882/metrics
curl: (52) Empty reply from server
root@vlado1:/>
when I remove the new flag --collector.store_labels
root@vlado1:/>vim /etc/systemd/system/prometheus-podman-exporter.service
root@vlado1:/>systemctl daemon-reload
root@vlado1:/>systemctl restart prometheus-podman-exporter
root@vlado1:/>systemctl status prometheus-podman-exporter
● prometheus-podman-exporter.service - Prometheus exporter for podman (v4) machine
Loaded: loaded (/etc/systemd/system/prometheus-podman-exporter.service; enabled; preset: disabled)
Active: active (running) since Tue 2024-02-27 13:39:02 UTC; 7s ago
Main PID: 139658 (prometheus-podm)
Tasks: 14 (limit: 48830)
Memory: 22.0M
CPU: 51ms
CGroup: /system.slice/prometheus-podman-exporter.service
└─139658 /usr/bin/prometheus-podman-exporter --collector.enable-all --web.listen-address 127.0.0.1:9882
Feb 27 13:39:02 vlado1 prometheus-podman-exporter[139658]: ts=2024-02-27T13:39:02.892Z caller=handler.go:93 level=info msg="enabled collectors"
Feb 27 13:39:02 vlado1 prometheus-podman-exporter[139658]: ts=2024-02-27T13:39:02.892Z caller=handler.go:104 level=info collector=container
Feb 27 13:39:02 vlado1 prometheus-podman-exporter[139658]: ts=2024-02-27T13:39:02.892Z caller=handler.go:104 level=info collector=image
Feb 27 13:39:02 vlado1 prometheus-podman-exporter[139658]: ts=2024-02-27T13:39:02.892Z caller=handler.go:104 level=info collector=network
Feb 27 13:39:02 vlado1 prometheus-podman-exporter[139658]: ts=2024-02-27T13:39:02.892Z caller=handler.go:104 level=info collector=pod
Feb 27 13:39:02 vlado1 prometheus-podman-exporter[139658]: ts=2024-02-27T13:39:02.892Z caller=handler.go:104 level=info collector=system
Feb 27 13:39:02 vlado1 prometheus-podman-exporter[139658]: ts=2024-02-27T13:39:02.892Z caller=handler.go:104 level=info collector=volume
Feb 27 13:39:02 vlado1 prometheus-podman-exporter[139658]: ts=2024-02-27T13:39:02.908Z caller=exporter.go:82 level=info msg="Listening on" address=127.0.0.1:9882
Feb 27 13:39:02 vlado1 prometheus-podman-exporter[139658]: ts=2024-02-27T13:39:02.908Z caller=tls_config.go:313 level=info msg="Listening on" address=127.0.0.1:9882
Feb 27 13:39:02 vlado1 prometheus-podman-exporter[139658]: ts=2024-02-27T13:39:02.908Z caller=tls_config.go:316 level=info msg="TLS is disabled." http2=false address=127.0.0.1:9882
root@vlado1:/>curl http://localhost:9882/metrics
# HELP go_gc_duration_seconds A summary of the pause duration of garbage collection cycles.
# TYPE go_gc_duration_seconds summary
go_gc_duration_seconds{quantile="0"} 5.7077e-05
go_gc_duration_seconds{quantile="0.25"} 5.87e-05
go_gc_duration_seconds{quantile="0.5"} 0.000106128
go_gc_duration_seconds{quantile="0.75"} 0.00020925
go_gc_duration_seconds{quantile="1"} 0.00020925
go_gc_duration_seconds_sum 0.000431155
...
Hi @db-wally007
Is it possible to run start the exporter manually in debug mode via:
/usr/bin/prometheus-podman-exporter --collector.enable-all --collector.store_labels --debug
and then fetch the metrics and send me the output of exporter logs.
root@vlado1:~>/usr/bin/prometheus-podman-exporter --collector.enable-all --collector.store_labels --web.listen-address "127.0.0.1:9882" --debug
ts=2024-02-28T08:27:52.594Z caller=exporter.go:65 level=info msg="Starting podman-prometheus-exporter" version="(version=1.8.0, branch=main, revision=1.el9)"
ts=2024-02-28T08:27:52.594Z caller=handler.go:93 level=info msg="enabled collectors"
ts=2024-02-28T08:27:52.594Z caller=handler.go:104 level=info collector=container
ts=2024-02-28T08:27:52.594Z caller=handler.go:104 level=info collector=image
ts=2024-02-28T08:27:52.594Z caller=handler.go:104 level=info collector=network
ts=2024-02-28T08:27:52.594Z caller=handler.go:104 level=info collector=pod
ts=2024-02-28T08:27:52.594Z caller=handler.go:104 level=info collector=system
ts=2024-02-28T08:27:52.594Z caller=handler.go:104 level=info collector=volume
ts=2024-02-28T08:27:52.600Z caller=events.go:17 level=debug msg="starting podman event streamer"
ts=2024-02-28T08:27:52.600Z caller=events.go:18 level=debug msg="update images"
ts=2024-02-28T08:27:52.614Z caller=exporter.go:82 level=info msg="Listening on" address=127.0.0.1:9882
ts=2024-02-28T08:27:52.615Z caller=tls_config.go:313 level=info msg="Listening on" address=127.0.0.1:9882
ts=2024-02-28T08:27:52.615Z caller=tls_config.go:316 level=info msg="TLS is disabled." http2=false address=127.0.0.1:9882
ts=2024-02-28T08:28:25.951Z caller=handler.go:34 level=debug msg="collect query:" filters=[]
ts=2024-02-28T08:28:25.952Z caller=collector.go:135 level=debug msg="collector succeeded" name=volume duration_seconds=0.000385168
ts=2024-02-28T08:28:25.952Z caller=collector.go:135 level=debug msg="collector succeeded" name=image duration_seconds=0.000469646
ts=2024-02-28T08:28:25.952Z caller=collector.go:135 level=debug msg="collector succeeded" name=network duration_seconds=0.000999003
ts=2024-02-28T08:28:25.962Z caller=collector.go:135 level=debug msg="collector succeeded" name=pod duration_seconds=0.01012683
panic: duplicate label names in constant and variable labels for metric "podman_container_info"
goroutine 74 [running]:
panic({0x562eb2c798e0, 0xc000571050})
/usr/lib/golang/src/runtime/panic.go:987 +0x3bb fp=0xc0006cfb78 sp=0xc0006cfab8 pc=0x562eb163da5b
github.com/prometheus/client_golang/prometheus.MustNewConstMetric(...)
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/vendor/github.com/prometheus/client_golang/prometheus/value.go:129
github.com/containers/prometheus-podman-exporter/collector.(*typedDesc).mustNewConstMetric(0xc000288d40?, 0xc?, {0xc00077c9c0?, 0xc?, 0xc0003f6a00?})
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/collector/collector.go:152 +0x54 fp=0xc0006cfbb8 sp=0xc0006cfb78 pc=0x562eb26e2ef4
github.com/containers/prometheus-podman-exporter/collector.(*containerCollector).Update(0xc00071e480, 0x0?)
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/collector/container.go:166 +0x229 fp=0xc0006cfe20 sp=0xc0006cfbb8 pc=0x562eb26e4149
github.com/containers/prometheus-podman-exporter/collector.execute({0x562eb2823e10, 0x9}, {0x562eb2ec0ce0, 0xc00071e480}, 0x0?, {0x562eb2ec1160, 0xc000220240})
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/collector/collector.go:115 +0x99 fp=0xc0006cff60 sp=0xc0006cfe20 pc=0x562eb26e2539
github.com/containers/prometheus-podman-exporter/collector.PodmanCollector.Collect.func1({0x562eb2823e10?, 0x0?}, {0x562eb2ec0ce0?, 0xc00071e480?})
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/collector/collector.go:103 +0x3d fp=0xc0006cffb0 sp=0xc0006cff60 pc=0x562eb26e243d
github.com/containers/prometheus-podman-exporter/collector.PodmanCollector.Collect.func2()
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/collector/collector.go:105 +0x36 fp=0xc0006cffe0 sp=0xc0006cffb0 pc=0x562eb26e23d6
runtime.goexit()
/usr/lib/golang/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0006cffe8 sp=0xc0006cffe0 pc=0x562eb16729a1
created by github.com/containers/prometheus-podman-exporter/collector.PodmanCollector.Collect
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/collector/collector.go:102 +0xd0
goroutine 1 [semacquire]:
runtime.gopark(0xc0000440e0?, 0x562eb2ece2f0?, 0xa0?, 0x51?, 0x562eb281ece9?)
/usr/lib/golang/src/runtime/proc.go:381 +0xd6 fp=0xc000325908 sp=0xc0003258e8 pc=0x562eb1640cd6
runtime.goparkunlock(...)
/usr/lib/golang/src/runtime/proc.go:387
runtime.semacquire1(0xc000221110, 0x18?, 0x1, 0x0, 0x71?)
/usr/lib/golang/src/runtime/sema.go:160 +0x214 fp=0xc000325970 sp=0xc000325908 pc=0x562eb1651d14
sync.runtime_Semacquire(0xc00041ae10?)
/usr/lib/golang/src/runtime/sema.go:62 +0x27 fp=0xc0003259a8 sp=0xc000325970 pc=0x562eb166e827
sync.(*WaitGroup).Wait(0x562eb2dd27e0?)
/usr/lib/golang/src/sync/waitgroup.go:116 +0x4b fp=0xc0003259d0 sp=0xc0003259a8 pc=0x562eb168f96b
golang.org/x/sync/errgroup.(*Group).Wait(0xc000221100)
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/vendor/golang.org/x/sync/errgroup/errgroup.go:56 +0x27 fp=0xc0003259f0 sp=0xc0003259d0 pc=0x562eb20a9927
github.com/prometheus/exporter-toolkit/web.ServeMultiple({0xc00011e520, 0x1, 0x7ffc0efc35ce?}, 0xc00059f0e0, 0xc00041adc8, {0x562eb2ec1160?, 0xc000220240})
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/vendor/github.com/prometheus/exporter-toolkit/web/tls_config.go:274 +0x151 fp=0xc000325a40 sp=0xc0003259f0 pc=0x562eb27104b1
github.com/prometheus/exporter-toolkit/web.ListenAndServe(0xc000246190?, 0xc00041adc8, {0x562eb2ec1160, 0xc000220240})
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/vendor/github.com/prometheus/exporter-toolkit/web/tls_config.go:307 +0x488 fp=0xc000325b18 sp=0xc000325a40 pc=0x562eb2710a08
github.com/containers/prometheus-podman-exporter/exporter.Start(0x0?, {0x0?, 0x0?, 0x0?})
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/exporter/exporter.go:95 +0x869 fp=0xc000325ca0 sp=0xc000325b18 pc=0x562eb2729629
github.com/containers/prometheus-podman-exporter/cmd.run(0x562eb3936ca0?, {0xc0007304b0?, 0x5?, 0x5?})
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/cmd/root.go:52 +0x1e fp=0xc000325ce0 sp=0xc000325ca0 pc=0x562eb272b55e
github.com/spf13/cobra.(*Command).execute(0x562eb3936ca0, {0xc0000401f0, 0x5, 0x5})
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/vendor/github.com/spf13/cobra/command.go:987 +0xa91 fp=0xc000325e80 sp=0xc000325ce0 pc=0x562eb22e3cd1
github.com/spf13/cobra.(*Command).ExecuteC(0x562eb3936ca0)
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/vendor/github.com/spf13/cobra/command.go:1115 +0x425 fp=0xc000325f58 sp=0xc000325e80 pc=0x562eb22e4605
github.com/spf13/cobra.(*Command).Execute(...)
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/vendor/github.com/spf13/cobra/command.go:1039
github.com/containers/prometheus-podman-exporter/cmd.Execute()
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/cmd/root.go:60 +0x25 fp=0xc000325f70 sp=0xc000325f58 pc=0x562eb272b625
main.main()
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/main.go:8 +0x17 fp=0xc000325f80 sp=0xc000325f70 pc=0x562eb272bb57
runtime.main()
/usr/lib/golang/src/runtime/proc.go:250 +0x212 fp=0xc000325fe0 sp=0xc000325f80 pc=0x562eb1640892
runtime.goexit()
/usr/lib/golang/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000325fe8 sp=0xc000325fe0 pc=0x562eb16729a1
goroutine 2 [force gc (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
/usr/lib/golang/src/runtime/proc.go:381 +0xd6 fp=0xc000076fb0 sp=0xc000076f90 pc=0x562eb1640cd6
runtime.goparkunlock(...)
/usr/lib/golang/src/runtime/proc.go:387
runtime.forcegchelper()
/usr/lib/golang/src/runtime/proc.go:305 +0xb0 fp=0xc000076fe0 sp=0xc000076fb0 pc=0x562eb1640b10
runtime.goexit()
/usr/lib/golang/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000076fe8 sp=0xc000076fe0 pc=0x562eb16729a1
created by runtime.init.7
/usr/lib/golang/src/runtime/proc.go:293 +0x25
goroutine 3 [GC sweep wait]:
runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?)
/usr/lib/golang/src/runtime/proc.go:381 +0xd6 fp=0xc000077780 sp=0xc000077760 pc=0x562eb1640cd6
runtime.goparkunlock(...)
/usr/lib/golang/src/runtime/proc.go:387
runtime.bgsweep(0x0?)
/usr/lib/golang/src/runtime/mgcsweep.go:319 +0xde fp=0xc0000777c8 sp=0xc000077780 pc=0x562eb162bc9e
runtime.gcenable.func1()
/usr/lib/golang/src/runtime/mgc.go:178 +0x26 fp=0xc0000777e0 sp=0xc0000777c8 pc=0x562eb1620f06
runtime.goexit()
/usr/lib/golang/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000777e8 sp=0xc0000777e0 pc=0x562eb16729a1
created by runtime.gcenable
/usr/lib/golang/src/runtime/mgc.go:178 +0x6b
goroutine 4 [GC scavenge wait]:
runtime.gopark(0x8d6b9fd25159?, 0x11cea8?, 0x0?, 0x0?, 0x0?)
/usr/lib/golang/src/runtime/proc.go:381 +0xd6 fp=0xc000077f70 sp=0xc000077f50 pc=0x562eb1640cd6
runtime.goparkunlock(...)
/usr/lib/golang/src/runtime/proc.go:387
runtime.(*scavengerState).park(0x562eb3a0c880)
/usr/lib/golang/src/runtime/mgcscavenge.go:400 +0x53 fp=0xc000077fa0 sp=0xc000077f70 pc=0x562eb1629b73
runtime.bgscavenge(0x0?)
/usr/lib/golang/src/runtime/mgcscavenge.go:633 +0x65 fp=0xc000077fc8 sp=0xc000077fa0 pc=0x562eb162a165
runtime.gcenable.func2()
/usr/lib/golang/src/runtime/mgc.go:179 +0x26 fp=0xc000077fe0 sp=0xc000077fc8 pc=0x562eb1620ea6
runtime.goexit()
/usr/lib/golang/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000077fe8 sp=0xc000077fe0 pc=0x562eb16729a1
created by runtime.gcenable
/usr/lib/golang/src/runtime/mgc.go:179 +0xaa
goroutine 5 [finalizer wait]:
runtime.gopark(0x0?, 0x562eb2eba8e8?, 0x0?, 0xa0?, 0x2000000020?)
/usr/lib/golang/src/runtime/proc.go:381 +0xd6 fp=0xc000076628 sp=0xc000076608 pc=0x562eb1640cd6
runtime.runfinq()
/usr/lib/golang/src/runtime/mfinal.go:193 +0x107 fp=0xc0000767e0 sp=0xc000076628 pc=0x562eb161fea7
runtime.goexit()
/usr/lib/golang/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000767e8 sp=0xc0000767e0 pc=0x562eb16729a1
created by runtime.createfing
/usr/lib/golang/src/runtime/mfinal.go:163 +0x45
goroutine 18 [GC worker (idle)]:
runtime.gopark(0x562eb1a5d955?, 0x562eb160b9fd?, 0x0?, 0x6?, 0xc0000787a8?)
/usr/lib/golang/src/runtime/proc.go:381 +0xd6 fp=0xc000078750 sp=0xc000078730 pc=0x562eb1640cd6
runtime.gcBgMarkWorker()
/usr/lib/golang/src/runtime/mgc.go:1275 +0xf1 fp=0xc0000787e0 sp=0xc000078750 pc=0x562eb1622c71
runtime.goexit()
/usr/lib/golang/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000787e8 sp=0xc0000787e0 pc=0x562eb16729a1
created by runtime.gcBgMarkStartWorkers
/usr/lib/golang/src/runtime/mgc.go:1199 +0x25
goroutine 19 [GC worker (idle)]:
runtime.gopark(0x8d73640774d2?, 0x3?, 0xf8?, 0x63?, 0x0?)
/usr/lib/golang/src/runtime/proc.go:381 +0xd6 fp=0xc000072750 sp=0xc000072730 pc=0x562eb1640cd6
runtime.gcBgMarkWorker()
/usr/lib/golang/src/runtime/mgc.go:1275 +0xf1 fp=0xc0000727e0 sp=0xc000072750 pc=0x562eb1622c71
runtime.goexit()
/usr/lib/golang/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000727e8 sp=0xc0000727e0 pc=0x562eb16729a1
created by runtime.gcBgMarkStartWorkers
/usr/lib/golang/src/runtime/mgc.go:1199 +0x25
goroutine 20 [GC worker (idle)]:
runtime.gopark(0x8d7364074dff?, 0xc000050aa0?, 0x1a?, 0x14?, 0x0?)
/usr/lib/golang/src/runtime/proc.go:381 +0xd6 fp=0xc000072f50 sp=0xc000072f30 pc=0x562eb1640cd6
runtime.gcBgMarkWorker()
/usr/lib/golang/src/runtime/mgc.go:1275 +0xf1 fp=0xc000072fe0 sp=0xc000072f50 pc=0x562eb1622c71
runtime.goexit()
/usr/lib/golang/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000072fe8 sp=0xc000072fe0 pc=0x562eb16729a1
created by runtime.gcBgMarkStartWorkers
/usr/lib/golang/src/runtime/mgc.go:1199 +0x25
goroutine 8 [GC worker (idle)]:
runtime.gopark(0x562eb3a40b40?, 0x1?, 0xca?, 0x80?, 0x0?)
/usr/lib/golang/src/runtime/proc.go:381 +0xd6 fp=0xc000078f50 sp=0xc000078f30 pc=0x562eb1640cd6
runtime.gcBgMarkWorker()
/usr/lib/golang/src/runtime/mgc.go:1275 +0xf1 fp=0xc000078fe0 sp=0xc000078f50 pc=0x562eb1622c71
runtime.goexit()
/usr/lib/golang/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000078fe8 sp=0xc000078fe0 pc=0x562eb16729a1
created by runtime.gcBgMarkStartWorkers
/usr/lib/golang/src/runtime/mgc.go:1199 +0x25
goroutine 9 [GC worker (idle)]:
runtime.gopark(0x8d7364074115?, 0x3?, 0x9?, 0xa5?, 0x0?)
/usr/lib/golang/src/runtime/proc.go:381 +0xd6 fp=0xc000079750 sp=0xc000079730 pc=0x562eb1640cd6
runtime.gcBgMarkWorker()
/usr/lib/golang/src/runtime/mgc.go:1275 +0xf1 fp=0xc0000797e0 sp=0xc000079750 pc=0x562eb1622c71
runtime.goexit()
/usr/lib/golang/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000797e8 sp=0xc0000797e0 pc=0x562eb16729a1
created by runtime.gcBgMarkStartWorkers
/usr/lib/golang/src/runtime/mgc.go:1199 +0x25
goroutine 34 [GC worker (idle)]:
runtime.gopark(0x8d7364071609?, 0x3?, 0x40?, 0xd?, 0x0?)
/usr/lib/golang/src/runtime/proc.go:381 +0xd6 fp=0xc000116750 sp=0xc000116730 pc=0x562eb1640cd6
runtime.gcBgMarkWorker()
/usr/lib/golang/src/runtime/mgc.go:1275 +0xf1 fp=0xc0001167e0 sp=0xc000116750 pc=0x562eb1622c71
runtime.goexit()
/usr/lib/golang/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0001167e8 sp=0xc0001167e0 pc=0x562eb16729a1
created by runtime.gcBgMarkStartWorkers
/usr/lib/golang/src/runtime/mgc.go:1199 +0x25
goroutine 21 [GC worker (idle)]:
runtime.gopark(0x562eb3a40b40?, 0x1?, 0x60?, 0xe0?, 0x0?)
/usr/lib/golang/src/runtime/proc.go:381 +0xd6 fp=0xc000073750 sp=0xc000073730 pc=0x562eb1640cd6
runtime.gcBgMarkWorker()
/usr/lib/golang/src/runtime/mgc.go:1275 +0xf1 fp=0xc0000737e0 sp=0xc000073750 pc=0x562eb1622c71
runtime.goexit()
/usr/lib/golang/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000737e8 sp=0xc0000737e0 pc=0x562eb16729a1
created by runtime.gcBgMarkStartWorkers
/usr/lib/golang/src/runtime/mgc.go:1199 +0x25
goroutine 10 [GC worker (idle)]:
runtime.gopark(0x8d7364074206?, 0x3?, 0x7c?, 0x29?, 0x0?)
/usr/lib/golang/src/runtime/proc.go:381 +0xd6 fp=0xc000079f50 sp=0xc000079f30 pc=0x562eb1640cd6
runtime.gcBgMarkWorker()
/usr/lib/golang/src/runtime/mgc.go:1275 +0xf1 fp=0xc000079fe0 sp=0xc000079f50 pc=0x562eb1622c71
runtime.goexit()
/usr/lib/golang/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000079fe8 sp=0xc000079fe0 pc=0x562eb16729a1
created by runtime.gcBgMarkStartWorkers
/usr/lib/golang/src/runtime/mgc.go:1199 +0x25
goroutine 13 [select, locked to thread]:
runtime.gopark(0xc0001187a8?, 0x2?, 0x52?, 0x10?, 0xc0001187a4?)
/usr/lib/golang/src/runtime/proc.go:381 +0xd6 fp=0xc000118618 sp=0xc0001185f8 pc=0x562eb1640cd6
runtime.selectgo(0xc0001187a8, 0xc0001187a0, 0x0?, 0x0, 0x0?, 0x1)
/usr/lib/golang/src/runtime/select.go:327 +0x7be fp=0xc000118758 sp=0xc000118618 pc=0x562eb1650c1e
runtime.ensureSigM.func1()
/usr/lib/golang/src/runtime/signal_unix.go:1004 +0x1b4 fp=0xc0001187e0 sp=0xc000118758 pc=0x562eb16690d4
runtime.goexit()
/usr/lib/golang/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0001187e8 sp=0xc0001187e0 pc=0x562eb16729a1
created by runtime.ensureSigM
/usr/lib/golang/src/runtime/signal_unix.go:987 +0xbd
goroutine 52 [syscall]:
runtime.notetsleepg(0x0?, 0x0?)
/usr/lib/golang/src/runtime/lock_futex.go:236 +0x34 fp=0xc000113fa0 sp=0xc000113f68 pc=0x562eb1612474
os/signal.signal_recv()
/usr/lib/golang/src/runtime/sigqueue.go:152 +0x2f fp=0xc000113fc0 sp=0xc000113fa0 pc=0x562eb166f04f
os/signal.loop()
/usr/lib/golang/src/os/signal/signal_unix.go:23 +0x19 fp=0xc000113fe0 sp=0xc000113fc0 pc=0x562eb1aa5cb9
runtime.goexit()
/usr/lib/golang/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000113fe8 sp=0xc000113fe0 pc=0x562eb16729a1
created by os/signal.Notify.func1.1
/usr/lib/golang/src/os/signal/signal.go:151 +0x2a
goroutine 53 [select]:
runtime.gopark(0xc0001147b0?, 0x2?, 0x0?, 0x0?, 0xc0001146ac?)
/usr/lib/golang/src/runtime/proc.go:381 +0xd6 fp=0xc000114518 sp=0xc0001144f8 pc=0x562eb1640cd6
runtime.selectgo(0xc0001147b0, 0xc0001146a8, 0x0?, 0x0, 0x0?, 0x1)
/usr/lib/golang/src/runtime/select.go:327 +0x7be fp=0xc000114658 sp=0xc000114518 pc=0x562eb1650c1e
github.com/containers/podman/v4/libpod/shutdown.Start.func1()
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/vendor/github.com/containers/podman/v4/libpod/shutdown/handler.go:48 +0x93 fp=0xc0001147e0 sp=0xc000114658 pc=0x562eb24a3613
runtime.goexit()
/usr/lib/golang/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0001147e8 sp=0xc0001147e0 pc=0x562eb16729a1
created by github.com/containers/podman/v4/libpod/shutdown.Start
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/vendor/github.com/containers/podman/v4/libpod/shutdown/handler.go:47 +0xe7
goroutine 54 [select]:
runtime.gopark(0xc00008bee8?, 0x2?, 0xd1?, 0x12?, 0xc00008bea4?)
/usr/lib/golang/src/runtime/proc.go:381 +0xd6 fp=0xc00008bc88 sp=0xc00008bc68 pc=0x562eb1640cd6
runtime.selectgo(0xc00008bee8, 0xc00008bea0, 0x0?, 0x0, 0x2?, 0x1)
/usr/lib/golang/src/runtime/select.go:327 +0x7be fp=0xc00008bdc8 sp=0xc00008bc88 pc=0x562eb1650c1e
github.com/containers/podman/v4/libpod.(*Runtime).libimageEvents.func2()
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/vendor/github.com/containers/podman/v4/libpod/runtime.go:749 +0xe9 fp=0xc00008bfe0 sp=0xc00008bdc8 pc=0x562eb25f9e89
runtime.goexit()
/usr/lib/golang/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00008bfe8 sp=0xc00008bfe0 pc=0x562eb16729a1
created by github.com/containers/podman/v4/libpod.(*Runtime).libimageEvents
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/vendor/github.com/containers/podman/v4/libpod/runtime.go:725 +0x10d
goroutine 55 [chan receive]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
/usr/lib/golang/src/runtime/proc.go:381 +0xd6 fp=0xc0001156f0 sp=0xc0001156d0 pc=0x562eb1640cd6
runtime.chanrecv(0xc000714ae0, 0xc0001157c8, 0x1)
/usr/lib/golang/src/runtime/chan.go:583 +0x49d fp=0xc000115780 sp=0xc0001156f0 pc=0x562eb160cc5d
runtime.chanrecv2(0x0?, 0x0?)
/usr/lib/golang/src/runtime/chan.go:447 +0x18 fp=0xc0001157a8 sp=0xc000115780 pc=0x562eb160c798
github.com/containers/podman/v4/libpod.(*Runtime).startWorker.func1()
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/vendor/github.com/containers/podman/v4/libpod/runtime_worker.go:9 +0x74 fp=0xc0001157e0 sp=0xc0001157a8 pc=0x562eb2616834
runtime.goexit()
/usr/lib/golang/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0001157e8 sp=0xc0001157e0 pc=0x562eb16729a1
created by github.com/containers/podman/v4/libpod.(*Runtime).startWorker
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/vendor/github.com/containers/podman/v4/libpod/runtime_worker.go:8 +0x96
goroutine 14 [syscall]:
runtime.cgocall(0x562eb272e680, 0xc00008aac8)
/usr/lib/golang/src/runtime/cgocall.go:157 +0x5c fp=0xc00008aaa0 sp=0xc00008aa68 pc=0x562eb1609e5c
github.com/coreos/go-systemd/v22/sdjournal._Cfunc_my_sd_journal_wait(0x7fce0824f1b0, 0x7fcdcc003bb0, 0x4c4b40)
_cgo_gotypes.go:607 +0x50 fp=0xc00008aac8 sp=0xc00008aaa0 pc=0x562eb22745b0
github.com/coreos/go-systemd/v22/sdjournal.(*Journal).Wait.func1(0x562eb282e0f1?, 0xf?, 0xc00008ab48?)
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/vendor/github.com/coreos/go-systemd/v22/sdjournal/journal.go:1032 +0x7c fp=0xc00008ab10 sp=0xc00008aac8 pc=0x562eb2276e3c
github.com/coreos/go-systemd/v22/sdjournal.(*Journal).Wait(0xc0004b4240, 0x12a05f200)
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/vendor/github.com/coreos/go-systemd/v22/sdjournal/journal.go:1032 +0xce fp=0xc00008ab58 sp=0xc00008ab10 pc=0x562eb2276d4e
github.com/containers/podman/v4/libpod/events.GetNextEntry({0x562eb2ece2f0, 0xc0000440e0}, 0x6?, 0x1, {0x0?, 0x562eb286107f?, 0x0?})
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/vendor/github.com/containers/podman/v4/libpod/events/journal_linux.go:277 +0x69 fp=0xc00008abf8 sp=0xc00008ab58 pc=0x562eb2281dc9
github.com/containers/podman/v4/libpod/events.EventJournalD.Read({{{0x562eb282286f, 0x8}, {0xc00011a400, 0x1d}, 0xf4240}}, {0x562eb2ece2f0, 0xc0000440e0}, {0xc000715020, {0x562eb3a3ec78, 0x0, ...}, ...})
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/vendor/github.com/containers/podman/v4/libpod/events/journal_linux.go:155 +0x765 fp=0xc00008adc8 sp=0xc00008abf8 pc=0x562eb2281285
github.com/containers/podman/v4/libpod/events.(*EventJournalD).Read(0x0?, {0x562eb2ece2f0?, 0xc0000440e0?}, {0xc000715020, {0x562eb3a3ec78, 0x0, 0x0}, 0x0, {0x0, 0x0}, ...})
<autogenerated>:1 +0xcc fp=0xc00008ae88 sp=0xc00008adc8 pc=0x562eb22853cc
github.com/containers/podman/v4/libpod.(*Runtime).Events(...)
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/vendor/github.com/containers/podman/v4/libpod/events.go:170
github.com/containers/podman/v4/pkg/domain/infra/abi.(*ContainerEngine).Events(0x0?, {0x562eb2ece2f0?, 0xc0000440e0?}, {0x0, 0xc000715020, {0x562eb3a3ec78, 0x0, 0x0}, 0x1, {0x0, ...}, ...})
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/vendor/github.com/containers/podman/v4/pkg/domain/infra/abi/events.go:12 +0x11a fp=0xc00008af50 sp=0xc00008ae88 pc=0x562eb26ac31a
github.com/containers/prometheus-podman-exporter/pdcs.StartEventStreamer.func1()
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/pdcs/events.go:28 +0x7d fp=0xc00008afe0 sp=0xc00008af50 pc=0x562eb26df3dd
runtime.goexit()
/usr/lib/golang/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00008afe8 sp=0xc00008afe0 pc=0x562eb16729a1
created by github.com/containers/prometheus-podman-exporter/pdcs.StartEventStreamer
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/pdcs/events.go:27 +0x298
goroutine 15 [select]:
runtime.gopark(0xc000117790?, 0x2?, 0x0?, 0x0?, 0xc00011773c?)
/usr/lib/golang/src/runtime/proc.go:381 +0xd6 fp=0xc0001175b0 sp=0xc000117590 pc=0x562eb1640cd6
runtime.selectgo(0xc000117790, 0xc000117738, 0x0?, 0x0, 0x0?, 0x1)
/usr/lib/golang/src/runtime/select.go:327 +0x7be fp=0xc0001176f0 sp=0xc0001175b0 pc=0x562eb1650c1e
github.com/containers/prometheus-podman-exporter/pdcs.StartEventStreamer.func2()
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/pdcs/events.go:36 +0xc8 fp=0xc0001177e0 sp=0xc0001176f0 pc=0x562eb26df108
runtime.goexit()
/usr/lib/golang/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0001177e8 sp=0xc0001177e0 pc=0x562eb16729a1
created by github.com/containers/prometheus-podman-exporter/pdcs.StartEventStreamer
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/pdcs/events.go:34 +0x31e
goroutine 16 [IO wait]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
/usr/lib/golang/src/runtime/proc.go:381 +0xd6 fp=0xc0000fca78 sp=0xc0000fca58 pc=0x562eb1640cd6
runtime.netpollblock(0x7fce084f57a8?, 0xb16094ef?, 0x2e?)
/usr/lib/golang/src/runtime/netpoll.go:527 +0xf7 fp=0xc0000fcab0 sp=0xc0000fca78 pc=0x562eb1639377
internal/poll.runtime_pollWait(0x7fce08552798, 0x72)
/usr/lib/golang/src/runtime/netpoll.go:306 +0x89 fp=0xc0000fcad0 sp=0xc0000fcab0 pc=0x562eb166c829
internal/poll.(*pollDesc).wait(0xc0005be600?, 0x4?, 0x0)
/usr/lib/golang/src/internal/poll/fd_poll_runtime.go:84 +0x32 fp=0xc0000fcaf8 sp=0xc0000fcad0 pc=0x562eb16e76b2
internal/poll.(*pollDesc).waitRead(...)
/usr/lib/golang/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Accept(0xc0005be600)
/usr/lib/golang/src/internal/poll/fd_unix.go:614 +0x2bd fp=0xc0000fcba0 sp=0xc0000fcaf8 pc=0x562eb16ecfbd
net.(*netFD).accept(0xc0005be600)
/usr/lib/golang/src/net/fd_unix.go:172 +0x35 fp=0xc0000fcc58 sp=0xc0000fcba0 pc=0x562eb1814135
net.(*TCPListener).accept(0xc00041ade0)
/usr/lib/golang/src/net/tcpsock_posix.go:148 +0x25 fp=0xc0000fcc80 sp=0xc0000fcc58 pc=0x562eb182f725
net.(*TCPListener).Accept(0xc00041ade0)
/usr/lib/golang/src/net/tcpsock.go:297 +0x3d fp=0xc0000fccb0 sp=0xc0000fcc80 pc=0x562eb182e4dd
net/http.(*onceCloseListener).Accept(0xc00039d950?)
<autogenerated>:1 +0x2a fp=0xc0000fccc8 sp=0xc0000fccb0 pc=0x562eb195e8ca
net/http.(*Server).Serve(0xc00059f0e0, {0x562eb2ecc960, 0xc00041ade0})
/usr/lib/golang/src/net/http/server.go:3059 +0x385 fp=0xc0000fcdf8 sp=0xc0000fccc8 pc=0x562eb1938ce5
github.com/prometheus/exporter-toolkit/web.Serve({0x562eb2ecc960, 0xc00041ade0}, 0xc00059f0e0, 0xc00041adc8, {0x562eb2ec1160?, 0xc000220240})
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/vendor/github.com/prometheus/exporter-toolkit/web/tls_config.go:317 +0x35f fp=0xc0000fcf38 sp=0xc0000fcdf8 pc=0x562eb2710e3f
github.com/prometheus/exporter-toolkit/web.ServeMultiple.func1()
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/vendor/github.com/prometheus/exporter-toolkit/web/tls_config.go:271 +0x31 fp=0xc0000fcf78 sp=0xc0000fcf38 pc=0x562eb2710551
golang.org/x/sync/errgroup.(*Group).Go.func1()
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/vendor/golang.org/x/sync/errgroup/errgroup.go:78 +0x64 fp=0xc0000fcfe0 sp=0xc0000fcf78 pc=0x562eb20a9ac4
runtime.goexit()
/usr/lib/golang/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000fcfe8 sp=0xc0000fcfe0 pc=0x562eb16729a1
created by golang.org/x/sync/errgroup.(*Group).Go
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/vendor/golang.org/x/sync/errgroup/errgroup.go:75 +0xa5
goroutine 66 [select]:
runtime.gopark(0xc000132eb0?, 0x2?, 0x18?, 0x42?, 0xc000132ccc?)
/usr/lib/golang/src/runtime/proc.go:381 +0xd6 fp=0xc000132b08 sp=0xc000132ae8 pc=0x562eb1640cd6
runtime.selectgo(0xc000132eb0, 0xc000132cc8, 0x0?, 0x0, 0x0?, 0x1)
/usr/lib/golang/src/runtime/select.go:327 +0x7be fp=0xc000132c48 sp=0xc000132b08 pc=0x562eb1650c1e
github.com/prometheus/client_golang/prometheus.(*Registry).Gather(0xc000730c30)
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/vendor/github.com/prometheus/client_golang/prometheus/registry.go:522 +0x9f1 fp=0xc000133128 sp=0xc000132c48 pc=0x562eb1a62e11
github.com/prometheus/client_golang/prometheus.Gatherers.Gather({0xc00021b000, 0x2, 0xc000226000?})
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/vendor/github.com/prometheus/client_golang/prometheus/registry.go:753 +0x166 fp=0xc000133490 sp=0xc000133128 pc=0x562eb1a64486
github.com/prometheus/client_golang/prometheus.(*Gatherers).Gather(0x562eb281b09f?)
<autogenerated>:1 +0x38 fp=0xc0001334b8 sp=0xc000133490 pc=0x562eb1a6da38
github.com/prometheus/client_golang/prometheus.(*noTransactionGatherer).Gather(0x0?)
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/vendor/github.com/prometheus/client_golang/prometheus/registry.go:1073 +0x22 fp=0xc0001334d0 sp=0xc0001334b8 pc=0x562eb1a66c62
github.com/prometheus/client_golang/prometheus/promhttp.HandlerForTransactional.func1({0x7fce082e70a8, 0xc0003fda90}, 0xc00072cb00)
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/vendor/github.com/prometheus/client_golang/prometheus/promhttp/http.go:140 +0x2b0 fp=0xc0001337b0 sp=0xc0001334d0 pc=0x562eb2713cb0
net/http.HandlerFunc.ServeHTTP(0x562eb2cfd660?, {0x7fce082e70a8?, 0xc0003fda90?}, 0x562eb2b3d1eb?)
/usr/lib/golang/src/net/http/server.go:2122 +0x2f fp=0xc0001337d8 sp=0xc0001337b0 pc=0x562eb193562f
github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerInFlight.func1({0x7fce082e70a8, 0xc0003fda90}, 0x562eb2eccb00?)
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/vendor/github.com/prometheus/client_golang/prometheus/promhttp/instrument_server.go:60 +0xd4 fp=0xc000133858 sp=0xc0001337d8 pc=0x562eb27150d4
net/http.HandlerFunc.ServeHTTP(0x562eb2eccb70?, {0x7fce082e70a8?, 0xc0003fda90?}, 0x562eb2c74180?)
/usr/lib/golang/src/net/http/server.go:2122 +0x2f fp=0xc000133880 sp=0xc000133858 pc=0x562eb193562f
github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerCounter.func1({0x562eb2eccb70?, 0xc00038f0a0?}, 0xc00072cb00)
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/vendor/github.com/prometheus/client_golang/prometheus/promhttp/instrument_server.go:147 +0xc5 fp=0xc000133990 sp=0xc000133880 pc=0x562eb27158a5
net/http.HandlerFunc.ServeHTTP(0xc0003fda40?, {0x562eb2eccb70?, 0xc00038f0a0?}, 0x1?)
/usr/lib/golang/src/net/http/server.go:2122 +0x2f fp=0xc0001339b8 sp=0xc000133990 pc=0x562eb193562f
github.com/containers/prometheus-podman-exporter/exporter.(*handler).ServeHTTP(0xc000220380, {0x562eb2eccb70, 0xc00038f0a0}, 0xc00072cb00)
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/exporter/handler.go:38 +0x456 fp=0xc000133a80 sp=0xc0001339b8 pc=0x562eb272a3d6
net/http.(*ServeMux).ServeHTTP(0x0?, {0x562eb2eccb70, 0xc00038f0a0}, 0xc00072cb00)
/usr/lib/golang/src/net/http/server.go:2500 +0x149 fp=0xc000133ad0 sp=0xc000133a80 pc=0x562eb1936fc9
net/http.serverHandler.ServeHTTP({0xc0001a5da0?}, {0x562eb2eccb70, 0xc00038f0a0}, 0xc00072cb00)
/usr/lib/golang/src/net/http/server.go:2936 +0x316 fp=0xc000133b80 sp=0xc000133ad0 pc=0x562eb1938776
net/http.(*conn).serve(0xc00039d950, {0x562eb2ece360, 0xc0001a5cb0})
/usr/lib/golang/src/net/http/server.go:1995 +0x612 fp=0xc000133fb8 sp=0xc000133b80 pc=0x562eb1934152
net/http.(*Server).Serve.func3()
/usr/lib/golang/src/net/http/server.go:3089 +0x2e fp=0xc000133fe0 sp=0xc000133fb8 pc=0x562eb19390ce
runtime.goexit()
/usr/lib/golang/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000133fe8 sp=0xc000133fe0 pc=0x562eb16729a1
created by net/http.(*Server).Serve
/usr/lib/golang/src/net/http/server.go:3089 +0x5ed
goroutine 67 [IO wait]:
runtime.gopark(0x10?, 0xb?, 0x0?, 0x0?, 0x8?)
/usr/lib/golang/src/runtime/proc.go:381 +0xd6 fp=0xc000117da0 sp=0xc000117d80 pc=0x562eb1640cd6
runtime.netpollblock(0x562eb16ca085?, 0xb16094ef?, 0x2e?)
/usr/lib/golang/src/runtime/netpoll.go:527 +0xf7 fp=0xc000117dd8 sp=0xc000117da0 pc=0x562eb1639377
internal/poll.runtime_pollWait(0x7fce085526a8, 0x72)
/usr/lib/golang/src/runtime/netpoll.go:306 +0x89 fp=0xc000117df8 sp=0xc000117dd8 pc=0x562eb166c829
internal/poll.(*pollDesc).wait(0xc0005be780?, 0xc0001a5db1?, 0x0)
/usr/lib/golang/src/internal/poll/fd_poll_runtime.go:84 +0x32 fp=0xc000117e20 sp=0xc000117df8 pc=0x562eb16e76b2
internal/poll.(*pollDesc).waitRead(...)
/usr/lib/golang/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc0005be780, {0xc0001a5db1, 0x1, 0x1})
/usr/lib/golang/src/internal/poll/fd_unix.go:167 +0x299 fp=0xc000117eb8 sp=0xc000117e20 pc=0x562eb16e8a99
net.(*netFD).Read(0xc0005be780, {0xc0001a5db1?, 0x562eb391f090?, 0x562eb2cbbc00?})
/usr/lib/golang/src/net/fd_posix.go:55 +0x29 fp=0xc000117f00 sp=0xc000117eb8 pc=0x562eb1811fa9
net.(*conn).Read(0xc000718098, {0xc0001a5db1?, 0x562eb2ec1160?, 0xc000220240?})
/usr/lib/golang/src/net/net.go:183 +0x45 fp=0xc000117f48 sp=0xc000117f00 pc=0x562eb1825da5
net.(*TCPConn).Read(0x0?, {0xc0001a5db1?, 0x100000000000000?, 0xc000221100?})
<autogenerated>:1 +0x29 fp=0xc000117f78 sp=0xc000117f48 pc=0x562eb183ac49
net/http.(*connReader).backgroundRead(0xc0001a5da0)
/usr/lib/golang/src/net/http/server.go:674 +0x3f fp=0xc000117fc8 sp=0xc000117f78 pc=0x562eb192de7f
net/http.(*connReader).startBackgroundRead.func2()
/usr/lib/golang/src/net/http/server.go:670 +0x26 fp=0xc000117fe0 sp=0xc000117fc8 pc=0x562eb192dda6
runtime.goexit()
/usr/lib/golang/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000117fe8 sp=0xc000117fe0 pc=0x562eb16729a1
created by net/http.(*connReader).startBackgroundRead
/usr/lib/golang/src/net/http/server.go:670 +0xca
goroutine 72 [semacquire]:
runtime.gopark(0x562eb1a6dd8c?, 0xc0007305f0?, 0x0?, 0xa3?, 0xc000074f20?)
/usr/lib/golang/src/runtime/proc.go:381 +0xd6 fp=0xc000074ef0 sp=0xc000074ed0 pc=0x562eb1640cd6
runtime.goparkunlock(...)
/usr/lib/golang/src/runtime/proc.go:387
runtime.semacquire1(0xc000045118, 0x0?, 0x1, 0x0, 0x20?)
/usr/lib/golang/src/runtime/sema.go:160 +0x214 fp=0xc000074f58 sp=0xc000074ef0 pc=0x562eb1651d14
sync.runtime_Semacquire(0x0?)
/usr/lib/golang/src/runtime/sema.go:62 +0x27 fp=0xc000074f90 sp=0xc000074f58 pc=0x562eb166e827
sync.(*WaitGroup).Wait(0xc000074f90?)
/usr/lib/golang/src/sync/waitgroup.go:116 +0x4b fp=0xc000074fb8 sp=0xc000074f90 pc=0x562eb168f96b
github.com/prometheus/client_golang/prometheus.(*Registry).Gather.func2()
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/vendor/github.com/prometheus/client_golang/prometheus/registry.go:472 +0x2f fp=0xc000074fe0 sp=0xc000074fb8 pc=0x562eb1a632cf
runtime.goexit()
/usr/lib/golang/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000074fe8 sp=0xc000074fe0 pc=0x562eb16729a1
created by github.com/prometheus/client_golang/prometheus.(*Registry).Gather
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/vendor/github.com/prometheus/client_golang/prometheus/registry.go:471 +0x61b
goroutine 73 [semacquire]:
runtime.gopark(0x0?, 0x0?, 0xe0?, 0x10?, 0x0?)
/usr/lib/golang/src/runtime/proc.go:381 +0xd6 fp=0xc000075568 sp=0xc000075548 pc=0x562eb1640cd6
runtime.goparkunlock(...)
/usr/lib/golang/src/runtime/proc.go:387
runtime.semacquire1(0xc000045128, 0x30?, 0x1, 0x0, 0x71?)
/usr/lib/golang/src/runtime/sema.go:160 +0x214 fp=0xc0000755d0 sp=0xc000075568 pc=0x562eb1651d14
sync.runtime_Semacquire(0xc000630720?)
/usr/lib/golang/src/runtime/sema.go:62 +0x27 fp=0xc000075608 sp=0xc0000755d0 pc=0x562eb166e827
sync.(*WaitGroup).Wait(0xc000075690?)
/usr/lib/golang/src/sync/waitgroup.go:116 +0x4b fp=0xc000075630 sp=0xc000075608 pc=0x562eb168f96b
github.com/containers/prometheus-podman-exporter/collector.PodmanCollector.Collect({0xc000120840?, {0x562eb2ec1160?, 0xc000220240?}}, 0xc000027800)
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/collector/collector.go:108 +0x24f fp=0xc000075700 sp=0xc000075630 pc=0x562eb26e234f
github.com/containers/prometheus-podman-exporter/collector.(*PodmanCollector).Collect(0xc0000757b0?, 0xc000075760?)
<autogenerated>:1 +0x3e fp=0xc000075730 sp=0xc000075700 pc=0x562eb26e8f1e
github.com/prometheus/client_golang/prometheus.(*Registry).Gather.func1()
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/vendor/github.com/prometheus/client_golang/prometheus/registry.go:455 +0x10d fp=0xc0000757e0 sp=0xc000075730 pc=0x562eb1a6340d
runtime.goexit()
/usr/lib/golang/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000757e8 sp=0xc0000757e0 pc=0x562eb16729a1
created by github.com/prometheus/client_golang/prometheus.(*Registry).Gather
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/vendor/github.com/prometheus/client_golang/prometheus/registry.go:547 +0xc09
goroutine 78 [chan receive]:
runtime.gopark(0x0?, 0xc000320cc0?, 0xab?, 0xca?, 0x562eb3a37338?)
/usr/lib/golang/src/runtime/proc.go:381 +0xd6 fp=0xc000320c78 sp=0xc000320c58 pc=0x562eb1640cd6
runtime.chanrecv(0xc0004404e0, 0xc000320d98, 0x1)
/usr/lib/golang/src/runtime/chan.go:583 +0x49d fp=0xc000320d08 sp=0xc000320c78 pc=0x562eb160cc5d
runtime.chanrecv1(0x562eb2d82860?, 0x10000000000?)
/usr/lib/golang/src/runtime/chan.go:442 +0x18 fp=0xc000320d30 sp=0xc000320d08 pc=0x562eb160c758
os/exec.(*Cmd).awaitGoroutines(0xc0002dc000, 0x0)
/usr/lib/golang/src/os/exec/exec.go:941 +0x233 fp=0xc000320e08 sp=0xc000320d30 pc=0x562eb1a70f73
os/exec.(*Cmd).Wait(0xc0002dc000)
/usr/lib/golang/src/os/exec/exec.go:908 +0x175 fp=0xc000320e70 sp=0xc000320e08 pc=0x562eb1a70bf5
os/exec.(*Cmd).Run(0x4?)
/usr/lib/golang/src/os/exec/exec.go:590 +0x39 fp=0xc000320e90 sp=0xc000320e70 pc=0x562eb1a6f5d9
os/exec.(*Cmd).Output(0xc0002dc000)
/usr/lib/golang/src/os/exec/exec.go:984 +0xc7 fp=0xc000320ee0 sp=0xc000320e90 pc=0x562eb1a71227
github.com/containers/common/pkg/version.queryPackageVersion({0xc0004402a0, 0x20?, 0x30?})
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/vendor/github.com/containers/common/pkg/version/version.go:21 +0x75 fp=0xc000320fb0 sp=0xc000320ee0 pc=0x562eb23aa195
github.com/containers/common/pkg/version.Package({0xc00011a900, 0x20})
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/vendor/github.com/containers/common/pkg/version/version.go:69 +0x4fd fp=0xc000321200 sp=0xc000320fb0 pc=0x562eb23aa95d
github.com/containers/common/libnetwork/netavark.(*netavarkNetwork).NetworkInfo(0xc000234100)
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/vendor/github.com/containers/common/libnetwork/netavark/network.go:357 +0x197 fp=0xc000321368 sp=0xc000321200 pc=0x562eb242e577
github.com/containers/podman/v4/libpod.(*Runtime).hostInfo(0xc00017c1e0)
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/vendor/github.com/containers/podman/v4/libpod/info.go:129 +0x323 fp=0xc0003218d0 sp=0xc000321368 pc=0x562eb25abf83
github.com/containers/podman/v4/libpod.(*Runtime).info(0xc00017c1e0)
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/vendor/github.com/containers/podman/v4/libpod/info.go:40 +0x1c5 fp=0xc000321b70 sp=0xc0003218d0 pc=0x562eb25ab605
github.com/containers/podman/v4/libpod.(*Runtime).Info(...)
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/vendor/github.com/containers/podman/v4/libpod/runtime.go:888
github.com/containers/podman/v4/pkg/domain/infra/abi.(*ContainerEngine).Info(0xc000014748, {0x0?, 0x0?})
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/vendor/github.com/containers/podman/v4/pkg/domain/infra/abi/system.go:27 +0x3c fp=0xc000321c80 sp=0xc000321b70 pc=0x562eb26d3efc
github.com/containers/prometheus-podman-exporter/pdcs.SystemInfo()
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/pdcs/system.go:26 +0x9e fp=0xc000321d40 sp=0xc000321c80 pc=0x562eb26e09fe
github.com/containers/prometheus-podman-exporter/collector.(*systemCollector).Update(0xc000730960, 0x0?)
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/collector/system.go:58 +0x38 fp=0xc000321e20 sp=0xc000321d40 pc=0x562eb26e80b8
github.com/containers/prometheus-podman-exporter/collector.execute({0x562eb281f4c9, 0x6}, {0x562eb2ec0d60, 0xc000730960}, 0x0?, {0x562eb2ec1160, 0xc000220240})
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/collector/collector.go:115 +0x99 fp=0xc000321f60 sp=0xc000321e20 pc=0x562eb26e2539
github.com/containers/prometheus-podman-exporter/collector.PodmanCollector.Collect.func1({0x562eb281f4c9?, 0x0?}, {0x562eb2ec0d60?, 0xc000730960?})
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/collector/collector.go:103 +0x3d fp=0xc000321fb0 sp=0xc000321f60 pc=0x562eb26e243d
github.com/containers/prometheus-podman-exporter/collector.PodmanCollector.Collect.func2()
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/collector/collector.go:105 +0x36 fp=0xc000321fe0 sp=0xc000321fb0 pc=0x562eb26e23d6
runtime.goexit()
/usr/lib/golang/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000321fe8 sp=0xc000321fe0 pc=0x562eb16729a1
created by github.com/containers/prometheus-podman-exporter/collector.PodmanCollector.Collect
/builddir/build/BUILD/prometheus-podman-exporter-1.8.0/collector/collector.go:102 +0xd0
goroutine 114 [runnable]:
runtime.gopark(0x7fce0858c288?, 0xb?, 0x0?, 0x0?, 0x28?)
/usr/lib/golang/src/runtime/proc.go:381 +0xd6 fp=0xc000165d08 sp=0xc000165ce8 pc=0x562eb1640cd6
runtime.netpollblock(0x562eb16ca085?, 0xb16094ef?, 0x2e?)
/usr/lib/golang/src/runtime/netpoll.go:527 +0xf7 fp=0xc000165d40 sp=0xc000165d08 pc=0x562eb1639377
internal/poll.runtime_pollWait(0x7fce085524c8, 0x72)
/usr/lib/golang/src/runtime/netpoll.go:306 +0x89 fp=0xc000165d60 sp=0xc000165d40 pc=0x562eb166c829
internal/poll.(*pollDesc).wait(0xc000440420?, 0xc000500000?, 0x1)
/usr/lib/golang/src/internal/poll/fd_poll_runtime.go:84 +0x32 fp=0xc000165d88 sp=0xc000165d60 pc=0x562eb16e76b2
internal/poll.(*pollDesc).waitRead(...)
/usr/lib/golang/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc000440420, {0xc000500000, 0x8000, 0x8000})
/usr/lib/golang/src/internal/poll/fd_unix.go:167 +0x299 fp=0xc000165e20 sp=0xc000165d88 pc=0x562eb16e8a99
os.(*File).read(...)
/usr/lib/golang/src/os/file_posix.go:31
os.(*File).Read(0xc00022c038, {0xc000500000?, 0xc000380050?, 0x0?})
/usr/lib/golang/src/os/file.go:118 +0x5e fp=0xc000165e78 sp=0xc000165e20 pc=0x562eb16f487e
io.copyBuffer({0x562eb2ec3fa0, 0xc000380050}, {0x562eb2ec3ea0, 0xc00022c038}, {0x0, 0x0, 0x0})
/usr/lib/golang/src/io/io.go:427 +0x1b2 fp=0xc000165ef8 sp=0xc000165e78 pc=0x562eb16bd672
io.Copy(...)
/usr/lib/golang/src/io/io.go:386
os/exec.(*Cmd).writerDescriptor.func1()
/usr/lib/golang/src/os/exec/exec.go:560 +0x3a fp=0xc000165f58 sp=0xc000165ef8 pc=0x562eb1a6f55a
os/exec.(*Cmd).Start.func2(0xc00071e480?)
/usr/lib/golang/src/os/exec/exec.go:717 +0x32 fp=0xc000165fc8 sp=0xc000165f58 pc=0x562eb1a702b2
os/exec.(*Cmd).Start.func3()
/usr/lib/golang/src/os/exec/exec.go:729 +0x2a fp=0xc000165fe0 sp=0xc000165fc8 pc=0x562eb1a7024a
runtime.goexit()
/usr/lib/golang/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000165fe8 sp=0xc000165fe0 pc=0x562eb16729a1
created by os/exec.(*Cmd).Start
/usr/lib/golang/src/os/exec/exec.go:716 +0xa33
goroutine 81 [runnable]:
runtime.gopark(0x0?, 0xb?, 0x0?, 0x0?, 0x26?)
/usr/lib/golang/src/runtime/proc.go:381 +0xd6 fp=0xc0001134b0 sp=0xc000113490 pc=0x562eb1640cd6
runtime.netpollblock(0x562eb16ca085?, 0xb16094ef?, 0x2e?)
/usr/lib/golang/src/runtime/netpoll.go:527 +0xf7 fp=0xc0001134e8 sp=0xc0001134b0 pc=0x562eb1639377
internal/poll.runtime_pollWait(0x7fce085522e8, 0x72)
/usr/lib/golang/src/runtime/netpoll.go:306 +0x89 fp=0xc000113508 sp=0xc0001134e8 pc=0x562eb166c829
internal/poll.(*pollDesc).wait(0xc000440360?, 0xc0003be420?, 0x1)
/usr/lib/golang/src/internal/poll/fd_poll_runtime.go:84 +0x32 fp=0xc000113530 sp=0xc000113508 pc=0x562eb16e76b2
internal/poll.(*pollDesc).waitRead(...)
/usr/lib/golang/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc000440360, {0xc0003be420, 0x3e0, 0x3e0})
/usr/lib/golang/src/internal/poll/fd_unix.go:167 +0x299 fp=0xc0001135c8 sp=0xc000113530 pc=0x562eb16e8a99
os.(*File).read(...)
/usr/lib/golang/src/os/file_posix.go:31
os.(*File).Read(0xc00022c018, {0xc0003be420?, 0xc000113668?, 0xc000113668?})
/usr/lib/golang/src/os/file.go:118 +0x5e fp=0xc000113620 sp=0xc0001135c8 pc=0x562eb16f487e
bytes.(*Buffer).ReadFrom(0xc0002f8060, {0x562eb2ec3ea0, 0xc00022c018})
/usr/lib/golang/src/bytes/buffer.go:202 +0x98 fp=0xc000113678 sp=0xc000113620 pc=0x562eb1712cf8
io.copyBuffer({0x562eb2ebffa0, 0xc0002f8060}, {0x562eb2ec3ea0, 0xc00022c018}, {0x0, 0x0, 0x0})
/usr/lib/golang/src/io/io.go:413 +0x14b fp=0xc0001136f8 sp=0xc000113678 pc=0x562eb16bd60b
io.Copy(...)
/usr/lib/golang/src/io/io.go:386
os/exec.(*Cmd).writerDescriptor.func1()
/usr/lib/golang/src/os/exec/exec.go:560 +0x3a fp=0xc000113758 sp=0xc0001136f8 pc=0x562eb1a6f55a
os/exec.(*Cmd).Start.func2(0xc000730960?)
/usr/lib/golang/src/os/exec/exec.go:717 +0x32 fp=0xc0001137c8 sp=0xc000113758 pc=0x562eb1a702b2
os/exec.(*Cmd).Start.func3()
/usr/lib/golang/src/os/exec/exec.go:729 +0x2a fp=0xc0001137e0 sp=0xc0001137c8 pc=0x562eb1a7024a
runtime.goexit()
/usr/lib/golang/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0001137e8 sp=0xc0001137e0 pc=0x562eb16729a1
created by os/exec.(*Cmd).Start
/usr/lib/golang/src/os/exec/exec.go:716 +0xa33
Aborted (core dumped)
... and here are my containers, pod and images
root@vlado1:~>podman ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
f66a6d882b48 localhost/podman-pause:4.6.1-1692961697 17 hours ago Up 17 hours 0.0.0.0:8045->8045/tcp 41e624ae7cfa-infra
40a7915e1238 refinst-docker-dev-local.artifactory.cloud.io/grafana-agent1:latest /bin/sh -c /usr/l... 17 hours ago Up 17 hours 0.0.0.0:8045->8045/tcp grafana-agent1
cab3e2ef3622 refinst-docker-dev-local.artifactory.cloud.io/vmware-exporter:latest 17 hours ago Up 17 hours 0.0.0.0:8045->8045/tcp vmware-exporter
root@vlado1:~>podman images
REPOSITORY TAG IMAGE ID CREATED SIZE
refinst-docker-dev-local.artifactory.cloud.io/vmware-exporter latest 4401d0ebb2e2 17 hours ago 940 MB
refinst-docker-dev-local.artifactory.cloud.io/grafana-agent1 latest 2f8f55020d26 17 hours ago 450 MB
localhost/podman-pause 4.6.1-1692961697 558a6d2ad262 17 hours ago 810 kB
refinst-docker-dev-local.artifactory.cloud.io/python39 latest c801b1ebfa4b 18 hours ago 880 MB
refinst-docker-dev-local.artifactory.cloud.io/grafana_agent_latest latest 60eba7832e4b 8 days ago 450 MB
root@vlado1:~>podman pod ls
POD ID NAME STATUS CREATED INFRA ID # OF CONTAINERS
41e624ae7cfa grafana-agent Running 18 hours ago f66a6d882b48 3
root@vlado1:~>podman pod inspect grafana-agent
{
"Id": "41e624ae7cfa64db43c9402a4008e8e9ada270a6f2baa30f6528865d647ac5a2",
"Name": "grafana-agent",
"Created": "2024-02-27T15:07:18.366215421Z",
"CreateCommand": [
"podman",
"pod",
"create",
"--name",
"grafana-agent",
"--hostname",
"grafana-agent1",
"--publish",
"8045:8045"
],
"ExitPolicy": "continue",
"State": "Running",
"Hostname": "grafana-agent1",
"CreateCgroup": true,
"CgroupParent": "machine.slice",
"CgroupPath": "machine.slice/machine-libpod_pod_41e624ae7cfa64db43c9402a4008e8e9ada270a6f2baa30f6528865d647ac5a2.slice",
"CreateInfra": true,
"InfraContainerID": "f66a6d882b48684dd67e9d9e5c15d96db903f40ce8d1b90311d4632ceeee4e98",
"InfraConfig": {
"PortBindings": {
"8045/tcp": [
{
"HostIp": "",
"HostPort": "8045"
}
]
},
"HostNetwork": false,
"StaticIP": "",
"StaticMAC": "",
"NoManageResolvConf": false,
"DNSServer": null,
"DNSSearch": null,
"DNSOption": null,
"NoManageHosts": false,
"HostAdd": null,
"Networks": [
"podman"
],
"NetworkOptions": null,
"pid_ns": "private",
"userns": "host",
"uts_ns": "private"
},
"SharedNamespaces": [
"uts",
"ipc",
"net"
],
"NumContainers": 3,
"Containers": [
{
"Id": "40a7915e1238a4c26cd8f15ffd41a18837f7fbe48093419df799d214d0f49ea4",
"Name": "grafana-agent1",
"State": "running"
},
{
"Id": "cab3e2ef3622245b72a635e2e29ed99c5735a47e6bad25bb568478fd73b75bb1",
"Name": "vmware-exporter",
"State": "running"
},
{
"Id": "f66a6d882b48684dd67e9d9e5c15d96db903f40ce8d1b90311d4632ceeee4e98",
"Name": "41e624ae7cfa-infra",
"State": "running"
}
],
"LockNumber": 0
}
root@vlado1:~>
fHi @db-wally007
Thanks for the information, will provide a fix soon.
Hi @db-wally007
#190 is fixing the issue, it will take 7-10 days for release and offical EPEL rpm package to be available.
Regards