node_exporter keeps throwing logs in /var/log/messages
neilvvakil opened this issue · comments
uname -a
Linux seieadb101 4.1.12-124.28.5.el7uek.x86_64 #2 SMP Thu Jun 20 12:00:04 PDT 2019 x86_64 x86_64 x86_64 GNU/Linux
./node_exporter --version
node_exporter, version 1.5.0 (branch: HEAD, revision: 1b48970)
build user: root@6e7732a7b81b
build date: 20221129-18:59:09
go version: go1.19.3
platform: linux/amd64
node_exporter command line flags
root@seieadb101 # cat /etc/systemd/system/node_exporter.service
[Unit]
Description=Node Exporter
RequiresMountsFor=/apps
After=network.target
[Service]
User=prometheus
ExecStart=/apps/prometheus/node_exporter/node_exporter
[Install]
WantedBy=default.target
node_exporter log output
Are you running node_exporter in Docker?
No
What did you do that produced an error?
error just comes in randomly
What did you see instead?
errors in /var/log/messages
Without the error messages there is nothing we can do.
not able to add /var/log/messages or log files
snippet from messages file
Oct 1 14:17:46 seieadb101 rpcbind[34879]: connect from 172.18.72.119 to set(nfs): indirect call not allowed
Oct 1 14:17:49 seieadb101 rpcbind[35119]: connect from 172.18.80.114 to set(nfs): indirect call not allowed
Oct 1 14:17:52 seieadb101 node_exporter: 2023/10/01 14:17:52 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:53 seieadb101 node_exporter: 2023/10/01 14:17:53 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:53 seieadb101 node_exporter: 2023/10/01 14:17:53 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:53 seieadb101 node_exporter: 2023/10/01 14:17:53 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:53 seieadb101 node_exporter: 2023/10/01 14:17:53 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:53 seieadb101 node_exporter: 2023/10/01 14:17:53 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:53 seieadb101 node_exporter: 2023/10/01 14:17:53 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:53 seieadb101 node_exporter: 2023/10/01 14:17:53 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:53 seieadb101 node_exporter: 2023/10/01 14:17:53 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:53 seieadb101 node_exporter: 2023/10/01 14:17:53 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:53 seieadb101 node_exporter: 2023/10/01 14:17:53 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:53 seieadb101 node_exporter: 2023/10/01 14:17:53 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:53 seieadb101 node_exporter: 2023/10/01 14:17:53 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:53 seieadb101 node_exporter: 2023/10/01 14:17:53 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:53 seieadb101 node_exporter: 2023/10/01 14:17:53 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:53 seieadb101 node_exporter: 2023/10/01 14:17:53 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:53 seieadb101 node_exporter: 2023/10/01 14:17:53 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:53 seieadb101 node_exporter: 2023/10/01 14:17:53 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:53 seieadb101 node_exporter: 2023/10/01 14:17:53 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:53 seieadb101 node_exporter: 2023/10/01 14:17:53 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:53 seieadb101 node_exporter: 2023/10/01 14:17:53 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:53 seieadb101 node_exporter: 2023/10/01 14:17:53 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:57 seieadb101 node_exporter: 2023/10/01 14:17:57 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:57 seieadb101 node_exporter: 2023/10/01 14:17:57 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:57 seieadb101 node_exporter: 2023/10/01 14:17:57 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:57 seieadb101 node_exporter: 2023/10/01 14:17:57 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:57 seieadb101 node_exporter: 2023/10/01 14:17:57 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:57 seieadb101 node_exporter: 2023/10/01 14:17:57 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:57 seieadb101 node_exporter: 2023/10/01 14:17:57 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:57 seieadb101 node_exporter: 2023/10/01 14:17:57 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:57 seieadb101 node_exporter: 2023/10/01 14:17:57 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:57 seieadb101 node_exporter: 2023/10/01 14:17:57 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:57 seieadb101 node_exporter: 2023/10/01 14:17:57 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:57 seieadb101 node_exporter: 2023/10/01 14:17:57 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:57 seieadb101 node_exporter: 2023/10/01 14:17:57 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:57 seieadb101 node_exporter: 2023/10/01 14:17:57 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:57 seieadb101 node_exporter: 2023/10/01 14:17:57 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:57 seieadb101 node_exporter: 2023/10/01 14:17:57 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:57 seieadb101 node_exporter: 2023/10/01 14:17:57 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:57 seieadb101 node_exporter: 2023/10/01 14:17:57 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:57 seieadb101 node_exporter: 2023/10/01 14:17:57 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:57 seieadb101 node_exporter: 2023/10/01 14:17:57 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:57 seieadb101 node_exporter: 2023/10/01 14:17:57 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:17:57 seieadb101 node_exporter: 2023/10/01 14:17:57 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:18:03 seieadb101 node_exporter: 2023/10/01 14:18:03 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:18:03 seieadb101 node_exporter: 2023/10/01 14:18:03 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:18:03 seieadb101 node_exporter: 2023/10/01 14:18:03 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:18:03 seieadb101 node_exporter: 2023/10/01 14:18:03 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:18:03 seieadb101 node_exporter: 2023/10/01 14:18:03 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:18:03 seieadb101 node_exporter: 2023/10/01 14:18:03 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:18:03 seieadb101 node_exporter: 2023/10/01 14:18:03 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:18:03 seieadb101 node_exporter: 2023/10/01 14:18:03 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:18:03 seieadb101 node_exporter: 2023/10/01 14:18:03 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:18:03 seieadb101 node_exporter: 2023/10/01 14:18:03 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:18:03 seieadb101 node_exporter: 2023/10/01 14:18:03 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:18:03 seieadb101 node_exporter: 2023/10/01 14:18:03 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:18:03 seieadb101 node_exporter: 2023/10/01 14:18:03 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:18:03 seieadb101 node_exporter: 2023/10/01 14:18:03 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:18:03 seieadb101 node_exporter: 2023/10/01 14:18:03 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:18:03 seieadb101 node_exporter: 2023/10/01 14:18:03 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:18:03 seieadb101 node_exporter: 2023/10/01 14:18:03 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:18:03 seieadb101 node_exporter: 2023/10/01 14:18:03 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:18:03 seieadb101 node_exporter: 2023/10/01 14:18:03 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:18:03 seieadb101 node_exporter: 2023/10/01 14:18:03 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:18:03 seieadb101 node_exporter: 2023/10/01 14:18:03 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:18:03 seieadb101 node_exporter: 2023/10/01 14:18:03 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:19:31 seieadb101 node_exporter: 2023/10/01 14:19:31 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
Oct 1 14:19:58 seieadb101 dbus[22443]: [system] Activating service name='org.freedesktop.problems' (using servicehelper)
Oct 1 14:19:58 seieadb101 dbus[22443]: [system] Successfully activated service 'org.freedesktop.problems'
Oct 1 14:20:11 seieadb101 rsyslogd: sd_journal_get_cursor() failed: 'Cannot assign requested address' [v8.24.0-34.el7]
You have something sending invalid HTTP requests to your node_exporter. This is not a bug in the exporter.
Can you assist on how to find that out ?