Dynamic log level change doesn't work the same for all environments
PhilipSchmid opened this issue · comments
What happened?
The Tetragon doc has a Change log level dynamically section that explains how to dynamically change it using SIGRTMIN+x
. However, SIGRTMIN
isn't the same value for every environment:
The Linux kernel supports a range of 33 different real-time signals, numbered 32 to 64. However, the glibc
POSIX threads implementation internally uses two (for NPTL) or three (for LinuxThreads) real-time signals (see
pthreads(7)), and adjusts the value of SIGRTMIN suitably (to 34 or 35).
It depends on libc, and, e.g., Alpine uses musl: https://git.musl-libc.org/cgit/musl/commit/src/signal/sigrtmin.c?id=016a5dc1925a66c7d1ffc14b862b1342f399cfda
We would likely need to develop a proper tetra
CLI command to set the debug level in a unified and environment-independent way.
I ran into this issue while deploying Tetragon on a Talos Linux environment. For reproduction, you could use https://github.com/isovalent/terraform-aws-talos/tree/main/example.
Relevant Tetragon part:
Lines 310 to 329 in e44f7df
Tetragon Version
v1.1.0
Kernel Version
6.6.29-talos
Kubernetes Version
v1.29.3
Bugtool
No response
Relevant log output
No response
Anything else?
No response