We can turn our cpu profile dumping to be crash-only (was: empty cpu profile (profiling haproxy on fedora))
chipitsine opened this issue · comments
some repro steps:
git clone https://github.com/haproxy/haproxy
cd haproxy
make all ERR=1 TARGET=linux-glibc USE_OPENSSL=1 LDFLAGS="-lprofiler"
curl -fsSL https://github.com/summerwind/h2spec/releases/download/v2.6.0/h2spec_linux_amd64.tar.gz -o h2spec.tar.gz
tar xvf h2spec.tar.gz
chmod +x h2spec
CPUPROFILE=haproxy.prof ./haproxy -f .github/h2spec.config -d &
sleep 5
./h2spec -Svtk -h 127.0.0.1 -p 8443
killall haproxy
file haproxy.prof appears, but it's empty
ilia@fedora:~/temp/haproxy-profile$ ls -l haproxy.prof
-rw-r--r-- 1 ilia ilia 0 Jan 7 16:05 haproxy.prof
ilia@fedora:~/temp/haproxy-profile$
OS: Fedora 39
gperftools: gperftools-libs-2.9.1-6.fc39.x86_64
I've just noticed that Fedora is using pretty old gperftools.
give me some time to try the latest release
well, I tested on 2.15, the same zero size haproxy.prof
This is because profile is dumped on program shutdown. If it dies from signal, there is no profile flushing.
Have your program stop normally, or have it stop normally on signal. Or arrange profiling for specific duration and Stop it yourself.
As somewhat crude (and not 100% reliable) workaround people have contributed CPUPROFILESIGNAL. Which allows you to toggle profiling on/off by sending that signal. Feel free to look it up in the code.
Yes, we could turn our profile dumping design to be continuous (via mmap-ed files) and crash-only. But this is an enhancement.
Let me turn this ticket into such enhancement. If there is anyone willing to volunteer several days of work into this, I am willing to guide such person.
hello, @alk
thank you for you help. I previously tried CPUPROFILESIGNAL, but it also behaves in strange way
as far as I understand, I must send a signal twice: to begin collecting and to end collecting.
git clone https://github.com/haproxy/haproxy
cd haproxy
make all ERR=1 TARGET=linux-glibc USE_OPENSSL=1 LDFLAGS="-lprofiler"
curl -fsSL https://github.com/summerwind/h2spec/releases/download/v2.6.0/h2spec_linux_amd64.tar.gz -o h2spec.tar.gz
tar xvf h2spec.tar.gz
chmod +x h2spec
CPUPROFILE=haproxy.prof CPUPROFILESIGNAL=12 ./haproxy -f .github/h2spec.config -d &
sleep 5
killall -12 haproxy
./h2spec -Svtk -h 127.0.0.1 -p 8443
killall -12 haproxy
sleep 5
killall haproxy
no haproxy.prof was created
ilia@fedora:~/temp/haproxy-profile$ ls -l haproxy.prof
ls: cannot access 'haproxy.prof': No such file or directory
ilia@fedora:~/temp/haproxy-profile$
nevermind, I've managed with "31" signal. 12 was just handled by haproxy itself