wqlat.py fails for 6.7 kernel with -w workqueue selection
chaiken opened this issue · comments
Linux remerge 6.7.0 #2 SMP PREEMPT_DYNAMIC Sun Jan 28 10:05:56 PST 2024 x86_64 GNU/Linux
Kernel compiled from commit v6.7. Kconfig is attached; most unusual change is NR_CPUS=8
6.7-Kconfig.txt
wqlat works fine with no flags:
alison@remerge:~/gitsrc/bcc$ sudo python3 tools/wqlat.py -T -W 1 2
In file included from /virtual/main.c:2:
In file included from include/uapi/linux/ptrace.h:183:
In file included from arch/x86/include/asm/ptrace.h:5:
In file included from arch/x86/include/asm/segment.h:7:
arch/x86/include/asm/ibt.h:77:8: warning: 'nocf_check' attribute ignored; use -fcf-protection to enable the attribute [-Wignored-attributes]
extern __noendbr u64 ibt_save(bool disable);
^
arch/x86/include/asm/ibt.h:32:34: note: expanded from macro '__noendbr'
#define __noendbr __attribute__((nocf_check))
^
arch/x86/include/asm/ibt.h:78:8: warning: 'nocf_check' attribute ignored; use -fcf-protection to enable the attribute [-Wignored-attributes]
extern __noendbr void ibt_restore(u64 save);
^
arch/x86/include/asm/ibt.h:32:34: note: expanded from macro '__noendbr'
#define __noendbr __attribute__((nocf_check))
^
2 warnings generated.
Tracing work queue request latency time... Hit Ctrl-C to end.
20:16:59
wqname = events_unbound
usecs : count distribution
0 -> 1 : 0 | |
2 -> 3 : 1 |******************** |
4 -> 7 : 0 | |
8 -> 15 : 1 |******************** |
16 -> 31 : 0 | |
32 -> 63 : 0 | |
64 -> 127 : 2 |****************************************|
It fails to load the bytecode with -w no matter which workqueue is designated:
alison@remerge:~/gitsrc/bcc$ sudo python3 tools/wqlat.py -w nvmet_tcp_wq 1
In file included from /virtual/main.c:2:
In file included from include/uapi/linux/ptrace.h:183:
In file included from arch/x86/include/asm/ptrace.h:5:
In file included from arch/x86/include/asm/segment.h:7:
arch/x86/include/asm/ibt.h:77:8: warning: 'nocf_check' attribute ignored; use -fcf-protection to enable the attribute [-Wignored-attributes]
extern __noendbr u64 ibt_save(bool disable);
^
arch/x86/include/asm/ibt.h:32:34: note: expanded from macro '__noendbr'
#define __noendbr __attribute__((nocf_check))
^
arch/x86/include/asm/ibt.h:78:8: warning: 'nocf_check' attribute ignored; use -fcf-protection to enable the attribute [-Wignored-attributes]
extern __noendbr void ibt_restore(u64 save);
^
arch/x86/include/asm/ibt.h:32:34: note: expanded from macro '__noendbr'
#define __noendbr __attribute__((nocf_check))
^
2 warnings generated.
bpf: Failed to load program: Invalid argument
jump out of range from insn 18 to 32683
processed 0 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0
Traceback (most recent call last):
File "/home/alison/gitsrc/bcc/tools/wqlat.py", line 161, in <module>
b = BPF(text=bpf_text)
^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/bcc/__init__.py", line 487, in __init__
self._trace_autoload()
File "/usr/lib/python3/dist-packages/bcc/__init__.py", line 1465, in _trace_autoload
fn = self.load_func(func_name, BPF.TRACEPOINT)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/bcc/__init__.py", line 526, in load_func
raise Exception("Failed to load BPF program %s: %s" %
Exception: Failed to load BPF program b'tracepoint__workqueue__workqueue_queue_work': Invalid argument
Similar for other workqueue choices:
alison@remerge:~/gitsrc/bcc$ sudo python3 tools/wqlat.py -w ext-rsv-conversion 2 5
In file included from /virtual/main.c:2:
In file included from include/uapi/linux/ptrace.h:183:
In file included from arch/x86/include/asm/ptrace.h:5:
In file included from arch/x86/include/asm/segment.h:7:
arch/x86/include/asm/ibt.h:77:8: warning: 'nocf_check' attribute ignored; use -fcf-protection to enable the attribute [-Wignored-attributes]
extern __noendbr u64 ibt_save(bool disable);
^
arch/x86/include/asm/ibt.h:32:34: note: expanded from macro '__noendbr'
#define __noendbr __attribute__((nocf_check))
^
arch/x86/include/asm/ibt.h:78:8: warning: 'nocf_check' attribute ignored; use -fcf-protection to enable the attribute [-Wignored-attributes]
extern __noendbr void ibt_restore(u64 save);
^
arch/x86/include/asm/ibt.h:32:34: note: expanded from macro '__noendbr'
#define __noendbr __attribute__((nocf_check))
^
2 warnings generated.
bpf: Failed to load program: Invalid argument
jump out of range from insn 18 to 32582
processed 0 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0
Traceback (most recent call last):
File "/home/alison/gitsrc/bcc/tools/wqlat.py", line 161, in <module>
b = BPF(text=bpf_text)
^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/bcc/__init__.py", line 487, in __init__
self._trace_autoload()
File "/usr/lib/python3/dist-packages/bcc/__init__.py", line 1465, in _trace_autoload
fn = self.load_func(func_name, BPF.TRACEPOINT)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/bcc/__init__.py", line 526, in load_func
raise Exception("Failed to load BPF program %s: %s" %
Exception: Failed to load BPF program b'tracepoint__workqueue__workqueue_queue_work': Invalid argument
hi, I used the latest release version 6.7.5 to test, it seems work well. you can see my testlog below.
admin@sandy-goop:/ping> uname -a/ping> sudo /usr/share/bcc/tools/wqlat -T -w writeback 1 2
Linux sandy-goop 6.7.5-150400.24.100-default #1 SMP PREEMPT_DYNAMIC Mon Feb 19 03:54:02 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
admin@sandy-goop:
Tracing work queue request latency time... Hit Ctrl-C to end.
07:34:05
usecs : count distribution
0 -> 1 : 0 | |
2 -> 3 : 0 | |
4 -> 7 : 0 | |
8 -> 15 : 1 ||
16 -> 31 : 1 ||
07:34:06
usecs : count distribution
0 -> 1 : 0 | |
2 -> 3 : 0 | |
4 -> 7 : 0 | |
8 -> 15 : 3 |****************************************|
16 -> 31 : 2 |************************** |
admin@sandy-goop:~/ping>