Tests panick with `misaligned pointer dereference: address must be a multiple of ... but is ...`
qmonnet opened this issue · comments
Tests consistently fail with recent versions of the toolchain, with messages such as:
thread 'test_jit_mbuff' panicked at 'misaligned pointer dereference: address must be a multiple of 0x4 but is 0x7f8038001011', src/jit.rs:100:5
Full error message
Running tests/misc.rs (target/debug/deps/misc-6f56f75d4e6d43c9)
running 20 tests
thread 'test_jit_mbuff' panicked at 'misaligned pointer dereference: address must be a multiple of 0x4 but is 0x7f8038001011', src/jit.rs:100:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'test_jit_mbuff' panicked at 'panic in a function that cannot unwind', library/core/src/panicking.rs:126:5
stack backtrace:
0: 0x562d63bf4351 - std::backtrace_rs::backtrace::libunwind::trace::h28494931c73179b2
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93:5
1: 0x562d63bf4351 - std::backtrace_rs::backtrace::trace_unsynchronized::h9032c52edccf7bd1
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
2: 0x562d63bf4351 - std::sys_common::backtrace::_print_fmt::hd90562e967f4e4e1
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/sys_common/backtrace.rs:65:5
3: 0x562d63bf4351 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h113657117676131e
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/sys_common/backtrace.rs:44:22
4: 0x562d63c1995f - core::fmt::rt::Argument::fmt::hd56cdfa11c364505
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/core/src/fmt/rt.rs:138:9
5: 0x562d63c1995f - core::fmt::write::h24c20284e5d6be9e
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/core/src/fmt/mod.rs:1094:21
6: 0x562d63bf1b41 - std::io::Write::write_fmt::hbf02c94f0e7342d1
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/io/mod.rs:1712:15
7: 0x562d63bf4165 - std::sys_common::backtrace::_print::he85212e2c716c859
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/sys_common/backtrace.rs:47:5
8: 0x562d63bf4165 - std::sys_common::backtrace::print::h888aaf3ad10f084e
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/sys_common/backtrace.rs:34:9
9: 0x562d63bf5cb7 - std::panicking::default_hook::{{closure}}::hba0edb58dc223add
10: 0x562d63bf5aa4 - std::panicking::default_hook::h1555b8bada2010d7
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/panicking.rs:288:9
11: 0x562d63b6cdd4 - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::h52e6cd440f597cb6
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/alloc/src/boxed.rs:1999:9
12: 0x562d63b6cdd4 - test::test_main::{{closure}}::hdcda637653172e31
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/test/src/lib.rs:134:21
13: 0x562d63bf62c7 - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::h07438796673f3d04
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/alloc/src/boxed.rs:1999:9
14: 0x562d63bf62c7 - std::panicking::rust_panic_with_hook::h72a06453beb2cbcb
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/panicking.rs:695:13
15: 0x562d63bf6001 - std::panicking::begin_panic_handler::{{closure}}::h0281d6cc05cfd2a4
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/panicking.rs:580:13
16: 0x562d63bf4796 - std::sys_common::backtrace::__rust_end_short_backtrace::h1c79565770be27d9
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/sys_common/backtrace.rs:150:18
17: 0x562d63bf5db2 - rust_begin_unwind
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/panicking.rs:578:5
18: 0x562d63b32b33 - core::panicking::panic_nounwind_fmt::hdb51e63c7c599e80
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/core/src/panicking.rs:96:14
19: 0x562d63b32bd7 - core::panicking::panic_nounwind::hc4511f17cef1f7da
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/core/src/panicking.rs:126:5
20: 0x562d63b32d63 - core::panicking::panic_cannot_unwind::hbac7ba3f5f929c6c
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/core/src/panicking.rs:188:5
21: 0x562d63b8f4e4 - rbpf::jit::emit4::haff8aacbfffbe8b6
at /home/runner/work/rbpf/rbpf/src/jit.rs:99:1
22: 0x562d63b8ff07 - rbpf::jit::emit_alu64_imm32::hd930f0d06f862872
at /home/runner/work/rbpf/rbpf/src/jit.rs:211:5
23: 0x562d63b911af - rbpf::jit::JitMemory::jit_compile::h0a631e7ea59867f2
at /home/runner/work/rbpf/rbpf/src/jit.rs:544:9
24: 0x562d63b93722 - rbpf::jit::compile::hb7b83a17bcef25db
at /home/runner/work/rbpf/rbpf/src/jit.rs:1014:5
25: 0x562d63b842d4 - rbpf::EbpfVmMbuff::jit_compile::h1c675e69584d7361
at /home/runner/work/rbpf/rbpf/src/lib.rs:643:25
26: 0x562d63b34ed0 - misc::test_jit_mbuff::h8eb4bceb4b166566
at /home/runner/work/rbpf/rbpf/tests/misc.rs:336:9
27: 0x562d63b37b57 - misc::test_jit_mbuff::{{closure}}::h55d6770297b8bea5
at /home/runner/work/rbpf/rbpf/tests/misc.rs:314:21
28: 0x562d63b377e5 - core::ops::function::FnOnce::call_once::hafeaeb2bdb95188a
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/core/src/ops/function.rs:250:5
29: 0x562d63b7232f - core::ops::function::FnOnce::call_once::heba63a2808b93cd2
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/core/src/ops/function.rs:250:5
30: 0x562d63b7232f - test::__rust_begin_short_backtrace::he9a5c6c59e1b0086
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/test/src/lib.rs:655:18
31: 0x562d63b3e22c - test::run_test::{{closure}}::h1b8c22c7d438b531
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/test/src/lib.rs:646:30
32: 0x562d63b3e22c - core::ops::function::FnOnce::call_once{{vtable.shim}}::h36a9083506ac2a12
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/core/src/ops/function.rs:250:5
33: 0x562d63b711f7 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h11e5ff78080e1397
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/alloc/src/boxed.rs:1985:9
34: 0x562d63b711f7 - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::hfbfbda226f7a7c27
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/core/src/panic/unwind_safe.rs:271:9
35: 0x562d63b711f7 - std::panicking::try::do_call::h42ac7b4bd35c5b5d
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/panicking.rs:485:40
36: 0x562d63b711f7 - std::panicking::try::hc6ad089a257e422e
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/panicking.rs:449:19
37: 0x562d63b711f7 - std::panic::catch_unwind::hdf4784c0c06ac0ba
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/panic.rs:140:14
38: 0x562d63b711f7 - test::run_test_in_process::hbe86cacc7510eb8b
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/test/src/lib.rs:678:27
39: 0x562d63b711f7 - test::run_test::run_test_inner::{{closure}}::hbaa8a391d0f22860
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/test/src/lib.rs:572:39
40: 0x562d63b38c28 - test::run_test::run_test_inner::{{closure}}::hb4232323322f922d
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/test/src/lib.rs:599:37
41: 0x562d63b38c28 - std::sys_common::backtrace::__rust_begin_short_backtrace::hca81c15946d6f154
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/sys_common/backtrace.rs:134:18
42: 0x562d63b3e45b - std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}::h97d990ba33daf685
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/thread/mod.rs:529:17
43: 0x562d63b3e45b - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::he0d612b643c049a4
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/core/src/panic/unwind_safe.rs:271:9
44: 0x562d63b3e45b - std::panicking::try::do_call::haf308dcb076117ac
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/panicking.rs:485:40
45: 0x562d63b3e45b - std::panicking::try::h51fd60d29cf6feee
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/panicking.rs:449:19
46: 0x562d63b3e45b - std::panic::catch_unwind::h8e7f9d8c4a32bc2f
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/panic.rs:140:14
47: 0x562d63b3e45b - std::thread::Builder::spawn_unchecked_::{{closure}}::h07454567f5413f25
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/thread/mod.rs:528:30
48: 0x562d63b3e45b - core::ops::function::FnOnce::call_once{{vtable.shim}}::hfb80b7cf501e16ff
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/core/src/ops/function.rs:250:5
49: 0x562d63bfa815 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h71c821b130855373
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/alloc/src/boxed.rs:1985:9
50: 0x562d63bfa815 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h1701cec9acb1061c
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/alloc/src/boxed.rs:1985:9
51: 0x562d63bfa815 - std::sys::unix::thread::Thread::new::thread_start::h6dd5ef62dde103f1
at /rustc/2f6bc5d259e7ab25ddfdd33de53b892770218918/library/std/src/sys/unix/thread.rs:108:17
52: 0x7f803c6e9b43 - <unknown>
53: 0x7f803c77ba00 - <unknown>
54: 0x0 - <unknown>
thread caused non-unwinding panic. aborting.
error: test failed, to rerun pass `--test misc`
This seems to be a consequence of rust-lang/rust#98112.
As an easy “fix”, we can disable debug assertions for the tests, but we risk introducing more bugs. Not sure how to fix it otherwise, given that we do support unaligned memory access in rbpf at this time.
It looks to me like the offending code is this:
Lines 81 to 82 in 196e537
This could do ptr.write_unaligned($data);
I'm not sure what you mean by
we do support unaligned memory access in rbpf at this time
The checks are inserted as part of compiling the Rust code that is the JIT itself, so I don't think the semantics of BPF here are relevant, right? I feel like I must be misunderstanding something here.
Hi Ben, thanks a lot for helping!
I feel like I must be misunderstanding something here.
This would be entirely possible, my experience with coding with Rust is limited to this project :)
In this case, let me explain more: in rbpf, we have two ways to execute eBPF code, the interpreter (contained in src/lib.rs) and the JIT-compiler (srs/jit.rs). I tested your suggestion to use ptr.write_unaligned($data); for the JIT-compiler, and it works well indeed for taming the asserts that appear when compiling the JIT itself, and fixes a number of tests 🎉. I'll push a PR with this change [edit: #80, now merged].
Then I've also observed error messages referring to the interpreter, and related to the tests using the interpreter for the eBPF programs, in particular the instructions for loading multiple bytes of memory into registers: ebpf::LD_H_REG/ebpf::LD_W_REG/ebpf::LD_DW_REG, around this line. This is what I was referring to earlier. Maybe there's a similar trick to use here with write_unaligned() or something else to tell Rust that these are possibly not aligned, I need to look more into it.
List of failing test (running on the interpreter), after fixing the JIT, for the record:
-
README.md:
- Block starting at line 393
-
test/misc.rs:
- test_vm_block_port
- test_vm_jit_ldabsh
- test_vm_jit_ldabsw
- test_vm_jit_ldabsdw
- test_vm_jit_ldindw
- test_vm_jit_ldinddw
-
test/ubpf_vm.rs
- test_vm_ldxdw
- test_vm_ldxw
- test_vm_stdw
- test_vm_stw
- test_vm_stxdw
- test_vm_stxw