Booting the VM kernel gives "Internal error: undefined instruction: 0 [#1] SMP" on Volterra
jglathe opened this issue · comments
Hi,
I was checking out if I could run firecracker-vm on the Volterra box (Windows Dev Kit 2023). It is booted into EL2, has /dev/kvm, and also an lxc VM running to check if VMs are generally available and working.
When booting it halts with the undefined instruction, not quite sure if it's a kernel config that needs to be changed.
[ 0.005519] ASID allocator initialised with 32768 entries
[ 0.005920] Hierarchical SRCU implementation.
[ 0.006492] EFI services will not be available.
[ 0.006825] smp: Bringing up secondary CPUs ...
[ 0.007143] smp: Brought up 1 node, 1 CPU
[ 0.007417] SMP: Total of 1 processors activated.
[ 0.007714] CPU features: detected: GIC system register CPU interface
[ 0.008131] CPU features: detected: Privileged Access Never
[ 0.008494] CPU features: detected: User Access Override
[ 0.008840] CPU features: detected: 32-bit EL0 Support
[ 0.009208] Internal error: undefined instruction: 0 [#1] SMP
[ 0.009578] Process migration/0 (pid: 10, stack limit = 0xffffff8008b78000)
[ 0.010030] CPU: 0 PID: 10 Comm: migration/0 Not tainted 4.14.174+ #14
[ 0.010443] Hardware name: linux,dummy-virt (DT)
[ 0.010739] task: ffffffc0068a0000 task.stack: ffffff8008b78000
[ 0.011147] PC is at arm64_set_ssbd_mitigation+0x64/0xa0
[ 0.011492] LR is at arm64_set_ssbd_mitigation+0x18/0xa0
[ 0.011851] pc : [<ffffff800808e154>] lr : [<ffffff800808e108>] pstate: 004000c5
[ 0.012346] sp : ffffff8008b7bd20
[ 0.012569] x29: ffffff8008b7bd20 x28: 0000000000000000
[ 0.012927] x27: ffffff800803bbd0 x26: 0000000000000001
[ 0.013282] x25: ffffff800871ee20 x24: ffffff8008854e66
[ 0.013637] x23: 0000000000000001 x22: 0000000000000040
[ 0.013997] x21: ffffff800803bc94 x20: ffffff800887c4a8
[ 0.014353] x19: 0000000000000001 x18: ffffffffffffffff
[ 0.014710] x17: 0000000000000007 x16: 0000000000000001
[ 0.015072] x15: ffffff80087fad08 x14: ffffff808888af37
[ 0.015416] x13: 0000000000000000 x12: 0000000000000001
[ 0.015765] x11: 0000000000000000 x10: 0000000000000a00
[ 0.016110] x9 : ffffff8008b7bd70 x8 : ffffffc0068a0a60
[ 0.016455] x7 : 0000000000000000 x6 : 00000000ffffffff
[ 0.016793] x5 : 0000003fff6eb000 x4 : 0000000000000004
[ 0.017089] x3 : 0000000000000000 x2 : 0000000000000001
[ 0.017432] x1 : 0000000000000001 x0 : 0000000000000001
[ 0.017776] Call trace:
[ 0.017941] Exception stack(0xffffff8008b7bbe0 to 0xffffff8008b7bd20)
[ 0.018363] bbe0: 0000000000000001 0000000000000001 0000000000000001 0000000000000000
[ 0.018878] bc00: 0000000000000004 0000003fff6eb000 00000000ffffffff 0000000000000000
[ 0.019342] bc20: ffffffc0068a0a60 ffffff8008b7bd70 0000000000000a00 0000000000000000
[ 0.019845] bc40: 0000000000000001 0000000000000000 ffffff808888af37 ffffff80087fad08
[ 0.020325] bc60: 0000000000000001 0000000000000007 ffffffffffffffff 0000000000000001
[ 0.020750] bc80: ffffff800887c4a8 ffffff800803bc94 0000000000000040 0000000000000001
[ 0.021178] bca0: ffffff8008854e66 ffffff800871ee20 0000000000000001 ffffff800803bbd0
[ 0.021646] bcc0: 0000000000000000 ffffff8008b7bd20 ffffff800808e108 ffffff8008b7bd20
[ 0.022099] bce0: ffffff800808e154 00000000004000c5 ffffff8008652778 ffffff800808a95c
[ 0.022592] bd00: ffffffffffffffff ffffff800808e108 ffffff8008b7bd20 ffffff800808e154
[ 0.023098] [<ffffff800808e154>] arm64_set_ssbd_mitigation+0x64/0xa0
[ 0.023505] [<ffffff800808ef14>] cpu_enable_ssbs+0x74/0xa0
[ 0.023846] [<ffffff800808e6b0>] __enable_cpu_capability+0x10/0x20
[ 0.024233] [<ffffff800813535c>] multi_cpu_stop+0x8c/0x110
[ 0.024572] [<ffffff8008135634>] cpu_stopper_thread+0xc4/0x148
[ 0.024893] [<ffffff80080c3d30>] smpboot_thread_fn+0x1a0/0x1d0
[ 0.025261] [<ffffff80080bf80c>] kthread+0x12c/0x130
[ 0.025567] [<ffffff8008084c50>] ret_from_fork+0x10/0x18
[ 0.025898] Code: d4000002 f9400bf3 a8c27bfd d65f03c0 (d503403f)
[ 0.026279] ---[ end trace 66dc7e40a2c28e42 ]---
[ 0.026574] note: migration/0[10] exited with preempt_count 1
Attached is the complete log, it boots the VM. And balks on the code.
To get the vmlinux kernel I have set export arch=aarch64, maybe it must be more specific?
Thanks for testing out KVM on the Dev Kit. I think this might need to be raised on the Firecracker repo for the right kind of help. https://github.com/firecracker-microvm/firecracker
I wonder if you'd be able to try out the cloud-hypervisor instructions for an aarch64 system too?
https://github.com/cloud-hypervisor/cloud-hypervisor
I've those steps on the Ampere Altra Developer Platform.
I'm not sure if the EL2 hypervisor mode is relevant or supported.