GPU doesn't get unbinded

[Slavik]

Environment:

I have Harvester cluster v1.3.1

• one master DELL T7820 with nVidia P620
• one worker DELL T7920 with nVidia P5000

Steps

• I enabled passthrough for both GPUs.
• I created Ubuntu 22 VM, added P5000 device to it. Started.

Errors

can't passthrough nVidia P5000 , as it still binded to host

looking at kernel logs:

dmesg -T
[Wed Jun 12 02:42:37 2024] vfio-pci 0000:b3:00.0: vfio_ecap_init: hiding ecap 0x19@0x900
[Wed Jun 12 02:42:37 2024] vfio-pci 0000:b3:00.0: BAR 1: can’t reserve [mem 0xe0000000-0xefffffff 64bit pref]
[Wed Jun 12 02:43:45 2024] vfio-pci 0000:b3:00.0: BAR 1: can’t reserve [mem 0xe0000000-0xefffffff 64bit pref]

Workaround:

only when I manually unbind it, I can pass-through:

echo 0 > /sys/class/vtconsole/vtcon0/bind
echo 0 > /sys/class/vtconsole/vtcon1/bind
echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/unbind

But those "unbinding" get reset on the node reboot, and I had to repeat it.