could AF_XDP co-exists with xdp dispatcher?

[Vincent Li]

I use xdp-loader to load XDP program on an interface that is already attached with AF_XDP, got error below:

xdp-loader load eno2 -m skb ./xdp_pass.o
libxdp: Generating multi-prog dispatcher for 1 programs
libxdp: Existing program is not using a dispatcher, can't replace; unload first
Couldn't attach XDP program on iface 'eno2': Device or resource busy(-16)

the program id 38 is used by AF_XDP

root@r220:~# xdp-loader status
CURRENT XDP PROGRAM STATUS:

Interface        Prio  Program name      Mode     ID   Tag               Chain actions
--------------------------------------------------------------------------------------
lo                     <No XDP program loaded!>
enp1s0f0               <No XDP program loaded!>
enp1s0f1               <No XDP program loaded!>
eno1                   <No XDP program loaded!>
eno2                                     skb      38   03b13f331978c78c 

use case is like packet -> XDP DDOS -> AF_XDP, is that possible?

[Vincent Li]

by the way, I am testing suricata AF_XDP

maybe suricata AF_XDP is not setup to use xdp dispatcher ?

src/source-af-xdp.c

    /* Socket configuration */
    ptv->xsk.cfg.rx_size = XSK_RING_CONS__DEFAULT_NUM_DESCS;
    ptv->xsk.cfg.tx_size = XSK_RING_PROD__DEFAULT_NUM_DESCS;
    ptv->xsk.cfg.xdp_flags = afxdpconfig->mode;
    ptv->xsk.cfg.bind_flags = afxdpconfig->bind_flags;

[Toke Høiland-Jørgensen]

Yeah, libxdp multiprog only works if every XDP user on the system uses it (or a compatible library). The AF_XDP support in libxdp itself *does* use the dispatcher, so any application that has upgraded from the older libbpf support to libxdp will automatically co-exist with other programs through the dispatcher. If Suricata does not use libxdp there is not much that can be done from the libxdp side; maybe submit a patch to Suricata to use libxdp for their AF_XDP attachment?

[Vincent Li]

Thanks @tohojo for replying :)

I had a look at Suricata code source-af-xdp.c L430-L446 , it calls xsk_socket__create which should eventually calls __xsk_setup_xdp_prog that should use dispatcher.

I also looked at bpf-examples xdpsock.c L1017-L1030 which also uses xsk_socket__create, and it uses dispatcher, so I am not sure what config setting is required to allow xsk_socket__create to use dispatcher, no need to set
libxdp_flags or libbpf_flags, right? maybe something else in Suricata stops it using dispatcher, still looking...

[Vincent Li]

it seems my libbpf version (0.7) mixed up with libxdp installation, and suricata might be confused too, after I re-install libbpf latest version, re-install libxdp, and re-install suricata, the issue is gone

/usr/bin/suricata -c /etc/suricata/suricata.yaml --af-xdp=eno2 -vvv
Notice: suricata: This is Suricata version 7.0.1-dev (4044d0287 2023-08-31) running in SYSTEM mode [LogVersion:suricata.c:1154]

Perf: af-xdp: 2 cores, so using 2 threads [ConfigSetThreads:runmode-af-xdp.c:125]
Info: runmodes: eno2: creating 1 thread [RunModeSetLiveCaptureWorkersForDevice:util-runmodes.c:255]
libbpf: elf: skipping unrecognized data section(8) .xdp_run_config
libbpf: elf: skipping unrecognized data section(9) xdp_metadata
libbpf: elf: skipping unrecognized data section(7) xdp_metadata
libbpf: elf: skipping unrecognized data section(7) xdp_metadata
libbpf: elf: skipping unrecognized data section(7) xdp_metadata
libbpf: elf: skipping unrecognized data section(7) xdp_metadata
Config: flow-manager: using 1 flow manager threads [FlowManagerThreadSpawn:flow-manager.c:948]
Config: flow-manager: using 1 flow recycler threads [FlowRecyclerThreadSpawn:flow-manager.c:1154]
Info: unix-manager: unix socket '/var/run/suricata/suricata-command.socket' [UnixNew:unix-manager.c:136]
Notice: threads: Threads created -> W: 1 FM: 1 FR: 1   Engine started. [TmThreadWaitOnThreadRunning:tm-threads.c:1890]

xdp-loader status shows xdp dispatcher being used

xdp-loader status
CURRENT XDP PROGRAM STATUS:

Interface        Prio  Program name      Mode     ID   Tag               Chain actions
--------------------------------------------------------------------------------------
lo                     <No XDP program loaded!>
enp1s0f0               <No XDP program loaded!>
enp1s0f1               <No XDP program loaded!>
eno1                   <No XDP program loaded!>
eno2                   xdp_dispatcher    skb      532  90f686eb86991928 
 =>              20     xsk_def_prog              541  8f9c40757cb0a6a2  XDP_PASS
br0                    <No XDP program loaded!>
wg0                    <No XDP program loaded!>
virbr0                 <No XDP program loaded!>