CESNET / ipfixprobe

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Cannot set CPU affinity in DPDK input

thorgrin opened this issue · comments

I have two NICs and I need to run two ipfixprobe processes to monitor the traffic. When I start as follows

ipfixprobe  --input 'dpdk;port=0;eal=ipfixprobe-eal -c 0x1 -a 0000:d8:00.0;q=1'

ipfixprobe  --input 'dpdk;port=0;eal=ipfixprobe-eal -c 0x2 -a 0000:3b:00.0 --file-prefix aaa;q=1'

both processes end up on first CPU, which is a great problem while actively polling for traffic - each process shows only 50% CPU usage, the packet reading often stalls in one or the other.

Possible solution

I've come up with the following workaround:

Comment out the call to set_thread_affinity function: https://github.com/CESNET/ipfixprobe/blob/master/input/dpdk.cpp#L330

Run with eal set as follows:

/usr/bin/ipfixprobe -i 'dpdk;p=0;q=1;e=ipfixprobe-eal --lcores (0-7)@(0,2,4,6,8,10,12,14,16,18,20,22) -a 0000:3b:00.0'
/usr/bin/ipfixprobe -i 'dpdk;p=0;q=1;e=ipfixprobe-eal --lcores (0-7)@(1,3,5,7,9,11,13,15,17,19,21,23) -a 0000:d8:00.0 --file-prefix=aaa'

The list of cores is set to the appropriate CPU by checking /sys/class/net/$ifc/device/local_cpulist.

Setup

OS: Ubuntu 22.04 LTS
Compilation as deb package using make deb with modified dh_auto_configure in debian/rules

Full configure params:

 ./configure --build=x86_64-linux-gnu --prefix=/usr --includedir=\${prefix}/include --mandir=\${prefix}/share/man --infodir=\${prefix}/share/info --sysconfdir=/etc --localstatedir=/var --disable-option-checking --disable-silent-rules --libdir=\${prefix}/lib/x86_64-linux-gnu --libexecdir=\${prefix}/lib/x86_64-linux-gnu --disable-maintainer-mode --disable-d
ependency-tracking --with-pcap --with-quic --with-msects --with-dpdk --enable-debug