Improve IXGBE driver performance
jiegec opened this issue · comments
Based on iperf.
Current status (tcp, maximum observed):
rcore -> linux 1.24Gb/s -> 1.40Gb/s -> 2.18Gb/s -> 2.62Gb/s -> 3.50Gb/s
linux -> rcore 308Mb/s -> 320Mb/s -> 379Mb/s -> 407Mb/s -> 1.40Gb/s -> 2.47Gb/s -> 2.53Gb/s -> 3.74Gb/s -> 4.44Gb/s
Udp is not working because smoltcp does not support segmentation.
Possible ways to improve:
- IP checksum TX
- IP checksum RX: little improvement
- UDP checksum TX
- UDP checksum RX
- TCP checksum TX
- TCP checksum RX
- TCP segmentation offload
- Interrupt throttling: good for tx, bad for rx without spinning
- Spin for some times before waiting for condvar: observable improvement for rx
- Enlarge TCP recv/trans buffers: observable improvement for rx
- Replace linked_list_allocator with buddy_system_allocator: a little improvement for rx
- Enable jumbo frame: observable improvement
Manually disable RX checksum for smoltcp seems to have little improvement.
I think we have met smoltcp's performance limit.
Test under linux:
Put two interfaces into two netns, then use iperf to test:
tcp: 9.38Gb/s
Idea:
Check performance of sending raw ethernet packets, skipping network stack.