Gather the data from the FreeBSD's tcp_log device.
Using a FreeBSD -head (r363032 minimum, to have the extra TCP stack headers installed), compile a new kernel with BBR and extra TCP stack enabled:
# cat /usr/src/sys/amd64/conf/BBR
include GENERIC-NODEBUG
ident BBR
options TCPHPTS
options RATELIMIT
makeoptions WITH_EXTRA_TCP_STACKS=1
# cat /etc/src.conf
KERNCONF=BBR
MALLOC_PRODUCTION=yes
Build and install this customized kernel. Checking for those files
- /boot/kernel/tcp_bbr.ko
- /boot/kernel/tcp_rack.ko
Load thoses modules during startup (sooner on /boot/loader.conf or later on /etc/rc.conf).
Example with the rc.conf:
sysrc kld_list+="tcp_rack tcp_bbr"
Configure the system to use BBR TCP stack by default:
echo 'net.inet.tcp.functions_default=bbr' >> /etc/sysctl.conf
Reboot and check if the system is using the BBR TCP stack:
# sysctl net.inet.tcp.functions_default
net.inet.tcp.functions_default: bbr
Enable BBR logging for all TCP sessions:
# sysctl net.inet.tcp.bb.log_auto_mode=4
# sysctl net.inet.tcp.bb.log_auto_all=1
# sysctl net.inet.tcp.bb.log_auto_ratio=1
Start tcplog_dumper:
# mkdir /var/log/tcplog_dumps
# chown nobody /var/log/tcplog_dumps
# tcplog_dumper
For each new TCP sessions, there will be multiples .pcapng files in the log directory: You can use read_bbrlog to interpret those files.
TCP Logs can be extracted from FreeBSD kernel core dumps using the gdb plugin
provided in the kgdb
directory. An example usage assuming relevant kernel
builds and coredumps looks like:
$ kgdb kernel-debug/kernel.debug vmcore.last
GNU gdb (GDB) 13.2 [GDB v13.2 for FreeBSD]
Copyright (C) 2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-portbld-freebsd15.0".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from coredump/kernel-debug/kernel.debug...
Unread portion of the kernel message buffer:
KDB: enter: sysctl debug.kdb.enter
__curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:57
57 __asm("movq %%gs:%P1,%0" : "=r" (td) : "n" (offsetof(struct pcpu,
(kgdb) source tcplog.py
(kgdb) tcplog_dump vnet0
processing struct tcpcb * 0xfffff80006e8ca80
_t_logstate: 4 _t_logpoint: 0 '\000' t_lognum: 25 t_logsn: 25
log written to 0xfffff80006e8ca80_tcp_log.bin
processing struct tcpcb * 0xfffff8000ec2b540
_t_logstate: 4 _t_logpoint: 0 '\000' t_lognum: 8 t_logsn: 8
log written to 0xfffff8000ec2b540_tcp_log.bin
processing struct tcpcb * 0xfffff80006bd9540
no logs
processing struct tcpcb * 0xfffff80006bd9a80
no logs
processing struct tcpcb * 0xfffff8001d837540
no logs
processing struct tcpcb * 0xfffff8001d837000
no logs
processed 1 vnets, dumped 2 logs
0xfffff80006e8ca80_tcp_log.bin 0xfffff8000ec2b540_tcp_log.bin
The generated files can be given to tcplog_dumper to generate pcaps like so:
$ tcplog_dumper -s -f 0xfffff80006e8ca80_tcp_log.bin