ospfd requires cap_sys_admin
javier-godoy opened this issue · comments
I had previously used quagga 1.2.4-r2 (particularly ospfd) in a docker container based on alpine 3.11.3
When trying to migrate to frr-7.3.1-r0 in a docker container based on alpine 3.12.3 (binaries installed with apk) I found that ospfd and zebra fail to initialize, with the following message:
privs_init: initial cap_set_proc failed: Operation not permitted
Wanted caps: = cap_net_admin,cap_net_raw,cap_sys_admin+p
Have caps: = cap_chown,cap_dac_override,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_net_bind_service,cap_net_admin,cap_net_raw,cap_sys_chroot,cap_mknod,cap_audit_write,cap_setfcap+ip
I'm not granting cap_sys_admin to the container, and I would like to avoid it if possible (Quagga's ospfd worked fine with only cap_net_admin and cap_net_raw).
I seem to understand that cap_sys_admin was added to FRR in #1818
and the workaround from #2007 is not possible since the container is not granted such permission.
Is there any way to avoid the check for cap_sys_admin?
- Did you check if this is a duplicate issue?
- Did you test it on the latest FRRouting/frr master branch?
We agree with you that minimum privileges is good practice. Unfortunately we need this cap in order to be able to switch network namespaces for namespace based VRFs. From setns(2):
Network, IPC, time, and UTS namespaces
In order to reassociate itself with a new network, IPC, time, or UTS namespace,
the caller must have the CAP_SYS_ADMIN capability both in its own user namespace
and in the user namespace that owns the target namespace.
Quagga didn't have this feature and so didn't require this capability.
I'll note that we only elevate privileges when necessary and drop them when done (you can grep for frr_with_privs(...) {...} blocks if you want to see how it's done).
Of course in theory we could make needing CAP_SYS_ADMIN be a thing thats only compiled in when you compile in namespace VRFs, but nobody's done the work for that and it's unlikely to be done given that it works as is. Sorry for the inconvenience. If you have a patch for it we can definitely work with you to get that in.
Thanks for your quick response. For now I'll recompile with --disable-capabilities (since I'm already restricting the capabilities of the docker container that runs FRR). Based on your explanation I understand that it should work since my usage of FRR doesn't require CAP_SYS_ADMIN.
Once I'm certain that it works as intended I'll submit a PR. My approach would be adding a --disable-cap-sys-admin configuration and initializing _caps_p with or without ZCAP_SYS_ADMIN depending on it (I think there is no configuration flag for compile in namespace VRFs). Would that be OK for you?
There is already a flag named '--vrfwnetns' that, if not present, should be used to disable cap-sys-admin wherever needed.
this flag is part of zebra, and zebra should send information via zapi to other daemons to inform that the backend is not vrfwnetns.
arent you in the case where you are using BGP in standalone mode as route reflector ?
@pguibert6WIND I'm not starting zebra with --vrfwnetns, and it fails immediately after starting. I think the failure is because the capability isn't permitted, and not an actual attempt to raise it.
Lines 312 to 315 in 58ba064
arent you in the case where you are using BGP in standalone mode as route reflector ?
I'm not using BGP, just OSPF.
My approach would be adding a --disable-cap-sys-admin configuration and initializing _caps_p with or without ZCAP_SYS_ADMIN depending on it
Sorry for the delay. Yeah, that sounds appropriate.
@pguibert6WIND I'm not starting zebra with
--vrfwnetns, and it fails immediately after starting. I think the failure is because the capability isn't permitted, and not an actual attempt to raise it.Lines 312 to 315 in 58ba064
arent you in the case where you are using BGP in standalone mode as route reflector ?
I'm not using BGP, just OSPF.
I meant, you don't need a new flag.
if you dont use vrfwnetns, frr should default to disable cap sys admin
Please see https://github.com/javier-godoy/frr-alpine/blob/test/init (using FRR 7.5.1 from Alpine 3.13.5)
chown -R frr:frr /etc/frr
/usr/lib/frr/zebra -v
/usr/lib/frr/zebra &
/usr/lib/frr/ospfd &
sleep 1
ps aux
When only NET_ADMIN is granted, zebra and ospfd fail with the following log and the processes are not started.
Note that it wants cap_sys_admin=p (not cap_sys_admin itself, but only as a permitted capability, I guess in order to effectively acquire it if/when needed). In this example init is the one who doesn't have cap_sys_admin=p so it cannot grant it to its children, even if they are never going to raise it as an effective capability.
Attaching to frrtest_testfrr_1
testfrr_1 | chown -R frr:frr /etc/frr
testfrr_1 | /usr/lib/frr/zebra -v
testfrr_1 | zebra version 7.5.1
testfrr_1 | Copyright 1996-2005 Kunihiro Ishiguro, et al.
testfrr_1 | configured with:
testfrr_1 | '--prefix=/usr' '--enable-exampledir=/usr/share/doc/frr/examples/' '--localstatedir=/run/frr' '--sbindir=/usr/lib/frr' '--sysconfdir=/etc/frr' '--libdir=/usr/lib/frr' '--with-moduledir=/usr/lib/frr/modules' '--disable-dependency-tracking' '--enable-systemd=no' '--enable-rpki' '--with-libpam' '--enable-doc' '--enable-doc-html' '--enable-snmp' '--enable-fpm' '--disable-protobuf' '--disable-zeromq' '--enable-ospfapi' '--enable-bgp-vnc' '--enable-multipath=256' '--enable-user=frr' '--enable-group=frr' '--enable-vty-group=frrvty' '--enable-configfile-mask=0640' '--enable-logfile-mask=0640' 'CC=gcc' 'CXX=g++' 'PYTHON=python3'
testfrr_1 | /usr/lib/frr/zebra &
testfrr_1 | /usr/lib/frr/ospfd &
testfrr_1 | sleep 1
testfrr_1 | privs_init: initial cap_set_proc failed: Operation not permitted
testfrr_1 | Wanted caps: cap_net_admin,cap_net_raw,cap_sys_admin=p
testfrr_1 | Have caps: cap_chown,cap_dac_override,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_net_bind_service,cap_net_admin,cap_net_raw,cap_sys_chroot,cap_mknod,cap_audit_write,cap_setfcap=ip
testfrr_1 | privs_init: initial cap_set_proc failed: Operation not permitted
testfrr_1 | Wanted caps: cap_net_bind_service,cap_net_admin,cap_net_raw,cap_sys_admin=p
testfrr_1 | Have caps: cap_chown,cap_dac_override,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_net_bind_service,cap_net_admin,cap_net_raw,cap_sys_chroot,cap_mknod,cap_audit_write,cap_setfcap=ip
testfrr_1 | ps aux
testfrr_1 | PID USER TIME COMMAND
testfrr_1 | 1 root 0:00 /sbin/tini -- init
testfrr_1 | 6 root 0:00 {init} /bin/sh /usr/local/bin/init
testfrr_1 | 12 root 0:00 ps aux
testfrr_1 | tail -f /dev/null
When both NET_ADMIN and SYS_ADMIN are granted, then zebra and ospfd start correctly:
Attaching to frrtest_testfrr_1
testfrr_1 | chown -R frr:frr /etc/frr
testfrr_1 | /usr/lib/frr/zebra -v
testfrr_1 | /usr/lib/frr/zebra &
testfrr_1 | /usr/lib/frr/ospfd &
testfrr_1 | sleep 1
testfrr_1 | zebra version 7.5.1
testfrr_1 | Copyright 1996-2005 Kunihiro Ishiguro, et al.
testfrr_1 | configured with:
testfrr_1 | '--prefix=/usr' '--enable-exampledir=/usr/share/doc/frr/examples/' '--localstatedir=/run/frr' '--sbindir=/usr/lib/frr' '--sysconfdir=/etc/frr' '--libdir=/usr/lib/frr' '--with-moduledir=/usr/lib/frr/modules' '--disable-dependency-tracking' '--enable-systemd=no' '--enable-rpki' '--with-libpam' '--enable-doc' '--enable-doc-html' '--enable-snmp' '--enable-fpm' '--disable-protobuf' '--disable-zeromq' '--enable-ospfapi' '--enable-bgp-vnc' '--enable-multipath=256' '--enable-user=frr' '--enable-group=frr' '--enable-vty-group=frrvty' '--enable-configfile-mask=0640' '--enable-logfile-mask=0640' 'CC=gcc' 'CXX=g++' 'PYTHON=python3'
testfrr_1 | 2021/06/04 00:45:06 ZEBRA: [EC 4043309111] Disabling MPLS support (no kernel support)
testfrr_1 | ps aux
testfrr_1 | PID USER TIME COMMAND
testfrr_1 | 1 root 0:00 /sbin/tini -- init
testfrr_1 | 6 root 0:00 {init} /bin/sh /usr/local/bin/init
testfrr_1 | 9 frr 0:00 /usr/lib/frr/zebra
testfrr_1 | 10 frr 0:00 /usr/lib/frr/ospfd
testfrr_1 | 15 root 0:00 ps aux
testfrr_1 | tail -f /dev/null
Yes, I'm not sure what Phillipe meant: several daemons unconditionally request CAP_SYS_ADMIN currently.
I think this issue shouldn't be limited to ospfd and also cover other daemons (zebra, bgpd, …).
I'm trying to setup a simple FRR instance as a BGP speaker (publish routes to load balancer and don't need to handle any incoming routes), and have the same issue.
Yes, I'm not sure what Phillipe meant: several daemons unconditionally request CAP_SYS_ADMIN currently.
I was thinking the capabilities were requested by vrfwnetns mode, whereas this is the container that requests it.
I think this issue shouldn't be limited to ospfd and also cover other daemons (zebra, bgpd, …).
I'm trying to setup a simple FRR instance as a BGP speaker (publish routes to load balancer and don't need to handle any incoming routes), and have the same issue.
As I am just experiment, I use --privileged to run docker container and no error