forward messages besides NLM_F_MULTI

[Florian Lehner]

Hi,
I've tried, to receive statistics from the conntrack subsystem.
Unfortunately, the first returned message has the flag NLM_F_MULTI. Because of this, there are only blocking recvmsg events and the actuall message is never returned.

Below, I've written a piece of code, which reproduces this issue:

package main

import (
	"fmt"

	"github.com/mdlayher/netlink"
	"github.com/mdlayher/netlink/nlenc"

	"golang.org/x/sys/unix"
)

func putExtraHeader(familiy, version uint8, resid uint16) []byte {
	buf := make([]byte, 2)
	nlenc.PutUint16(buf, resid)
	return append([]byte{familiy, version}, buf...)
}

func main() {
	con, err := netlink.Dial(unix.NETLINK_NETFILTER, nil)
	if err != nil {
		fmt.Println("Could not open socket to NETLINK_NETFILTER")
		return
	}

	data := putExtraHeader(unix.AF_UNSPEC, unix.NFNETLINK_V0, 0)

	req := netlink.Message{
		Header: netlink.Header{
			Type: netlink.HeaderType( 0x1 << 8 | 5),
			Flags: netlink.HeaderFlagsRequest | netlink.HeaderFlagsDump,
		},
		Data: data,
	}
	con.Execute(req)
}

If you run this peace of code with strace, you see the request is sent to the kernel and then follows a blocking recvmsg event.

[Florian Lehner]

@ti-mo gave me the hint, to replace netlink.HeaderFlagsDump with netlink.HeaderFlagsAcknowledge. This way, the kernel returns only one value and not a series of messages, starting with a NLM_F_MULTI flaged message. 👍

As a result, maybe it would be an idea, to provide an call to get messages directly?

[Timo Beckers]

@florianl What change are you suggesting?

The call you're running in particular (I assume IPCTNL_MSG_CT_GET_STATS by the looks of it) returns, by definition, a multipart response (https://github.com/torvalds/linux/blob/v4.16/net/netfilter/nf_conntrack_netlink.c#L2115). Because it's a query that only has a single response attribute implemented (CTA_STATS_GLOBAL_ENTRIES), with the possibility of being extended, you need to signal to the underlying (netlink) layer to follow up with an ack, or it will expect the next message until it sees an ack. This behaviour should be altered by the dump flag, but the function I linked obeys no such flag, presumably because it only knows a single type.

I don't see this as a bug in the library; this is a kernel oddity (one of many), and reading the source is the only way to write client code for it. This call's sibling, IPCTNL_MSG_CT_GET_STATS_CPU, does not suffer from this behaviour. 🙂

[Florian Lehner]

You've spotted IPCTNL_MSG_CT_GET_STATS correctly. Since kernel 4.18, you get two values [1] with this request.
I also don't see it as a bug of netlink. It's more like an idea to forward messages from the kernel directly, instead of handling them all, if the NLM_F_MULTI flag is set.

[1] https://github.com/torvalds/linux/blob/v4.18/net/netfilter/nf_conntrack_netlink.c#L2188