How to use headroom in UMEM frame?

[ZhalgasFromMSU]

Hi!

I want to write an application that transmits packets and I would like to set a headroom when creating UMEM.

It is written on kernel.org:

There is also an option to set the headroom of each single buffer in the UMEM. If you set this to N bytes, it means that the packet will start N bytes into the buffer leaving the first N bytes for the application to use.

Imagine that I want to store size of a packet along with its data in UMEM. So, when creating UMEM, I also pass frame_headroom = sizeof(unsigned int). I assume that headroom is first sizeof(unsigned int) bytes in each frame in UMEM, so if I wanted to store packet (located at data) with size size at the very beginning of UMEM buffer (located at umem_buffer), I would do it like this:

/*
|---headroom---|---rest-of-frame---|
|-----size-----|---payload---|
*/
memcpy(umem_buffer, &size, sizeof(size)); // size is unsigned int
memcpy((char*)umem_buffer + sizeof(size), data, size);

Now I want to send this packet. I would assume that my kernel already knows about headroom offsets, so I do it like this:

xdp_desc* desc = xsk_ring_prod__tx_desc(&tx_ring, tx_entry);
desc->addr = 0;
desc->len = size;
desc->options = 0;

Contrary to my assumptions, kernel sends data in frame headroom along with the rest of packet, truncating last sizeof(unsigned int) bytes of packet payload. In other words, it sends exactly size bytes, which include headroom and part of packet that fits into size - sizeof(size) bytes.

I tried to change desc->addr to point to actual beginning of packet payload (that is desc->addr = 0 + sizeof(size)), but that way packets are not sent at all (I assume that is because I didn't set unaligned UMEM mode).

How do I use the headroom properly? I only want to send actual packet payload

[Magnus Karlsson]

You have to set desc->addr to the start of what you would like to send. It does not care about any frame_headroom. In fact, this is only for the Rx part in which case an extra X bytes (on top of the mandatory 256 bytes of headroom for XDP) is added to the headroom of the packet.

So you do not have to use the frame_headroom option at all. Just put your packet where you want it in the umem and make sure it does not cross a chunk boundary (as you are not using the unaligned mode). I would expect that it would work in your example too unless you are not within a chunk. Please try again without the frame_headroom and check if it works. Try sending something simple and small first.

[ZhalgasFromMSU]

Hi!

Thanks for the reply, that was very useful.

Regarding starting address of a packet in UMEM frame: I tried to send packet that is not aligned with start of UMEM frame, and it was sent successfully. It seems like frame_headroom option somehow messes up this process.

Nevertheless, I got my answer, I think that issue can be closed