Driver splits UDP packets larger than 730 bytes
juhaylinen opened this issue · comments
If I send UDP datagram with size 1000 to echo server and try to receive the datagram back, recvfrom() call returns packet size 730 bytes and consecutive call returns 270 bytes. The recvfrom() call should return complete datagram but the driver seems to split UDP packets.
You can use the example code from issue #12 to experiment. Just increase the buffer size to 1000 bytes (#define BUF_SIZE 1000).
Board: NUCLEO_F401RE
Shield: IDW01M1
Compiler: GCC_ARM
Driver version: v1.1.8
Not sure if this can be fixed since there seems to be some kind of FW limitation
https://github.com/ARMmbed/wifi-x-nucleo-idw01m1/blob/master/SPWFSAxx.cpp#L540
Ciao @juhaylinen,
unfortunately, you are absolutely right that this cannot be fixed at least with the current version of the module's FW, as the FW itself would split data packages into chunks of 730 bytes. The driver just uses this feature in order to decide when to check for asynchronous notifications while sending.
I'm not so familiar with standards, but am wondering if there is a kind of required max size for datagrams or if any size needs to be supported for compliancy. Do you know?
I was browsing a little bit around in the Internet regarding the question of maximum UDP size and found the following (which more or less summarizes this page regarding datagram size):
The maximum safe UDP payload is 508 bytes. This is a packet size of 576, minus the maximum 60-byte IP header and the 8-byte UDP header. Any UDP payload this size or smaller is guaranteed to be deliverable over IP (though not guaranteed to be delivered).
So, it seems as if your test is going beyond what the respective standard RFCs require.
Could you pls. comment on this?
@SeppoTakalo can you comment on max UDP payload size?
From IETF Official Internet Protocol Standards and section Internet Standards go to:
RFC1122 - Requirements for Internet Hosts - Communication Layers: 3.3.2 Reassembly
3.3.2 Reassembly
The IP layer MUST implement reassembly of IP datagrams.
We designate the largest datagram size that can be reassembled
by EMTU_R ("Effective MTU to receive"); this is sometimes
called the "reassembly buffer size". EMTU_R MUST be greater
than or equal to 576, SHOULD be either configurable or
indefinite, and SHOULD be greater than or equal to the MTU of
the connected network(s).
As a Ethernet connected device, you must be able to receive packets as big as can fit into one ethernet frame.
SHOULD be greater than or equal to the MTU of the connected network
And Ethernet MTU is 1500.
The limit 576 is something that ALL technologies must be able to handle. And it should be considered when writing UDP applications. It is written into IETF's Best Current Practice: Unicast UDP Usage Guidelines for Application Designers: 3.2. Message Size Guidelines
Also this splitting breaks the meaning of the datagram protocol.
For example RFC768 - User Datagram Protocol
IP Interface
-------------
The UDP module must be able to determine the source and destination
internet addresses and the protocol field from the internet header. One
possible UDP/IP interface would return the whole internet datagram
including all of the internet header in response to a receive operation.
Such an interface would also allow the UDP to pass a full internet
datagram complete with header to the IP to send. The IP would verify
certain fields for consistency and compute the internet header checksum.
The word datagram is poorly defined, but always assumed to be one complete frame.
RFC1594 - FYI on Questions and Answers
Answers to Commonly asked "New Internet User" Questions
Datagram
A self-contained, independent entity of data carrying
sufficient information to be routed from the source
to the destination computer without reliance on earlier
exchanges between this source and destination computer and
the transporting network.
As a Ethernet connected device, you must be able to receive packets as big as can fit into one ethernet frame.
SHOULD be greater than or equal to the MTU of the connected network
And Ethernet MTU is 1500.
What about Wi-Fi, is it the same size, etc.?
The word datagram is poorly defined, but always assumed to be one complete frame.
"Assumed" by whom?
Has this assumption ever been written down explicitly somewhere?
The limit 576 is something that ALL technologies must be able to handle. And it should be considered when writing UDP applications.
Does this - in your opinion - mean that @juhaylinen's UDP test application is going beyond what is actually guaranteed by the standards?
@SeppoTakalo, considering the last paragraph of my previous post, do you think we can close this issue?
No, we should not immediately close this. This is seriously violating RFCs
Regarding the MTU, for Ethernet and Wi-Fi it is 1500. Therefore it should be assumed that device should be able to handle that sized packets.
IPv4 required is 576 and for IPv6 it is 1280, but MTU is higher so it should dictate.
By using payload size 1200 we are not going over these limits. (IPv6 header 40 bytes, IPv4 20 bytes, UDP header 8 bytes. Total 1248).
This splitting of UDP payload is not allowed by any Internet standard. If device is not capable of operating over some specific limit, it should just drop the datagram and return ICMP error code. Payload should never be fragmented. The fragmentation and reassembly should never be visible on the user API.
For IPv6, we should always be able to expect 1280 packets to work. For IPv4, we are recommended to to Path MTU discovery if we go over 576, however all modern devices are currently capable of handling frame sizes required by IPv6.
I'm unsure of whether this limitation is coming from the driver implementation, or from the device itself. However, it should be studied whether it can be fixed in the driver.
It's a limitation of the module FW, so it can not be fixed in the driver.
The module FW splits payloads w/o giving itself any awareness of this fact to the user. As already said above, the driver just exploits this behavior to improve asynchronous notifications management, but it's indeed just a macro in the driver which could simply be changed to match with e.g. the Wi-Fi MTU.
Pls. let me know how you wish to proceed!
OK. So this is similar limitation than the #11
Either we leave it open forever, or add a note to README.md telling that effective MTU for this driver is 730 (+headers?).
Was this 730 the payload size, or packet size?
For TCP, it should not be so much visible. I believe the device adjusts its TCP MSS accordingly. So this would only be visible on UDP where datagrams are handed to the application layer.
I have added a point regarding this issue to the Known limitations section in README.md, citing basically what is concretely said in the respective reference manual.
The modification has already been uploaded to GitHub, so pls. give it a try.
@SeppoTakalo, what do you think, can we close the issue now?
Closing the issue.
@SeppoTakalo, pls. feel free to re-open if you do not agree.
Based on the discussion, isn't a firmware update a must?
The Wifi chip is in end-of-life status
- https://www.st.com/en/ecosystems/x-nucleo-idw01m1.html
- https://www.st.com/en/wireless-connectivity/spwf01sa.html
So don't expect FW updates.
The issue exists in the latest FW version 3.6 (released on June 21)