RTMP acknowledge overflow causes the encoder to stop streaming
winlinvip opened this issue · comments
ACK SIZE refers to the number of bytes that the PEER should send an Acknowledge message after receiving. In other words, it indicates the number of bytes that the PEER has received.
RTMP is 32, which means it is 4 bytes long, but it does not explicitly state how overflow should be handled, whether it is int32 or uint32.
5.3. Acknowledgement (3)
The client or the server sends the acknowledgment to the peer after
receiving bytes equal to the window size. The window size is the
maximum number of bytes that the sender sends without receiving
acknowledgment from the receiver. The server sends the window size to
the client after application connects. This message specifies the
sequence number, which is the number of the bytes received so far.
0 1 2 3
01234567890123456789012345678901
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| sequence number (4 byte) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 4 shows the payload for the protocol message 'Acknowledgement'.
sequence number: 32 bits
This field holds the number of bytes received so far.
Previously, the bitrate of videos was very low, such as 100kbps, so there would be no overflow issue with these 4 bytes. Now, the bitrate is 80 to 100 times higher, which easily leads to overflow. Some encoders have different rules for handling this overflow, causing them to receive incorrect values and mistakenly assume that the server has not acknowledged, resulting in re-streaming.
TRANS_BY_GPT3
Gao Sheng has encountered something similar, but did not explain it clearly: #598
TRANS_BY_GPT3
Play a stream with a bitrate of 8Mbps using Flash player, set the ack size to 10*1024*1024, and print out the acknowledge messages sent by the Flash player.
`ack=1` indicates an acknowledge message.
`seq` is the sequence number, which represents the number of bytes received by the client.
`sent` is the number of bytes displayed as transmitted by the server.
[2017-01-06 11:23:30.321][warn][13607][114][35] ack=1, seq=0x500ac2, sent=0x5766fb, diff=482361
[2017-01-06 11:23:35.493][warn][13607][114][35] ack=1, seq=0xa0192a, sent=0xa0f73a, diff=56848
[2017-01-06 11:23:39.153][warn][13607][114][35] ack=1, seq=0xf0229e, sent=0xf4e6bb, diff=312349
[2017-01-06 11:23:43.197][warn][13607][114][35] ack=1, seq=0x140241e, sent=0x140499e, diff=9600
The ack size is set to 10*1024*1024, which means an acknowledge message should be sent after the player has played 10MB of data.
For a flash player, the interval between sending messages, which is the difference in seq, is 0xa0192a-0x500ac2=5246568=5*1024*1024, which means it sends approximately half of the ack size.
Subsequently, it sends acknowledgements after receiving 5.002307891845703MB and 5.0003662109375MB, which is roughly half of the window size.
The following messages are:
[2017-01-06 11:50:16.010][warn][13607][114][35] ack=1, seq=0x68ba6930, sent=0x68c14790, diff=450144
[2017-01-06 11:50:20.063][warn][13607][114][35] ack=1, seq=0x690a7297, sent=0x690d58ae, diff=189975
[2017-01-06 11:50:23.727][warn][13607][114][35] ack=1, seq=0x695a79a2, sent=0x695dafa8, diff=210438
[2017-01-06 11:50:28.507][warn][13607][114][35] ack=1, seq=0x69aa7fe2, sent=0x69b1d0f1, diff=479503
The respective values are: 5.00229549407959, 5.0017194747924805, 5.00152587890625.
Therefore, it can be confirmed that the behavior of the flash player is to send an acknowledge confirmation message to the server when it receives more than 1/2 of the ack size (window size).
TRANS_BY_GPT3
Regarding the server side, my colleague has tested two of them:
FMS: 0xee77d0a7
NGINX-RTMP: 0xf0000000
Once these two thresholds are exceeded, the server will reset the counter to 0.
NGINX-RTMP code: https://github.com/arut/nginx-rtmp-module/blob/c0bf381d10de05c135f913921c58272838d5e1ee/ngx_rtmp_handler.c#L275
if (s->in_bytes >= 0xf0000000) {
ngx_log_debug0(NGX_LOG_DEBUG_RTMP, c->log, 0,
"resetting byte counter");
s->in_bytes = 0;
s->in_last_ack = 0;
}
This number has no basis and there is no comment explaining its source.
In nginx-rtmp, the ack is sent when it exceeds the ack window (unlike Flash, which sends it when it exceeds half the window).
if (s->ack_size && s->in_bytes - s->in_last_ack >= s->ack_size) {
s->in_last_ack = s->in_bytes;
ngx_log_debug1(NGX_LOG_DEBUG_RTMP, c->log, 0,
"sending RTMP ACK(%uD)", s->in_bytes);
if (ngx_rtmp_send_ack(s, s->in_bytes)) {
ngx_rtmp_finalize_session(s);
return;
}
}
TRANS_BY_GPT3
About the handling of flash player overflow, the log is as follows:
[2017-01-06 12:24:16.129][warn][13607][114][35] ack=1, seq=0xed47b9cd, sent=0xed49da95, diff=139464
[2017-01-06 12:24:20.549][warn][13607][114][35] ack=1, seq=0xed97c0e6, sent=0xed992be4, diff=92926
[2017-01-06 12:24:26.044][warn][13607][114][35] ack=1, seq=0xede7c793, sent=0xeded7128, diff=371093
[2017-01-06 12:24:29.706][warn][13607][114][35] ack=1, seq=0xee37cff6, sent=0xee3d82a9, diff=373427
[2017-01-06 12:24:34.113][warn][13607][114][35] ack=1, seq=0xee87dca3, sent=0xee90a823, diff=576384
[2017-01-06 12:24:37.781][warn][13607][114][35] ack=1, seq=0xeed7e36a, sent=0xeedd0cbf, diff=338261
[2017-01-06 12:24:42.201][warn][13607][114][35] ack=1, seq=0xef27ed1a, sent=0xef2e1b6d, diff=405075
[2017-01-06 12:24:46.589][warn][13607][114][35] ack=1, seq=0xef77f9ad, sent=0xef7d5424, diff=350839
[2017-01-06 12:24:51.016][warn][13607][114][35] ack=1, seq=0xefc806b8, sent=0xefc8896a, diff=33458
[2017-01-06 12:24:55.432][warn][13607][114][35] ack=1, seq=0x500eb7, sent=0xf01a1a8a, diff=-271971373
[2017-01-06 12:25:00.178][warn][13607][114][35] ack=1, seq=0xa01113, sent=0xf06da792, diff=-271739265
[2017-01-06 12:25:04.577][warn][13607][114][35] ack=1, seq=0xf0186e, sent=0xf0bc6d49, diff=-271821605
[2017-01-06 12:25:09.373][warn][13607][114][35] ack=1, seq=0x140210b, sent=0xf10fe9e5, diff=-271595302
[2017-01-06 12:25:12.673][warn][13607][114][35] ack=1, seq=0x19021c6, sent=0xf15a736a, diff=-271953500
[2017-01-06 12:25:17.435][warn][13607][114][35] ack=1, seq=0x1e0287d, sent=0xf1aa3d1b, diff=-271969122
[2017-01-06 12:25:22.213][warn][13607][114][35] ack=1, seq=0x2302f34, sent=0xf1f9d01b, diff=-271998745
[2017-01-06 12:25:26.996][warn][13607][114][35] ack=1, seq=0x2803a9b, sent=0xf24c3340, diff=-271845211
[2017-01-06 12:25:31.382][warn][13607][114][35] ack=1, seq=0x2d03ed4, sent=0xf299ecd1, diff=-271995395
[2017-01-06 12:25:36.534][warn][13607][114][35] ack=1, seq=0x3204cee, sent=0xf2f33cf8, diff=-271388662
[2017-01-06 12:25:40.566][warn][13607][114][35] ack=1, seq=0x37053d6, sent=0xf33de12b, diff=-271741611
[2017-01-06 12:25:45.332][warn][13607][114][35] ack=1, seq=0x3c05a44, sent=0xf38b265b, diff=-271922153
The overflowing data is padded with either 32 bits or 31 bits, which is significantly different from what flash sends. If we directly set it to zero, it can match, just like in nginx-rtmp, where it is set to zero when it exceeds 0xf0000000.
TRANS_BY_GPT3
This problem has only two solutions:
- If the accumulation of
acknowledgeexceeds0xf0000000, it is set to zero. - Send an
acknowledgemessage if it exceeds half of theack size.
Supported in version 2.0.225.
TRANS_BY_GPT3
In addition, some encoders do not send an ack size (usually indicating that they do not need to send an acknowledge confirmation message), but they still want the server to send a confirmation message. Therefore, it is necessary to support configuration that can change the ack size for both input and output.
Supported in 3.0.13.
TRANS_BY_GPT3