3.0 version of HLS generates a large number of TS slice files.
baiyu0820 opened this issue · comments
Description
Version 3.0 generates a large number of ts slice files in HLS. Version 2.6 does not have this issue.
1. Operating System: CentOS 7
Environment
Streaming command: C:\MinGW\msys\1.0\bin>ffmpeg.exe -re -i c:/1.mp4 -vcodec libx264 -acodec aac -f
flv rtmp://192.168.0.39:1935/live/b1
Logs: srs.log
Configuration file: srs.conf.txt
Video file (28mb): https://pan.baidu.com/s/1hPu46MmYOllJe1C0AXiM8g (Password: 5vdm)
Image: 
TRANS_BY_GPT3
Your configuration is set to not clean up HLS, so of course there will be a lot of TS files.
hls_dispose 0;
hls_cleanup off;
TRANS_BY_GPT3
Your stream will generate a large number of small TS files, with a duration exceeding 100 seconds, which is also a problem.
Mac:live chengli.ycl$ ffmpeg -i livestream-3899.ts
Input #0, mpeg, from 'livestream-3899.ts':
Duration: 00:00:59.63, start: 157.570611, bitrate: 0 kb/s
Stream #0:0[0x1c0]: Audio: mp2, 0 channels, s16p
Stream #0:1[0x1e0]: Video: none, none, 90k tbr, 90k tbn
TRANS_BY_GPT3
The fourth one is abnormal.
-rw-r--r-- 1 chengli.ycl staff 185932 Dec 5 18:03 livestream-4.ts
ffmpeg -i livestream-4.ts
ffmpeg version 3.2.4 Copyright (c) 2000-2017 the FFmpeg developers
[h264 @ 0x7fc072004200] non-existing PPS 0 referenced
Last message repeated 1 times
[h264 @ 0x7fc072004200] decode_slice_header error
[h264 @ 0x7fc072004200] no frame!
[h264 @ 0x7fc072004200] non-existing PPS 0 referenced
Last message repeated 1 times
[h264 @ 0x7fc072004200] decode_slice_header error
[h264 @ 0x7fc072004200] no frame!
[h264 @ 0x7fc072004200] non-existing PPS 0 referenced
Last message repeated 1 times
[h264 @ 0x7fc072004200] decode_slice_header error
[h264 @ 0x7fc072004200] no frame!
[h264 @ 0x7fc072004200] non-existing PPS 0 referenced
Last message repeated 1 times
[h264 @ 0x7fc072004200] decode_slice_header error
[h264 @ 0x7fc072004200] no frame!
Input #0, mpegts, from 'livestream-4.ts':
Duration: 00:00:16.99, start: 33.994011, bitrate: 87 kb/s
Program 1
Stream #0:0[0x101]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 32000 Hz, stereo, fltp, 132 kb/s
Stream #0:1[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuvj420p(pc, bt709, progressive), 1920x1080, 90k tbr, 90k tbn, 180k tbc
If audio is disabled and only video is pushed, there is no problem.
hls {
enabled on;
hls_acodec an;
}
ffmpeg -re -i 1.mp4 -vcodec copy -an -f flv -y rtmp://127.0.0.1/live/livestream
Mac:live chengli.ycl$ ffmpeg -i livestream-4.ts
ffmpeg version 3.2.4 Copyright (c) 2000-2017 the FFmpeg developers
Input #0, mpegts, from 'livestream-4.ts':
Duration: 00:00:10.97, start: 44.016000, bitrate: 221 kb/s
Program 1
Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuvj420p(pc, bt709, progressive), 1920x1080, 90k tbr, 90k tbn, 180k tbc
So this is definitely related to audio. The audio packet is over 500 bytes, which doesn't seem like a normal AAC packet. SRS uses the size of AAC packets to calculate timestamps, which makes it more accurate and avoids the problem of inaccurate timestamps causing audio distortion. Reference: #547 (comment)
If it is changed to pure audio HLS, it will be easier to troubleshoot.
hls {
enabled on;
hls_vcodec vn;
}
ffmpeg -re -i 1.mp4 -acodec copy -vn -f flv -y rtmp://127.0.0.1/live/livestream
Debugging the received data, the first one is the audio sequence header encoding header, and the second packet is audio data with 501 bytes.
$4 = timestamp = 0, stream_id = 1, size = 501
The third audio packet is 48 milliseconds and has 522 bytes.
timestamp = 48, stream_id = 1, size = 522
Generally, AAC has a sampling rate of 44100Hz, and one frame contains 1024 samples, so one frame is usually 23 milliseconds.
1024/44100.0=0.02321995s=23.21995ms
However, this stream is 48 milliseconds, so it should contain 2048 samples, which is larger than the usual. Due to the previous logic being fixed at 1024 samples or 960 samples to calculate the timestamp, it resulted in incorrect calculations.
// Use the diff to guess whether the samples is 1024 or 960.
int nb_samples_per_frame = 1024;
if (diff > 100 && diff < 950) {
nb_samples_per_frame = 960;
}
// Recalc the DTS by the samples of AAC.
int64_t dts = 90000 * aac_samples / srs_flv_srates[format->acodec->sound_rate];
aac_samples += nb_samples_per_frame;
After modifying the guess algorithm for the number of samples, the pure audio slicing is working correctly. Please refer to 31f341e for more information.
-rw-r--r--@ 1 chengli.ycl staff 456840 Dec 5 20:09 livestream-1.ts
-rw-r--r-- 1 chengli.ycl staff 456840 Dec 5 20:10 livestream-2.ts
-rw-r--r-- 1 chengli.ycl staff 456652 Dec 5 20:10 livestream-3.ts
-rw-r--r-- 1 chengli.ycl staff 457028 Dec 5 20:11 livestream-4.ts
-rw-r--r-- 1 chengli.ycl staff 456652 Dec 5 20:11 livestream-5.ts
After the modification, there are still abnormalities when audio and video are together, and the slices are getting smaller and smaller.
-rw-r--r-- 1 chengli.ycl staff 112K Dec 5 20:14 livestream-4.ts
-rw-r--r-- 1 chengli.ycl staff 223K Dec 5 20:14 livestream-5.ts
-rw-r--r-- 1 chengli.ycl staff 90K Dec 5 20:14 livestream-6.ts
-rw-r--r-- 1 chengli.ycl staff 266K Dec 5 20:14 livestream-7.ts
-rw-r--r-- 1 chengli.ycl staff 45K Dec 5 20:14 livestream-8.ts
-rw-r--r-- 1 chengli.ycl staff 1.1K Dec 5 20:15 livestream-499.ts
Debugging to retrieve two audio packets.
timestamp = 102304, stream_id = 1, size = 501
diff=32, nb_samples_per_frame=1024, aac_samples=3416704, dts=77499.5ms
timestamp = 102336, stream_id = 1, size = 504
diff=32, nb_samples_per_frame=1024, aac_samples=3418752, dts=77522ms
The timestamp of the initial AAC packet was 48ms, but later it changed to 32ms, which caused incorrect calculation of the audio's DTS. It seems that estimating the timestamp based on the AAC sample size is not accurate. It would be more reasonable to calculate it directly based on the timestamp. A configuration option needs to be added for this. Please refer to: b4870a6.
hls {
enabled on;
hls_dts_directly on;
}
-rw-r--r-- 1 chengli.ycl staff 488K Dec 5 20:44 livestream-7.ts
-rw-r--r-- 1 chengli.ycl staff 491K Dec 5 20:45 livestream-8.ts
-rw-r--r-- 1 chengli.ycl staff 489K Dec 5 20:45 livestream-9.ts
Please note that you need to update to SRS 3.0.68 in order to use this configuration.
Improvement measures:
- The number of samples can be determined by the sampling rate and the difference between frames, instead of fixedly setting the sample size to 1024.
- Provide a configuration option to disable the functionality of calculating timestamps based on the AAC sample count, and calculate them directly based on the package timestamps.
TRANS_BY_GPT3
When the video is not encoded in H.264, it will cause SRS to crash.
hls {
enabled on;
hls_acodec an;
}
ffmpeg -re -i 1.mp4 -acodec copy -f flv -y rtmp://127.0.0.1/live/livestream
In fact, this is actually encoding the video as H.263, which causes the codec to be unable to parse it.
srs_error_t SrsFormat::on_video(int64_t timestamp, char* data, int size)
{
......
if (codec_id != SrsVideoCodecIdAVC) {
return err;
}
As a result, format->vcodec and format-video are both NULL, triggering an assert error.
srs_error_t SrsHls::on_video(SrsSharedPtrMessage* shared_video, SrsFormat* format)
{
......
srs_assert(format->video);
Fixed in 52c6c7c
TRANS_BY_GPT3