HLS: SRS2 processes audio-only streams into HLS format, experiencing lag and static noise issues on Apple systems.
lewang321 opened this issue · comments
When slicing the pure audio stream with srs_2.0 into HLS, if the configuration option hls_vcodec vn; hls_acodec aac; is selected, there will be no video information in the PMT of ts. However, when playing the audio using Apple iOS system, there will be stuttering and noise, while it plays normally on Android system.
hls.conf configuration file:
listen 1935;
max_connections 1000;
vhost __defaultVhost__ {
hls {
enabled on;
hls_fragment 5;
hls_td_ratio 1.5;
hls_aof_ratio 1.0;
hls_on_error ignore;
hls_storage disk;
# hls_ts_floor off;
hls_acodec aac;
hls_vcodec vn;
# hls_cleanup on;
# hls_nb_notify 64;
# hls_wait_keyframe off;
hls_window 60;
hls_path ./objs/nginx/html;
hls_m3u8_file [app]/[stream].m3u8;
hls_ts_file [app]/[stream]-[seq].ts;
}
TRANS_BY_GPT3
Topology is RTMP ----> SRS -----> IOS.
TRANS_BY_GPT3
The RTMP stream is pushed to SRS through FMLE. It is in the pure audio AAC format.
TRANS_BY_GPT3
What is the sampling rate of the SRS logs and audio?
TRANS_BY_GPT3
[2015-10-27 10:08:08.715][trace][26340][0] srs checking config...
[2015-10-27 10:08:08.715][trace][26340][0] detect intranet address: 127.0.0.1, ifname=lo
[2015-10-27 10:08:08.715][trace][26340][0] retrieve local ipv4 ip=10.12.4.61, index=0
[2015-10-27 10:08:08.716][trace][26340][0] detect intranet address: 10.12.4.61, ifname=eth0
[2015-10-27 10:08:08.716][trace][26340][0] retrieve local ipv4 ip=172.17.42.1, index=1
[2015-10-27 10:08:08.716][trace][26340][0] detect intranet address: 172.17.42.1, ifname=docker0
[2015-10-27 10:08:08.716][warn][26340][0][0] stats network use index=0, ip=10.12.4.61
[2015-10-27 10:08:08.716][warn][26340][0][0] stats disk not configed, disk iops disabled.
[2015-10-27 10:08:08.716][trace][26340][0] write log to file ./objs/srs.log
[2015-10-27 10:08:08.716][trace][26340][0] you can: tailf ./objs/srs.log
[2015-10-27 10:08:08.716][trace][26340][0] @see: https://github.com/simple-rtmp-server/srs/wiki/v1_CN_SrsLog
[2015-10-27 10:08:08.716][trace][26340][0] srs(simple-rtmp-server) 2.0.195
[2015-10-27 10:08:08.716][trace][26340][0] license: The MIT License (MIT), Copyright (c) 2013-2015 SRS(simple-rtmp-server)
[2015-10-27 10:08:08.716][trace][26340][0] primary/master: SRS/1.0release
[2015-10-27 10:08:08.716][trace][26340][0] authors: winlin,wenjie.zhao
[2015-10-27 10:08:08.716][trace][26340][0] contributors: winlin<winlin@vip.126.com> wenjie.zhao<740936897@qq.com> xiangcheng.liu<liuxc0116@foxmail.com> naijia.liu<youngcow@youngcow.net> alcoholyi<alcoholyi@qq.com> byteman<wangchen2011@gmail.com> chad.wang<chad.wang.cn@gmail.com> suhetao<suhetao@gmail.com> Johnny<fengjihu@163.com> karthikeyan<keyanmca@gmail.com> StevenLiu<lq@chinaffmpeg.org> zhengfl<zhengfl_1989@126.com> tufang14<breadbean1449@gmail.com> allspace<allspace@gmail.com> niesongsong<nie950@gmail.com> rudeb0t<nimrod@themanxgroup.tw>
[2015-10-27 10:08:08.716][trace][26340][0] uname: Linux MediaWiki 2.6.32-504.el6.x86_64 #1 SMP Wed Oct 15 04:27:16 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
[2015-10-27 10:08:08.716][trace][26340][0] build: 2015-10-26 16:49:02, little-endian
[2015-10-27 10:08:08.716][trace][26340][0] configure: --disable-all --with-ssl --with-hls --with-nginx --with-ffmpeg --with-transcode
[2015-10-27 10:08:08.716][trace][26340][0] features: --prefix=/usr/local/srs --with-hls --without-hds --without-dvr --with-nginx --with-ssl --with-ffmpeg --with-transcode --without-ingest --without-stat --without-http-callback --without-http-server --without-stream-caster --without-http-api --without-librtmp --without-research --without-utest --without-gperf --without-gmc --without-gmp --without-gcp --without-gprof --without-arm-ubuntu12 --without-mips-ubuntu12 --log-trace
[2015-10-27 10:08:08.716][trace][26340][0] conf: conf/hls.conf, limit: 1000
[2015-10-27 10:08:08.716][trace][26340][0] writev limits write 1024 iovs a time
[2015-10-27 10:08:08.716][warn][26340][0][0] SRS 2.0.195 is not stable, please use stable branch 1.0release instead
[2015-10-27 10:08:08.716][trace][26340][0] cpu is amd64, glibc 2.12
[2015-10-27 10:08:08.716][trace][26340][0] cpu is x86_64, glibc 2.12
[2015-10-27 10:08:08.716][trace][26340][0] check feature rtmp handshake: on
[2015-10-27 10:08:08.716][trace][26340][0] check feature hls: on
[2015-10-27 10:08:08.716][warn][26340][0][0] check feature hds: off
[2015-10-27 10:08:08.716][warn][26340][0][0] check feature http callback: off
[2015-10-27 10:08:08.716][warn][26340][0][0] check feature http api: off
[2015-10-27 10:08:08.716][warn][26340][0][0] check feature http server: off
[2015-10-27 10:08:08.716][warn][26340][0][0] check feature http parser: off
[2015-10-27 10:08:08.716][warn][26340][0][0] check feature dvr: off
[2015-10-27 10:08:08.716][trace][26340][0] check feature transcode: on
[2015-10-27 10:08:08.716][warn][26340][0][0] check feature ingest: off
[2015-10-27 10:08:08.716][warn][26340][0][0] check feature system stat: off
[2015-10-27 10:08:08.716][trace][26340][0] check feature compile nginx: on
[2015-10-27 10:08:08.716][trace][26340][0] check feature compile ffmpeg: on
[2015-10-27 10:08:08.716][warn][26340][0][0] stream caster: off
[2015-10-27 10:08:08.716][trace][26340][0] MR(merged-read): on, @see https://github.com/simple-rtmp-server/srs/issues/241
[2015-10-27 10:08:08.716][trace][26340][0] MR(merged-read) default 0 sleep 350
[2015-10-27 10:08:08.716][trace][26340][0] MW(merged-write) default sleep 350
[2015-10-27 10:08:08.716][trace][26340][0] read chunk stream cache cid [0, 16)
[2015-10-27 10:08:08.716][trace][26340][0] default gop cache 1, play queue 30s
[2015-10-27 10:08:08.716][trace][26340][0] complex send algorithm enabled.
[2015-10-27 10:08:08.716][warn][26340][0][0] TCP_NODELAY enabled, may hurts performance.
[2015-10-27 10:08:08.716][trace][26340][0] auto guess socket send buffer by merged write
[2015-10-27 10:08:08.716][trace][26340][0] system default latency in ms: mw(0-350) + mr(0-350) + play-queue(0-30000)
[2015-10-27 10:08:08.716][trace][26340][0] start deamon mode...
[2015-10-27 10:08:08.716][trace][26341][0] father process exit. ret=0
[2015-10-27 10:08:08.716][trace][26342][0] son(deamon) process running.
[2015-10-27 10:08:08.716][trace][26342][0] st_set_eventsys to epoll
[2015-10-27 10:08:08.717][trace][26342][0] st_init success, use epoll
[2015-10-27 10:08:08.717][trace][26340][0] grandpa process exit.
[2015-10-27 10:08:08.717][trace][26342][100] server main cid=100
[2015-10-27 10:08:08.717][trace][26342][100] write pid=26342 to ./objs/srs.pid success!
[2015-10-27 10:08:08.727][trace][26342][100] RTMP listen at tcp://0.0.0.0:1935, fd=8
[2015-10-27 10:08:08.727][trace][26342][100] signal installed
[2015-10-27 10:08:10.605][trace][26342][104] RTMP client ip=10.16.0.3
[2015-10-27 10:08:10.605][trace][26342][104] srand initialized the random.
[2015-10-27 10:08:10.607][trace][26342][104] simple handshake success.
[2015-10-27 10:08:10.607][trace][26342][104] connect app, tcUrl=rtmp://10.12.4.61/live, pageUrl=, swfUrl=rtmp://10.12.4.61/live, schema=rtmp, vhost=__defaultVhost__, port=1935, app=live, args=null
[2015-10-27 10:08:10.607][trace][26342][104] out chunk size to 60000
[2015-10-27 10:08:10.845][trace][26342][104] client identified, type=fmle-publish, stream_name=mytest1, duration=-1.00
[2015-10-27 10:08:10.855][trace][26342][104] source url=/live/mytest1, ip=10.16.0.3, cache=1, is_edge=0, source_id=-1[-1]
[2015-10-27 10:08:11.041][trace][26342][104] hls: win=60.00, frag=5.00, prefix=, path=./objs/nginx/html, m3u8=[app]/[stream].m3u8, ts=[app]/[stream]-[seq].ts, aof=1.00, floor=0, clean=1, waitk=1, dispose=0
[2015-10-27 10:08:11.052][trace][26342][104] start publish mr=0/350, p1stpt=20000, pnt=20000, tcp_nodelay=0, rtcid=106
[2015-10-27 10:08:11.382][trace][26342][104] got metadata
[2015-10-27 10:08:11.382][trace][26342][104] 4B audio sh, codec(10, profile=LC, 2channels, 0kbps, 44100HZ), flv(16bits, 2channels, 44100HZ)
[2015-10-27 10:08:11.382][trace][26342][104] input chunk size to 366
[2015-10-27 10:08:11.386][trace][26342][104] input chunk size to 398
[2015-10-27 10:08:11.386][trace][26342][104] audio drop ts segment, sequence_no=0, uri=mytest1-0.ts, duration=59839.19, start=0
[2015-10-27 10:08:11.391][trace][26342][104] input chunk size to 410
[2015-10-27 10:08:11.396][trace][26342][104] input chunk size to 501
[2015-10-27 10:08:14.896][trace][26342][104] input chunk size to 505
[2015-10-27 10:08:15.399][trace][26342][104] input chunk size to 514
[2015-10-27 10:08:16.394][trace][26342][104] input chunk size to 528
[2015-10-27 10:08:22.397][trace][26342][104] input chunk size to 539
[2015-10-27 10:08:32.885][trace][26342][104] input chunk size to 545
[2015-10-27 10:08:51.052][trace][26342][104] <- CPB time=20001, okbps=0,0,0, ikbps=129,129,0, mr=0/350, p1stpt=20000, pnt=20000
FMLE configuration is as follows: format: AAC, Channels: Stereo, Sample Rate: 44100 Hz, Bit Rate: 128 Kbps.
TRANS_BY_GPT3
Try reducing the bitrate to 64. Also, please send me the slices and m3u8 file so that I can package and transfer them to the cloud storage.
TRANS_BY_GPT3
After adjustment, modify the FMLE configuration to:
Format: AAC
Channels: Stereo
Sample Rate: 44100 Hz
Bit Rate: 48 Kbps
TRANS_BY_GPT3
There is still a regular noise 'pupu' when playing pure audio on Apple systems.
TRANS_BY_GPT3
RTMP to TS audio explosion, the reason is because the TBN of RTMP is 1/1000, which is in milliseconds, but TS is 1/90000, which is in 1/90 milliseconds. Therefore, TS has a higher resolution.
To convert from lower resolution to higher resolution, the timestamps need to be recalculated. This is available in SRS1/nginx-rtmp, but the calculation seems to be incomplete. SRS2 does not calculate it and directly uses the RTMP timestamps.
This may cause audio explosion issues, typically occurring once per slice. Therefore, if the slice can be increased to 10 or 30 seconds, the impact will be minimal.
However, I will fix it and see how to solve it most effectively. You can increase the size of the slice to confirm if this is the issue.
TRANS_BY_GPT3
I listened carefully, and there is a slight pause every 5 seconds, but the static noise from the microphone is constant.
Can you also record the RTMP stream and send me both the RTMP and corresponding TS files?
TRANS_BY_GPT3
I found out that there has been too much emphasis on PCR, actually it is not necessary.
TRANS_BY_GPT3
Every PCR causes a timestamp error between RTMP and TS, resulting in crackling sound and significant gaps between segments.
TRANS_BY_GPT3
C:\Desktop\rtmpdump-2.4-git-010913-windows>rtmpdump.exe -r rtmp://10.12.4.61:1935/live/mytest -v -B 10 -o out.flv
RTMPDump v2.4
(c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
Connecting ...
INFO: Connected...
Starting Live Stream
For duration: 10.000 sec
INFO: Metadata:
INFO: presetname Custom
INFO: creationdate Tue Oct 27 11:01:12 2015
INFO: audiodevice IN 1+2 (Digigram PCX924e)
INFO: audiosamplerate 44100.00
INFO: audiochannels 2.00
INFO: audioinputvolume 59.00
INFO: audiocodecid mp4a
INFO: audiodatarate 48.00
INFO: server SRS/2.0.195(ZhouGuowen)
INFO: srs_primary SRS/1.0release
INFO: srs_authors winlin,wenjie.zhao
INFO: server_version 2.0.195
64.294 kB / 9.70 sec
ERROR: RTMP_ReadPacket, failed to read RTMP packet header
66.581 kB / 10.05 sec
Download complete
The above was recorded using the rtmpdump tool for rtmp streams.
TRANS_BY_GPT3
I adjusted the hls.conf configuration as follows:
listen 1935;
max_connections 1000;
vhost **defaultVhost** {
hls {
enabled on;
hls_fragment 30;
hls_window 60;
hls_acodec aac;
hls_vcodec vn;
hls_aof_ratio 1.0;
hls_path ./objs/nginx/html;
hls_m3u8_file [app]/[stream].m3u8;
hls_ts_file [app]/[stream]-[seq].ts;
}
Note: The hls_fragment parameter has been changed to 30.
TRANS_BY_GPT3
Here are the current startup logs and environment:
[2015-10-27 11:59:25.257][trace][26675][0] srs checking config...
[2015-10-27 11:59:25.257][trace][26675][0] detect intranet address: 127.0.0.1, ifname=lo
[2015-10-27 11:59:25.257][trace][26675][0] retrieve local ipv4 ip=10.12.4.61, index=0
[2015-10-27 11:59:25.257][trace][26675][0] detect intranet address: 10.12.4.61, ifname=eth0
[2015-10-27 11:59:25.258][trace][26675][0] retrieve local ipv4 ip=172.17.42.1, index=1
[2015-10-27 11:59:25.258][trace][26675][0] detect intranet address: 172.17.42.1, ifname=docker0
[2015-10-27 11:59:25.258][warn][26675][0][0] stats network use index=0, ip=10.12.4.61
[2015-10-27 11:59:25.258][warn][26675][0][0] stats disk not configed, disk iops disabled.
[2015-10-27 11:59:25.258][trace][26675][0] write log to file ./objs/srs.log
[2015-10-27 11:59:25.258][trace][26675][0] you can: tailf ./objs/srs.log
[2015-10-27 11:59:25.258][trace][26675][0] @see: https://github.com/simple-rtmp-server/srs/wiki/v1_CN_SrsLog
[2015-10-27 11:59:25.258][trace][26675][0] srs(simple-rtmp-server) 2.0.195
[2015-10-27 11:59:25.258][trace][26675][0] license: The MIT License (MIT), Copyright (c) 2013-2015 SRS(simple-rtmp-server)
[2015-10-27 11:59:25.258][trace][26675][0] primary/master: SRS/1.0release
[2015-10-27 11:59:25.258][trace][26675][0] authors: winlin,wenjie.zhao
[2015-10-27 11:59:25.258][trace][26675][0] contributors: winlinwinlin@vip.126.com wenjie.zhao740936897@qq.com xiangcheng.liuliuxc0116@foxmail.com naijia.liuyoungcow@youngcow.net alcoholyialcoholyi@qq.com bytemanwangchen2011@gmail.com chad.wangchad.wang.cn@gmail.com suhetaosuhetao@gmail.com Johnnyfengjihu@163.com karthikeyankeyanmca@gmail.com StevenLiulq@chinaffmpeg.org zhengflzhengfl_1989@126.com tufang14breadbean1449@gmail.com allspaceallspace@gmail.com niesongsongnie950@gmail.com rudeb0tnimrod@themanxgroup.tw
[2015-10-27 11:59:25.258][trace][26675][0] uname: Linux MediaWiki 2.6.32-504.el6.x86_64 #1 SMP Wed Oct 15 04:27:16 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
[2015-10-27 11:59:25.258][trace][26675][0] build: 2015-10-26 16:49:02, little-endian
[2015-10-27 11:59:25.258][trace][26675][0] configure: --disable-all --with-ssl --with-hls --with-nginx --with-ffmpeg --with-transcode
[2015-10-27 11:59:25.258][trace][26675][0] features: --prefix=/usr/local/srs --with-hls --without-hds --without-dvr --with-nginx --with-ssl --with-ffmpeg --with-transcode --without-ingest --without-stat --without-http-callback --without-http-server --without-stream-caster --without-http-api --without-librtmp --without-research --without-utest --without-gperf --without-gmc --without-gmp --without-gcp --without-gprof --without-arm-ubuntu12 --without-mips-ubuntu12 --log-trace
[2015-10-27 11:59:25.258][trace][26675][0] conf: conf/hls.conf, limit: 1000
[2015-10-27 11:59:25.258][trace][26675][0] writev limits write 1024 iovs a time
[2015-10-27 11:59:25.258][warn][26675][0][0] SRS 2.0.195 is not stable, please use stable branch 1.0release instead
[2015-10-27 11:59:25.258][trace][26675][0] cpu is amd64, glibc 2.12
[2015-10-27 11:59:25.258][trace][26675][0] cpu is x86_64, glibc 2.12
[2015-10-27 11:59:25.258][trace][26675][0] check feature rtmp handshake: on
[2015-10-27 11:59:25.258][trace][26675][0] check feature hls: on
[2015-10-27 11:59:25.258][warn][26675][0][0] check feature hds: off
[2015-10-27 11:59:25.258][warn][26675][0][0] check feature http callback: off
[2015-10-27 11:59:25.258][warn][26675][0][0] check feature http api: off
[2015-10-27 11:59:25.258][warn][26675][0][0] check feature http server: off
[2015-10-27 11:59:25.258][warn][26675][0][0] check feature http parser: off
[2015-10-27 11:59:25.258][warn][26675][0][0] check feature dvr: off
[2015-10-27 11:59:25.258][trace][26675][0] check feature transcode: on
[2015-10-27 11:59:25.258][warn][26675][0][0] check feature ingest: off
[2015-10-27 11:59:25.258][warn][26675][0][0] check feature system stat: off
[2015-10-27 11:59:25.258][trace][26675][0] check feature compile nginx: on
[2015-10-27 11:59:25.258][trace][26675][0] check feature compile ffmpeg: on
[2015-10-27 11:59:25.258][warn][26675][0][0] stream caster: off
[2015-10-27 11:59:25.258][trace][26675][0] MR(merged-read): on, @see https://github.com/simple-rtmp-server/srs/issues/241
[2015-10-27 11:59:25.258][trace][26675][0] MR(merged-read) default 0 sleep 350
[2015-10-27 11:59:25.258][trace][26675][0] MW(merged-write) default sleep 350
[2015-10-27 11:59:25.258][trace][26675][0] read chunk stream cache cid [0, 16)
[2015-10-27 11:59:25.258][trace][26675][0] default gop cache 1, play queue 30s
[2015-10-27 11:59:25.258][trace][26675][0] complex send algorithm enabled.
[2015-10-27 11:59:25.258][warn][26675][0][0] TCP_NODELAY enabled, may hurts performance.
[2015-10-27 11:59:25.258][trace][26675][0] auto guess socket send buffer by merged write
[2015-10-27 11:59:25.258][trace][26675][0] system default latency in ms: mw(0-350) + mr(0-350) + play-queue(0-30000)
[2015-10-27 11:59:25.258][trace][26675][0] start deamon mode...
[2015-10-27 11:59:25.258][trace][26676][0] father process exit. ret=0
[2015-10-27 11:59:25.259][trace][26675][0] grandpa process exit.
[2015-10-27 11:59:25.259][trace][26677][0] son(deamon) process running.
[2015-10-27 11:59:25.259][trace][26677][0] st_set_eventsys to epoll
[2015-10-27 11:59:25.259][trace][26677][0] st_init success, use epoll
[2015-10-27 11:59:25.259][trace][26677][100] server main cid=100
[2015-10-27 11:59:25.260][trace][26677][100] write pid=26677 to ./objs/srs.pid success!
[2015-10-27 11:59:25.270][trace][26677][100] RTMP listen at tcp://0.0.0.0:1935, fd=8
[2015-10-27 11:59:25.270][trace][26677][100] signal installed
[2015-10-27 11:59:25.547][trace][26677][104] RTMP client ip=192.168.11.53
[2015-10-27 11:59:25.547][trace][26677][104] srand initialized the random.
[2015-10-27 11:59:25.548][trace][26677][104] simple handshake success.
[2015-10-27 11:59:25.548][trace][26677][104] connect app, tcUrl=rtmp://10.12.4.61:1935/live, pageUrl=, swfUrl=rtmp://10.12.4.61:1935/live, schema=rtmp, vhost=**defaultVhost**, port=1935, app=live, args=null
[2015-10-27 11:59:25.548][trace][26677][104] out chunk size to 60000
[2015-10-27 11:59:28.795][trace][26677][106] RTMP client ip=10.16.0.3
[2015-10-27 11:59:28.796][trace][26677][106] simple handshake success.
[2015-10-27 11:59:28.796][trace][26677][106] connect app, tcUrl=rtmp://10.12.4.61/live, pageUrl=, swfUrl=rtmp://10.12.4.61/live, schema=rtmp, vhost=**defaultVhost**, port=1935, app=live, args=null
[2015-10-27 11:59:28.796][trace][26677][106] out chunk size to 60000
[2015-10-27 11:59:29.045][trace][26677][106] client identified, type=fmle-publish, stream_name=mytest, duration=-1.00
[2015-10-27 11:59:29.055][trace][26677][106] source url=/live/mytest, ip=10.16.0.3, cache=1, is_edge=0, source_id=-1[-1]
[2015-10-27 11:59:29.255][trace][26677][106] hls: win=60.00, frag=30.00, prefix=, path=./objs/nginx/html, m3u8=[app]/[stream].m3u8, ts=[app]/[stream]-[seq].ts, aof=1.00, floor=0, clean=1, waitk=1, dispose=0
[2015-10-27 11:59:29.265][trace][26677][106] start publish mr=0/350, p1stpt=20000, pnt=20000, tcp_nodelay=0, rtcid=108
[2015-10-27 11:59:29.982][trace][26677][106] got metadata
[2015-10-27 11:59:29.989][trace][26677][106] 4B audio sh, codec(10, profile=LC, 2channels, 0kbps, 44100HZ), flv(16bits, 2channels, 44100HZ)
[2015-10-27 11:59:29.989][trace][26677][106] input chunk size to 143
[2015-10-27 11:59:29.989][trace][26677][106] input chunk size to 154
[2015-10-27 11:59:29.989][trace][26677][106] audio drop ts segment, sequence_no=0, uri=mytest-0.ts, duration=3343.64, start=0
[2015-10-27 11:59:30.491][trace][26677][106] input chunk size to 158
[2015-10-27 11:59:32.483][trace][26677][106] input chunk size to 161
[2015-10-27 11:59:37.484][trace][26677][106] input chunk size to 163
[2015-10-27 11:59:38.484][trace][26677][106] input chunk size to 166
[2015-10-27 11:59:40.499][trace][26677][106] input chunk size to 193
[2015-10-27 11:59:41.996][trace][26677][106] input chunk size to 197
[2015-10-27 11:59:45.987][trace][26677][106] input chunk size to 199
[2015-10-27 11:59:55.500][trace][26677][106] input chunk size to 211
[2015-10-27 11:59:55.747][warn][26677][104][62] client disconnect peer. ret=1004
[2015-10-27 11:59:59.762][trace][26677][110] RTMP client ip=192.168.11.53
[2015-10-27 11:59:59.762][trace][26677][110] simple handshake success.
[2015-10-27 11:59:59.762][trace][26677][110] connect app, tcUrl=rtmp://10.12.4.61:1935/live, pageUrl=, swfUrl=rtmp://10.12.4.61:1935/live, schema=rtmp, vhost=**defaultVhost**, port=1935, app=live, args=null
[2015-10-27 11:59:59.763][trace][26677][110] out chunk size to 60000
[2015-10-27 12:00:09.265][trace][26677][106] <- CPB time=20000, okbps=0,0,0, ikbps=51,51,0, mr=0/350, p1stpt=20000, pnt=20000
TRANS_BY_GPT3
This is the current 30S slice.
http://pan.baidu.com/s/1i3523lF
TRANS_BY_GPT3
At present, the sound of puffing is not so obvious, but there are still occasional intermittent situations.
TRANS_BY_GPT3
Unable to use rtmpdump for recording, use srs_rtmp_dump instead, which can be found in the objs directory compiled. Rtmpdump will alter the timestamps.
TRANS_BY_GPT3
Okay.
TRANS_BY_GPT3
[root@ objs]# ./srs_rtmp_dump -r rtmp://127.0.0.1:1935/live/mytest -o output.flv
Take a screenshot of the RTMP stream through srs_rtmp_dump
http://pan.baidu.com/s/1eQrvxHC
TRANS_BY_GPT3
Can the issue you just mentioned about the time stamp mismatch between RTMP and TS caused by each PCR, which leads to popping sound and large gaps between clips, be improved by modifying the configuration file parameters?
TRANS_BY_GPT3
It seems that there is no issue with accuracy. I previously figured it out, but unfortunately, I have forgotten now and I don't remember it in my notes either. It is possible that nginx-rtmp, in order to reduce HLS overload, combines multiple AAC frames and writes them into ts files, which requires calculating time differences and such. I will figure out and fix this issue.
TRANS_BY_GPT3
Okay, and I just tried SRS_1.0 and found that it seems to not support hls_acodec and hls_vcodec parameters, right?
TRANS_BY_GPT3
I changed to write each audio package, but this will increase the overhead of HLS. This can be avoided by changing the audio bitrate. The recommended bitrate is:
For the example video doc/source.200kbps.768x320.flv, the video bitrate is 212kbps. The audio bitrate:
- When it is 30kbps, the TS size is 358892, FLV size is 294177, and the overhead is 21.9%.
- When it is 48kbps, the TS size is 361336, FLV size is 317177, and the overhead is 13.9%.
- When it is 54kbps, the TS size is 382768, FLV size is 324831, and the overhead is 17.8%.
Therefore, the overhead is minimized when the audio bitrate is between 48 and 54. If the video bitrate increases, the impact of the audio will be smaller.
TRANS_BY_GPT3
Is that necessary to modify the source code or modify the configuration file?
TRANS_BY_GPT3
Okay, I will adjust the audio bitrate to between 48-54. Currently, the bitrate is 48.
TRANS_BY_GPT3
This doesn't have any impact, it's just a large expense, which means there is a lot of wasted bytes. I have no issues with watching videos and listening to audio; although, when it comes to pure audio, there is still a popping sound.
TRANS_BY_GPT3
Almost certain to be related to DTS, but not related to PCR.
- In the case of video+audio, it is possible to write one frame for each audio separately without any problem, although it may incur a higher cost.
- When it is audio only, it must not be written one audio at a time, as it cannot be played. Multiple audio must be written as a single frame.
TRANS_BY_GPT3
2.0.196
There is no problem with video+audio.
The audio is slightly better, sometimes there is still distortion, but it has improved a lot.
I can't solve this problem for now, I don't know the behavior of the IOS player.
Let's stick with SRS2 for now, and we will solve this problem properly with SRS3.
TRANS_BY_GPT3
Okay, I will continue to pay attention to the SRS product. Thank you very much for taking the time to reply to my question.
TRANS_BY_GPT3
I will be downloading version 2.0.196 and testing it in the near future.
TRANS_BY_GPT3
Yes, it was also found during the testing process that as long as VIDEO and AUDIO are enabled, the iOS playback is normal. Now I understand what's going on.
TRANS_BY_GPT3
The DTS of the audio needs to be consistent. If it is inconsistent, there will be a popping sound (in the case of both audio and video), but I have not tried it with pure audio.
TRANS_BY_GPT3
It is caused by the cumulative error of the audio. For example, for AV streams (which include both audio and video), if multiple RTMP audio packets are combined into one AAC packet (to save the payload of TS), the total length of these packets should be calculated again based on the AAC timestamp (calculated by the number of samples and the sampling rate, with AAC generally fixed at 1024 bytes per sample, which can be obtained after parsing).
Suppose there are 10 A packets: A0-A9, converting their original encoder timestamps to 1/1000th of RTMP precision leads to loss of accuracy. Therefore, combining the errors of the 10 packets will display their errors. For example, if the RTMP timestamp for A0 is 100 milliseconds and A9 is 300 milliseconds, the actual combined AAC frame may have a timestamp of 303 milliseconds. The next RTMP timestamp is 300 milliseconds, resulting in some gaps.
Therefore, when combining multiple RTMP AAC packets into TS packets, the timestamps must be recalculated. Using 1024 bytes as the sample length directly in NGINX-RTMP is inaccurate. SRS can handle AV streams without combining audio packets, but for pure audio, packet combination is necessary; otherwise, it cannot be played.
TRANS_BY_GPT3
I will find time to fix it completely.
TRANS_BY_GPT3
This morning, I checked and it appears to be like this:
An AAC frame is encoded using 1024 sample samples (not 1024 Bytes).
The playback time of an audio frame (in seconds) = number of sample samples in an AAC frame / sample rate.
If the sample rate is 44100Hz, according to the above, the playback time of this audio frame = 1024/44100 = 0.02232s * 1000 = 22.32ms.
The "pts" base clock is calculated based on 1000ms and reused in "ts", where the base clock is 90k, so it should also be multiplied by 90.
So, I'm looking at https://github.com/arut/nginx-rtmp-module/blob/e08959247dc840bb42cdf3389b1f5edb5686825f/hls/ngx_rtmp_hls_module.c#L1809.
est_pts = ctx->aframe_base + ctx->aframe_num * 90 * 1000 * 1024 / codec_ctx->sample_rate;
It seems that merging multiple frames and calculating pts is not a problem, right?
@winlinvip
TRANS_BY_GPT3
Didn't you see that he annotated TODO?
/* TODO: We assume here AAC frame size is 1024
* Need to handle AAC frames with frame size of 960 */
TRANS_BY_GPT3
And it only explains how to calculate, but does not explain why it should be calculated this way (recalculating timestamps based on sampling instead of using rtmp timestamps). This is the biggest problem in my opinion, in fact, it is unnecessary to calculate in HLS, only necessary when merging audio frames. If both audio and video streams are available in HLS, an audio frame can be written without needing to calculate DTS, directly using rtmp is fine.
TRANS_BY_GPT3
1024 or 960 for AAC-LC
2048 or 1920 for HE-AAC (v2)
512 or 480 for AAC-LD and AAC-ELD
What determines the number of sampling samples required for encoding?
Once this is understood, his "need to" is resolved.
TRANS_BY_GPT3
Calculating DTS based on the number of AAC samples is determined by TBN, and this is clear. The TBN for FLV is 1/1000, while the TBN for TS is 1/90000. TS has higher precision, and in reality, the original encoder also loses precision when outputting FLV/RTMP.
TRANS_BY_GPT3
AAC Audio - Encoder Delay and Synchronization
https://developer.apple.com/library/content/technotes/tn2258/_index.html
It's also unclear here. Perhaps it's necessary to carefully examine the source code of fdkaac in order to understand.
TRANS_BY_GPT3
The issue of explosive sound in HLS pure audio (there is a popping sound between each segment), Apple's reply email states that AAC files should be used instead of TS files, and the timestamps should be recalculated.
TRANS_BY_GPT3
If TBN is 44100, then converting from FLV with TBN=1000 will result in loss, for example, a 34-millisecond FLV audio converted to TBN44100 would be 34*44100/1000=1499, converting it back to FLV with TBN44100 would be 1499*1000/44100=33, resulting in a loss of 1 millisecond. However, TS with TBN90000 is exactly 90 times the FLV's TBN, so it doesn't have this issue.
TRANS_BY_GPT3
The issue of loud noise has been identified and it is caused by the sampling rate. Please refer to: #547 (comment).
TRANS_BY_GPT3