Giters

versatica / mediasoup

Cutting Edge WebRTC Video Conferencing

Home Page:https://mediasoup.org

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

opus dtx incorrectly judged

buptlsp opened this issue · comments

commented

Bug Report

I changed the ptime of the opus and found that ignoreDtx does not work. My dtx packet is as follows:

image

I checked the code in worker/src/RTC/Codecs/Opus.cpp and found that the code is too simple to handle this situation. The code just checks if the payload length of the packet is 1. If it is 1 it assumes it is a dtx, if not it is not.

image

According to https://datatracker.ietf.org/doc/rfc6716/, there are four types of packets: code 0, code 1, code 2, and code 3.

image

My packets are dtx packets, using swb, frame size = 20ms, using code 3, 3 frames per packet, using CBR, no padding, each frame size is 0, but packet length is equal to 2. the code will think it's a normal packet instead of dtx packet.

commented

It makes sense. Definitely the OPUS DTX detection in mediasoup is weak.

Honestly we are terribly busy with other mediasoup tasks and I don't think I'm gonna hace time to read the spec and work on this in the short/medium term. @buptlsp is this something you could contribute with a PR and hopefully C++ side tests? Note that we have tests in mediasoup/worker/test/RTC/Codecs folder for video codecs but not for OPUS.

commented

NOTE: I've been told that in libwebrtc the DTX packet detection is following:

bool IsDtxPacket() const override { return payload_.size() <= 2; }

Would this be ok?

commented

as rfc6716 said, there may be 4 types， the length of dtx packet should be:

  • code 0 : one byte toc with no data. payload length = 1.
  • code 1 : one byte toc with no data( 2 * 0 cbr data). payload length = 1.
  • code 2: one byte toc with 1 byte length. payload length = 2.
  • code 3: one byte toc with 1 byte frame count byte. if CBR payload length = 2. if VBR, the payload length should be 2 + M-1 (M is the frame count). but in this case VBR is waste byte, opus codec may just use CBR (I am not sure).

so, I think payload_.size() <= 2 may be right. this judgement may have this problem:

  • code 3 VBR problem as I said.
  • if the ptime is very small, for example, 2.5ms, using code 0, the data may very small to be 1 byte . the total payload length is 1 byte toc + 1 byte data.

The probability of these two situations occurring is too small to be considered.

there is other situation may cause the payload length <=2. the rtp packet may be total padding without data. the payload is not opus packet, so the length may be 2 also. I have discussed this problem with you before. if you have already removed the padding, this problem will not exist.

commented

there is other situation may cause the payload length <=2. the rtp packet may be total padding without data. the payload is not opus packet, so the length may be 2 also. I have discussed this problem with you before. if you have already removed the padding, this problem will not exist.

This is not a problem since we know the payload size and the padding size separately.

commented

so, I think payload_.size() <= 2 may be right. this judgement may have this problem:

code 3 VBR problem as I said.
if the ptime is very small, for example, 2.5ms, using code 0, the data may very small to be 1 byte . the total payload length is 1 byte toc + 1 byte data.

So I'm a bit worried here. Can't we know if those cases contain real OPUS data or DTX? If we were an OPUS decoder couldn't we know it by parsing these bytes/bits?

commented

if you just use length to judge, It may cause small problem, but it is very easy. the code may by 1 line.
if you check toc of opus, it's can be judged absolutely correct, but a little complicated. the code may be 100 line.