VideoRoom track's owner identification
Qizot opened this issue · comments
Participant identification
Jitsi just like Membrane support multiple peers on a single shared peer connection.
There is a problem of identifying each peer based on the incoming tracks. The SFU needs to
keep track of its participants (things like display name ect.) and pass that information
to the client when needed.
How Jitsi does that?
The low-level code that is responsible for identifying incoming tracks with participants is available here:
Jitsi listens on remote tracks being added. Then it does manual search on the remoteDescription.sdp string to obtain necessary information.
First it checks the transceiver for mid field, if not found then if falls back to searching for msid that equals to the current stream id.
(It all depends on what plan the browser is using but as far as I know Unifed Plan is now a standard).
Then the sdp is filtered for lines that include mid or msid fields that has been found in previous step.
In resulting lines we filter out received lines to get corresponding ssrc.
The main goal here is to obtain corresponding ssrc of incoming track. Then a request is sent
to the server with given ssrc to return track's participant id. And further down the line if participant
is not in client's cache, again a request is sent to obtain participant's information.
How should we do that?
What fields can we use to query information about given participant? Each participant will see the same mid values,
from what I recall msid will differ for each participant same goes with ssrc and when querying the server it will
have to identify the caller to get the proper mapping of outgoing streams (with mid it can do a really quick check).
And how should we get participants information, after getting a new track by sending query request or the server itself should first send information about incoming participant?
cc @mickel8
I would try to associate participant data with its msid/mid. I am not sure if we have to search for ssrc. At this moment each peer has its own msid/mid. The msid/mid cannot be reused by other participant's stream. The flow would be as follow
- A new paricipant joins. Server receives
new_peernotification and peer custom data (like username). - SDP offer/answer exchange. Now server knows msid/mid of the new peer.
- Server broadcasts this data to the other peers.
This task is also connected with membraneframework/membrane_webrtc_plugin#8
@FelonEkonom is checking how exactly msid works and who generates its value
- A new paricipant joins. Server receives
new_peernotification and peer custom data (like username).- SDP offer/answer exchange. Now server knows msid/mid of the new peer.
- Server broadcasts this data to the other peers.
I find a problem with 3. point as we have to perform the sending operation twice, once when a new peer connects (then broadcast to everyone else) and the other time when a new peer joins we need to send him existing peers information.
If we postpone the query for peer's information to the moment a new track appears we wouldn't duplicate the logic of broadcasting/sending it to the clients and moreover it would simplify the server logic as it would not care about sending anything unless asked explicitly.
From experience when I had been working on screensharing on a shared peer connection with video feed there was a synchronization problem when somebody joined the room when someone was in process of starting the screen share and the problem solution was not pretty.
Couldn't we just send the mapping of (mid/msid/whatever) => user_data along with SDP? This case is a bit different than screensharing, at least if we assume that the user data changes only when SDP changes.
In this case client would send its data with SDP answer and receive other peers data in a separate message or it would send its data in new_peer notification and receive other peers data with SDP offer?
For me the best scenario would be to send peer data in new_peer notification and request it in other peers from server when needed. But scenario with SDP offer/answer looks nice too. Anyway I am wondering why Jitsi uses ssrc?
I've just created a simple POC and sending participants list with offer works great.
Nice 🎉 So let's go for it
For me the best scenario would be to send peer data in
new_peernotification and request it in other peers from server when needed. But scenario with SDP offer/answer looks nice too. Anyway I am wondering why Jitsi uses ssrc?
I believe that they use the ssrc for something else as well and they may use it as a unification among different webrtc plan implementations?
Maybe. It's worth mentioning that besides ssrc there is also cname which stands for canonical name and is probably more valuable id than ssrc.