shinyoshiaki / werift-webrtc

Thank you for this awesome Webrtc Implementation.

I'm using your SFU server, updated it to use the latest werift, and it works fine, I want to create a MESH network of SFU servers so for that I want to forward/send Rtp data to remote server, with that i can sync video calls ( 1 user from server A & another user from Server B on the same room ).
I can succesfully transfer the rtp data to remote server but on another side it feels little out of sync and if the data size get's increase ( stream from more users ) video & audio gets out of sync ( maybe JitterBuffer ? but don't know how to use that in werift ).
I send Rtp data's through data-channels, also tried with hyperswarm ( which is DHT service )but other side get's the out of sync data.
here is the structure to send rtp data to remote server

const { unSubscribe } = msTrack.onReceiveRtp.subscribe(async (rtp) => {
      sendSFUData(JSON.stringify({
        type: 'rtpData',
        roomName: this.roomName,
        info: this.info,
        quality: this.info.simulcast ? msTrack.rid : 'single',
        packet: rtp.serialize(),
      }))
    })

then on subscribe the media, i do JSON.parse the object then

const newRtp = RtpPacket.deSerialize(Buffer.from(packet))
sender.sender.sendRtp(newRtp)

so basically i serialize & deSerialize the Rtp packets just like in the example rtp_forward.
but still the data get's out of sync, if you can give any suggestion that will be great, thank you

oops I forgot to do receiver.sendRtcpPLI(ssrc) on every 2 seconds just like in your normal SFU server , now i feel like it's almost in sync, but not sure it's the right way, any suggestions on this ?

werift's Track does not lip-sync on a per-stream basis.

Please refer to the Recording sample code for an example of lip-sync.

werift-webrtc/examples/save_to_disk/pipeline.ts

Line 48 in cc5ebf1

const lipsync = new LipsyncCallback();

The lip-sync module used in the source code above is not suitable for your use since it does not interface with RTP. Please use it with appropriate modifications.

Thank you for your reply.
I'm trying to implement mediasoup like pipeTransports between different SFUs ( SFU-->SFU data transfer but on different hosts ).
currently I achived it with sending media-info, rtp packets via data-channel this way i keep all the connected (MESH network of SFU servers ) sync in realtime. it works fine even the server have 30mbps(total) internet speed, but when the server bandwidth reduce the video or audio get out-of sync but within some seconds it get in sync again ( just like the user have bad internet)
If werift can bring the mediasoup like pipeTransports feature or some examples will be great( maybe something like pion to pion example )

Forward Rtp packets