Feat - delay local monitor audio to make it possible to play in time with incomming audio
PowerUser64 opened this issue · comments
One of the biggest problems with playing music with people over the internet is the latency. If someone tries to play in time with the incoming audio, they will sound far behind on the other end, causing the other person to want to slow down to make them sound "on the beat" to them. The reason for this is that there is latency going over the internet, but there is no latency locally. If SonoBus could enforce the same latency for all clients and add latency to the local playback, the main issue with latency would be removed, at the cost of everyone having more latency.
(When I say "enforce a latency," I mean add latency to faster connections to make all latencies equal to the highest one-way latency of a group.)
Adding latency to local playback will force players to play notes earlier than they would normally for notes to sound in time to their player. If the delay is set to the highest latency in the group, the time it takes a note to make sound on one person's end will be equivalent to the time it takes for it to travel across the internet and make sound on the other clients.
This could be taken even further to let group members adjust the latency based on how much they are comfortable with. For example, if there is a group of two musicians, one playing an acoustic instrument like the trumpet and another playing an electronic synthesizer, it's going to be much easier for the synthesizer player to add latency to their instrument and play with high latency than the trumpet player because the trumpet inherently has no latency, so the trumpet player could lower their latency to zero, doubling the added local latency for the synthesizer player while keeping the trumpet player's latency low. Of course, not everyone can have zero latency at once, but being able to balance it so players who need lower latency can get it would be a very nice addition to this.
I should also note that I used this tactic in jam sessions with people over the internet using bespoke synth and it has made everything much more enjoyable.
Yep, this feature is already in there in the form of Monitor Delay, in the Input Mixer. Open the Input Mixer then press the MFx button on the right side of any of your input groups, then you can set the extra monitor delay manually or by averaging the one way or round trip latency to your peers. Enable it with the “power” button. There is also a global toggle at the top of the input mixer (shortcut Command/Ctrl B).
As for the other, there is a Group Latency Match feature if you press the group button (at top just below the connect button), and select group latency match. It works OK, best if most people have good latencies already.
Oh, cool! Thanks for the response, I will definitely use this. One other thing, I was initially thinking of making this about adding OSC output for the maximum latency of a group, so the value could be used to sync other things inside synthesizers (like automating the monitor delay that turns out to already be a feature).
Taking the idea even further though, it would be cool if sonobus could send OSC data over the network and keep it in sync with the audio. Possibly even taking in midi notes and sending them over OSC too. Should I make issues for either of these?
Sending midi and other events (like OSC) along with the audio, sample accurately, is on the todo list for SonoBus 2.0.
And there are some other very cool plans, like integrating a NinJam-like musical interval latency feature that you could enable at will among the people in the group :)
Yooo! I'm looking forward to it :D
Thanks for all you do to support sonobus! Can't wait for version 2.0!