Synchronisation
exp opened this issue · comments
Given two cameras observing the same scene, each with acceptable jitter, how closely should the two views be synced?
Each camera will have its own free-running but quite inaccurate clock periodically checked by NTP, but there's no guarantee their NTP servers are closely linked and significant jitter and delay is to be expected no matter what.
Each frame is captured with an offset which should be very close between cameras but will ultimately vary based on its local clock rate and other internal factors such as temperature or system load.
Each media stream encodes its understanding of the current time into the PCR (Program Clock Reference) and this is used to synchronise individual decoders, but it too is limited to the clock quality inside a cheap network appliance, ie not great.
None of these can be trivially synchronised with each other without a process of clock reconstruction, that is determining on the receiver side at which time the sender actually captured or transmitted the frame (PTS vs PCR).
Receivers must also undergo this process as multiple cloud instances are very unlikely to have strong clock synchronisation. There are protocols to get a much better idea of what time it actually is, ie the Precision Time Protocol. I know that AWS has or does support some form of PTP use.
This is just a lot of blogging but really the question is: Do you even care about showing the same scene from two angles and having them be well synced, or would a few frames either way make little difference? If it does matter, a careful process of clock reconstruction has to happen on input, and the datastore you use must be able to handle different frame to frame timings, and even the offsets that occur when playing back two slightly mis-timed sources. (Axis Cam1 could be 60.0002fps wheras AxisCam2 could be 59.9994).