AV Sync

NDI relies on timestamps to synchronize incoming audio and video. You can of course fill these in yourself although by default NDI will use the system time. If these system times are synchronized (via NTP, PTP, GPS, or similar) multiple streams from different systems may be synchronized.

This API allows you to synchronize audio and video from either the same or different NDI sources with accuracy approaching one audio sample if a sender uses sample-accurate timestamps. While this of course can be achieved at a regular API user level this is a non-trivial task (for instance by ensuring that you audio and video frames have the same exact timestamps), with this API helping solve at least the following problems in a relatively simple way:

• The audio and video streams might be generated by applications that are not generating nanosecond accurate timestamps and so a significant level of filtering and accuracy improvement is needed so that frames can be aligned with sample-level accuracy.

• Packetizing the audio so that it correctly matches the video frames exactly is complex, and preserving and computing the correct timestamps, metadata and timecodes in this process is non-trivial.

• Often a particular pattern of output samples is needed for interfacing with hardware is needed (e.g., NTSC often requires audio samples on frames in alternating patterns of 1601 and 1602 samples) and it is very beneficial to ask the API for the number of samples wanted and it correctly aligns audio to this pattern.

• Correctly detecting when a stream has no audio or there are format changes requires special consideration.

• Timeouts and ensuring that errors remain within the Nyquist sampling limits when running on a computer that might already be under load or have inaccurate clocks makes a solution to this challenging.

This API is a little more complex than some of the others in this SDK because it allows for great flexibility within just a few functions. Please pay close attention to the exact definition of the return results and how they work.