Guidelines

This API will take audio for a NDIlib_recv_instance_t instance, returning a NDIlib_avsync_instance_t object. It will then look at the audio being received on this input. One may then take video frames from either the same NDIlib_recv_instance_t instance or a different one and one may query the NDIlib_avsync_instance_t for the audio that matches that video frame. This takes the timestamps that are attached to each frame to synchronize the audio and video with high prevision.

It is important when working with this object that you ensure that the following conditions are met for the best results.

• This API does not resample or otherwise modify the audio samples. Special care must be taken when trying to synchronize audio and video streams that are not synchronized (genlocked).

• The audio and video do not need to be on the same clock (i.e., they do not need to be genlocked). However, if the streams are not synchronized then the number of audio samples returned with the video frame might not be exactly what is expected and can vary over time. For instance, if the audio being received is running on a clock that is 1% faster than the video clock then you will receive the exactly synchronized audio although there will be 1% more audio samples than one might expect based on the audio sample rate.

• The audio and video are expected to behave relatively well (but not perfectly). If there are significant gaps in the audio or video streams, or the streams are delivered such that they have very significant jitter (well beyond the Nyquist sampling limit) then it is hard to ensure that they are reconstructed perfectly although all effort is made to act gracefully in these situations. As an example, if audio is delivered significantly later than the corresponding video then this device will end up needing to run behind on the video to find the exactly matching audio.

• You may pass an NDIlib_recv_instance_t into NDIlib_avsync_create and the returned NDIlib_avsync_instance_t will capture the audio from the device. You may simultaneously use this same NDIlib_recv_instance_t to capture the audio and use the corresponding NDIlib_avsync_instance_t as the means of synchronizing the audio and video from the same device. You similarly capture audio and video from different devices if you wish to use an NDIlib_avsync_instance_t to synchronize audio and video from different sources, however, note the comments above on the clocking from disparate sources.

• This implementation will only support and process audio that is supported on your platform by the decoders within NDI (e.g., on Linux AAC audio might not correctly be used with this implementation).