LogoLogo
🛠️ Tools🗂️ SDK📄 White Paper
  • Getting Started
    • 🟣What is NDI?
    • 🆕Release Notes
    • 📄White Paper
      • Discovery & Registration
        • mDNS
        • Discovery Service
        • Manual Connection
        • NDI Groups
      • NDI Protocols
        • Reliable UDP - NDI 5
        • Multipath TCP - NDI 4
        • UDP with Forward Error Correction – NDI 3
        • Single TCP – NDI 1
      • NDI Related Network Ports
      • Getting video across the network
      • Network Layout
      • Bandwidth
        • NDI High Bandwidth based on SpeedHQ2 (8bit 4:2:2)
        • NDI High Bandwidth based on SpeedHQ7 (8bit 4:2:2:4)
        • NDI HX2 h.264 (8bit 4:2:0)
        • NDI HX2 h.265 (8bit 4:2:0)
        • NDI HX3 h.264 (8bit 4:2:0)
        • NDI HX3 h.265 (8bit 4:2:0)
        • NDI Proxy and bandwidth optimization
      • Network Interface Settings
      • NIC Selection
      • Encoding and Decoding
      • Multicast
      • NDI Administrative Settings
      • Synchronization
      • NDI in the Cloud
    • Glossary
      • NDI Terminology
      • Industry Terminology
  • Using NDI
    • Introduction
    • NDI for Video
      • Digital Video Basics
      • NDI Video Codecs and Format Matrix
      • NDI Encoding Support Matrix
        • Practical NDI Receivers Format Support
        • Practical NDI Transmitters Support
        • Summary Table
      • Interoperability Scenarios
    • NDI for Audio
      • Digital Audio Fundamentals
      • Audio Over IP
      • Technical Facts About NDI for Audio
      • Use Cases
      • Products Using NDI for Audio
    • ⚒️NDI Tools
      • Release Notes
      • Installing NDI Tools
        • Software License Agreement
        • Privacy Policy
      • NDI Tools Launcher
      • NDI Tools for Windows
        • Access Manager
        • Bridge
          • NDI Bridge automation
          • Configuring Port Forwarding
          • Bridge Tool Logging
        • Remote
        • Router
        • Screen Capture
        • Screen Capture HX
        • Studio Monitor
        • Test Patterns
        • Webcam Input
      • NDI Tools for Mac
        • Access Manager
        • Scan Converter
        • Router
        • Test Patterns
        • Video Monitor
        • Virtual Input
      • Plugins
        • NDI for After Effects
        • NDI for Premiere Pro
        • NDI Output for Final Cut Pro
        • NDI for VLC
        • Audio Direct
        • OBS
    • Utilities
      • Analysis
    • Using NDI with Software
      • Getting Started with NDI in OBS for Windows or Mac
      • Using OBS Studio as a Commentary System
      • Using NDI Tools as a virtual camera in Mac
      • Using NDI and Dante on the same Network
      • Use OBS video + audio on Zoom with macOS
    • Using NDI with Hardware
      • NDI HX upgrades for cameras
      • How to Activate Panasonic Cameras for NDI HX1 — Step-by-Step
      • Recommended Network Switch Settings for NDI
  • Developing with NDI
    • Introduction
    • 📂SDK
      • Release Notes
      • Licensing
      • Software Distribution
        • Header Files
        • Binary Files
        • Redistributables
        • Content Files
        • Libraries
        • NDI-SEND
        • NDI-FIND
        • NDI-RECEIVE
        • Utilities
        • Command Line Tools
      • CPU Requirements
      • Dynamic Loading of NDI Libraries
      • Performance and Implementation
      • Startup and Shutdown
      • Example Code
      • Port Numbers
      • 🔧Configuration Files
      • Platform Considerations
      • NDI-SEND
      • NDI-FIND
      • NDI-RECV
      • NDI Routing
      • HDR
      • Command Line Tools
      • Frame Types
        • Video Frames
        • Audio Frames
        • Metadata Frames
      • Windows DirectShow Filter
      • 3rd Party Rights
      • Support
    • Advanced SDK
      • Release Notes
      • Licensing
      • Overview
      • Configuration Files
      • NDI SDK Review
        • Sending
          • Asynchronous Sending Completions
          • NDI Sending On High Latency Connections
        • Receiving
          • Custom Allocators
            • Video Allocators
            • Audio Allocators
          • Dynamic Bandwidth Adjustment
        • Finding
        • Video Formats
          • Receiver Codec Support Level
          • Frame Synchronization
      • Genlock
      • AV Sync
        • Guidelines
        • Creating and Destroying Devices
        • Recovering Audio
      • Using H.264, H.265, and AAC Codecs
        • Sending Audio Frames
        • Sending Video Frames
        • H.264 Support
        • H.265 Support
        • AAC Support
        • OPUS Support
        • Latency of Compressed Streams
        • Stream Validation
      • External Tally Support
      • KVM Support
      • NDI Advanced SDK FPGA Example Designs
        • Prebuilt uSD Images
        • NDI FPGA Reference Design
          • FPGA Projects
            • Changelog
          • C++ Application Code
            • Changelog
          • linux_kernel
            • Changelog
          • uSD Image Builder
            • Changelog
        • Changelog
    • Utilities
      • Unreal Engine SDK
        • Release Notes
        • Licensing
        • NDI Plugin Installation
        • Simple Setup of Broadcast and Receivers
        • NDI Broadcast Actor
        • NDI Receiver Actor
        • NDI Media Assets
        • Getting Started with Example Blueprint Projects
        • Advanced
      • Free Audio
      • Bridge Service
        • Installation
          • Silent Installation
        • Configuration
          • Web UI
          • Configuration File
          • Manual API Key Management
        • Webhooks
        • WebSockets
        • API
    • NDI Certified
      • Certification Guidelines
        • Interoperability Requirements
        • Technical Requirements
      • Certification Process
        • Pre-certification Checklist
        • Device Testing Methods
          • Camera
          • HDMI Encoder
          • SDI Encoder
          • Decoder
          • NDI Controller
          • NDI Monitor
          • Multicast Testing
        • Detailed process
    • Metadata
      • Metadata Sources
      • Metadata for XML
      • XML Validation
      • Metadata Elements
      • Proposed New Metadata Messages
      • PTZ and Control Messages
      • Undocumented Mysteries
  • Developer Guides
    • Decoding with NDI
    • NDI Bridge Deployment
  • FAQ
    • Index
    • NDI Tools
      • What is the NDI Analysis Tool and where do I get it from?
      • Why does my NDI connection stay active once the source is offline?
      • Why are my changes to the NDI JSON configuration file not being saved?
      • Why is the license for my Panasonic camera not active?
      • How does registration for NDI Tools work?
      • The time code on my file is incorrect, how do I change it?
      • How can I make NDI Tools launch automatically?
      • What is the NDI ECCN?
      • How do I use NDI output with Microsoft Teams
      • Where is Screen Capture (HX) for Mac?
    • NDI Certified
      • What is the NDI Certification Program?
      • Why did you start this certification program?​
      • What happens to my device after it's certified?
      • Is certification mandatory to be a partner of NDI?​
      • I am an OEM manufacturer, can my products be certified?​
      • How long does the certification process take?
      • How do I become certified?​
      • What happens if my product doesn’t meet the requirements for Certification?​
      • Are there any fees to become NDI Certified?
    • Common Issues
      • I'm having trouble getting multicast set up.
      • I'm having trouble with my NDI HX License
      • Why can't I find my Android 14-based NDI devices on my network?
      • Why won’t NDI Tools install on my Windows PC?
      • How do I enable NDI in “New” Microsoft Teams (Windows only)?
      • Why can’t HX Capture display the full resolution of my iPad or iPhone?
      • The NDI HX Camera app won't launch on older phones and iOS
      • Why won't MacOS Sonoma (14.1) recognize NDI Tools as a virtual camera?
      • I'm having issues with Virtual Input for macOS
      • I'm having issues with Final Cut Pro
      • NDI Camera App Issue
    • SDK
      • Where can I find the source code for the FPGA board?
      • What system resources are required to support a design including the NDI FPGA Codecs?
      • Why can’t my h264/265 video be received by an NDI receiver when using the embedded SDK?
      • Can I use the Unreal SDK on Mac?
      • What are the Differences Between the NDI SDK and the NDI Advanced SDK
    • ✨NDI 6
      • Do I need to upgrade to NDI 6 if I'm not using the new features?
      • Is there a fee to upgrade to NDI 6
      • Why doesn't my existing Vendor ID work with NDI 6?
      • How can I get a previous version of NDI Tools or the SDK?
    • NDI HX License Upgrades
      • What is happening with NDI HX Upgrades?
      • Will my existing HX-upgraded camera be affected?
      • I bought a camera before June 30, 2025, but didn't buy an HX license. Can I still get one?
      • I bought a license and a camera but didn't redeem it until after. Can I still get an HX license?
      • If my HX upgrade fails can I transfer the license?
      • Can I sell my upgraded HX camera and keep the license?
      • Will my NDI version work with my current HX license?
      • What does the HX upgrade sunset program mean for camera manufacturers?
      • What will NDI Support do for licenses after the sunset date?
      • I factory reset my camera and forgot the license. Can you help me get it back?
Powered by GitBook

2024 @ NDI Vizrt AB.

  1. Developing with NDI
  2. Advanced SDK
  3. AV Sync

Recovering Audio

Recovering Audio

The most typical use of the synchronizing function would be to pass capture a video frame by the means that you normally would, then to make a call to:

NDIlib_avsync_ret_e NDIlib_avsync_synchronize(
    NDIlib_avsync_instance_t p_avsync, 
    const NDIlib_video_frame_v2_t *p_video_frame, 
    NDIlib_audio_frame_v3_t* p_audio_frame
);

This function is however deceptively powerful and the key to understanding this is correctly passing the correct values into the NDIlib_audio_frame_v3_t* p_audio_frame parameter of this function. The following describes the values that may be specified.

Parameter
Description

no_samples

If this value is “0” when you pass it into the function, then if audio may be recovered then the synchronization function will automatically return the exact length of audio that matches the video frame that was passed in as the first parameter. The number of samples returned in this way will almost exactly match the values related to the timestamps but can be assumed to very accurately reflect the audio that matches the frame. Because the timestamps are often subject to noise when frames are stamped, the number of samples might vary slightly. This will result in a return code of NDIlib_avsync_ret_success.

If this value is some constant (e.g., 1601 or 1602) then the AV sync will attempt to return this number of samples if it is close to the true number of audio samples that are aligned with this frame. This is designed so that you let this class correctly recover audio that might follow some external constraint on the number of samples that are used with video frames. If the number of samples requested is sufficiently close to the number of audio samples that match this frame, then a return code of NDIlib_avsync_ret_success is returned. If it was not possible to correctly return this number of samples because it did not closely match the number of samples that are truly associated with this video frame, then the function will instead return the correct audio (which might be too many or too few samples) and return a value NDIlib_avsync_ret_success_num_samples_not_matched. It would then be the responsibility of the caller to determine how to best handle this condition. This condition is normally caused by trying to synchronize audio and video that are not on the same clock.

When requesting a specific number of audio samples, this is normally computed externally to this function under the assumption of some known audio sample rate. Because incoming audio might change sample rates which would render the number of requested samples invalid, please review the section below on how specifying the sample_rate parameter of p_audio_frame.

sample_rate

When this function is “0”, then there is no assumed sample rate, and the function will return an audio frame that specifies the current audio sample rate.

If you are specifying the number of samples to be captured as non-zero, it is likely that this was computed at a given audio sample-rate. If you specify this on the p_audio_frame as input, then if the sample rate of this audio source does not match it will not capture any audio and will return a result of NDIlib_avsync_ret_format_changed and fill in the audio format only in the returned structure. One can then simply recompute the number of required audio samples and simply call the function again to capture the audio with that video frame.

If the NDIlib_video_frame_v2_t *p_video_frame is not specified (is NULL) then the audio parameter can be used either to capture all current audio (no_samples=0) or a specified number of audio samples (no_samples is not zero) and behaves exactly as specified above although all audio is handled and not simply the audio associated with the current video frame.

Please note that this function will fill all parameters of the return frame, including the timecode, timestamp, and metadata. The metadata is chosen from the closest matching audio frame and is returned just one time. If the audio frames are all much smaller in duration than the corresponding video frames, then some metadata fields might be missing since they no longer have corresponding audio data to be assigned too.

It is important that you call NDIlib_avsync_free_audio in order to return the frames returned by NDIlib_avsync_syncronize.

The full set of return codes from this function are documented below. Please note that these have integer values which are positive for success and negative for failure.

Error code
Meaning

*_success

This function succeeded and returned audio that matches the frame, and if you specified sample_rate or no_samples then correctly match those constraints.

*t_success_num_samples_not_matched

This function succeeded, but you specified a no_samples that could not be matched exactly because this would push the audio and video frame sufficiently out of alignment. The full audio samples associated with this video frame are returned and it does not match no_samples. It might be a higher or lower number.

*_no_audio_stream_received

This indicates that there is currently no audio stream and so it would not be possible to return any audio data. This might indicate a video only stream.

*_ret_no_samples_found

This indicates that this video frame did not have matching audio data. This might be because the time at which this video frame was sent there was no corresponding audio. It might also indicate that the sending of the audio and video streams is sufficiently unaligned that they cannot easily be resynchronized; for instance, the audio data arriving is more than a second out of sync with the corresponding video data.

*_format_changed

A sample_rate and no_samples was specified; however the sample rate did not match and so this function has returned and correctly filled in the

*_ret_internal_error

This function was called with incorrect

Last updated 5 months ago

Was this helpful?