Networked Media Open Specifications

AMWA BCP-007-01: NMOS With NDI

Index↑

Introduction

NDI (Network Display Interface) is an IP transport and control technology created by Newtek, a division of Vizrt Group. It includes definitions of encoding, transport and provides a full SDK to implement IP media transport. This document outlines how NDI devices can be managed through NMOS IS-04 and IS-05.

Familiarity with the JT-NM Reference Architecture and the NDI® SDK are assumed.

See also the NMOS Technical Overview.

Use of Normative Language

The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in RFC-2119.

Dependencies

This document was written based upon NDI® Advanced SDK Version 5.5, however devices which implement other versions of the SDK MAY also be supported.

This document depends upon the following reference documents:

Definitions

The NMOS terms ‘Controller’, ‘Node’, ‘Source’, ‘Flow’, ‘Sender’, ‘Receiver’ are used as defined in the NMOS Glossary.

This specification also defines the following terms:

NDI SDK

The NDI Software Development Kit (SDK) is offered in two variants. The base version is referred to here as NDI Standard SDK, The Advanced SDK is be referred to here as NDI Advanced SDK. References to NDI SDK apply to both NDI Standard SDK and NDI Advanced SDK.

Native NDI Device

A “device” as defined in the NDI SDK. This is not the same as a “Device” as defined in the NMOS Glossary. Note that the same physical or logical apparatus MAY simultaneously act as an NMOS Device and a Native NDI Device.

Non-NMOS NDI Device

A Native NDI Device which does not support the NMOS specifications.

NDI Stream

A “stream” as defined in the NDI SDK. An NDI Stream is a multiplexed structure that could include one or more video, audio and metadata flows.

Native NDI Sender

A “sender” of an NDI stream as defined in the NDI SDK. This is not the same as a “Sender” as defined in the NMOS Glossary. Note that the same physical or logical apparatus MAY simultaneously act as an NMOS Sender and a Native NDI Sender.

Native NDI Receiver

A “receiver” of an NDI stream as defined in the NDI SDK. This is not the same as a “Receiver” as defined in the NMOS Glossary. Note that the same physical or logical apparatus MAY simultaneously act as an NMOS Receiver and a Native NDI Receiver.

NDI Device

A Native NDI Device which implements IS-04 and allows registration in an NMOS Registry and implements IS-05 for connection management.

NDI Sender

An NMOS Sender which implements the NDI transport.

NDI Receiver

An NMOS Receiver which implements the NDI transport.

NDI Full Bandwidth

An NDI stream which utilizes proprietary codecs for audio and video. NDI Full Bandwidth is supported by both the NDI Standard SDK and NDI Advanced SDK.

NDI HX, NDI HX2, NDI HX3

NDI High Efficiency profiles, named NDI HX, NDI HX2, and NDI HX3, utilize H.264/AVC, H.265/HEVC, AAC and Opus codecs. These are supported by the NDI Advanced SDK only.

NDI HX utilizes Long-GOP H.264 video encoding at maximum Full HD resolution and AAC audio encoding. NDI HX is sometimes stylized NDI|HX in some documentation.

NDI HX2 utilizes H.264 or H.265 Long-GOP video coding up to UHD resolution and AAC or Opus audio encoding. NDI HX2 is sometimes stylized NDI|HX2 in some documentation.

NDI HX3 utilizes H.264 or H.265 Short-GOP video coding up to UHD resolution and AAC or Opus audio encoding. NDI HX3 is sometimes stylized NDI|HX3 in some documentation.

NDI

This document uses the term “NDI” when referring to all NDI variants, and specify “NDI Full Bandwidth”, “NDI HX”, “NDI HX2”, or “NDI HX3” where the text applies to specific NDI variants.

Native NDI Model

NDI Streams utilize a variety of codecs to compress media. In many cases, the NDI SDK negotiates between Native NDI Devices to select the transport and encoding parameters used for an NDI Stream. A Native NDI Receiver does not learn about the encoding and transport parameters of an NDI Stream until a connection is established.

The NDI SDK, by default, automatically selects and negotiates encoding parameters between Native NDI Devices. Media content enters the Native NDI Sender as raw, uncompressed media and raw, uncompressed media emerges from Native NDI Receivers.

The NDI Advanced SDK supports compressed NDI Streams utilizing H.264, H.265, AAC and Opus codecs. Media content enters the Native NDI Sender as compressed media and compressed media emerges from Native NDI Receivers.

NDI Metadata

Metadata connections can be implicitly established by the NDI SDK when video connections are established. In some cases, bi-directional metadata connections can be established by the NDI SDK between the Native NDI Sender and Native NDI Receiver. In such cases, NDI Metadata MAY NOT be represented explicitly in NMOS.

NMOS-NDI Model

Native NDI Devices, Native NDI Receivers and Native NDI Senders are represented by NMOS Devices/Nodes, Receivers and Senders respectively.

A controller which supports NDI connection management via IS-05 MUST support connection of NDI Receivers to NDI Senders. This controller MAY also support connection of NDI Receivers to Native NDI senders, however the discovery of such senders is outside the scope of this document.

Non-NMOS Connections to NDI Devices

NDI Receivers MUST NOT preclude connections established with Native NDI Senders via the NDI SDK. If an NDI Stream is connected to an NDI Receiver outside of IS-05, the transport parameters of the Connection API of the Receiver MUST be updated to reflect the running parameters which have been set via the alternate protocol. The Receiver MUST update its IS-04 resource subscription.active property accordingly, set subscription.sender_id to null and increment it’s version number. If the resource has been registered with an NMOS registry the entry MUST be updated to match the Node API.

Multiplexed Flow Model

Receivers of NDI Streams MUST have the format attribute set to urn:x-nmos:format:mux and Senders of NDI Streams MUST be associated with a Flow having the format attribute set to urn:x-nmos:format:mux, as NDI connections could contain multiple essences including video, audio and metadata.

A Flow of urn:x-nmos:format:mux format MUST have parents video or audio sub-Flows. The current NDI SDK limits to a maximum of one of each video, audio and metadata sub-Flow, but this specification supports future SDK versions which MAY support multiple audio, video or metadata sub-Flows.

MUX Sender Model

NDI IS-04 Resources

Flows

MUX Flow

An NDI Sender MUST be associated with a mux Flow having the format attribute set to urn:x-nmos:format:mux and the media_type attribute set to application/ndi.

The mux Flow MUST have parent video and/or audio sub-Flows identifying the sub-Flows multiplexed in the NDI Stream. The mux Flow MUST be associated with a Source of format urn:x-nmos:format:mux.

Video sub-Flows

For NDI Full Bandwidth the video sub-Flows media_type attribute MUST be video/raw.

For NDI HX, HX2 and HX3 the video sub-Flows media_type attribute MUST be video/H264 or video/H265.

NDI Senders SHOULD map the employed NDIlib_FourCC_video_type to the NMOS bit-depth, component and sub-sampling properties.

NDI video+alpha video flows MUST be modeled as a single video sub-Flow, including a channel labelled A in the components parameters for the alpha channel.

Audio sub-Flows

For NDI Full Bandwidth the audio sub-Flows media_type SHOULD be a supported PCM type such as audio/L16, audio/L20, or audio/L24.

For NDI HX, HX2 and HX3 the audio sub-Flows media_type attribute MUST be audio/mpeg4-generic or audio/opus.

Sources

The Source associated with a mux Flow MUST have its format attribute set to urn:x-nmos:format:mux and its parents attribute MUST enumerate the Sources associated with the sub-Flows.

Senders

NDI Senders do not utilize SDP to describe the NDI Stream; therefore NDI Senders MUST have their manifest_href attribute set to null.

NDI Senders MUST have their transport attribute set to urn:x-nmos:transport:ndi.

NDI Senders MUST be associated with a mux Flow.

Receivers

The NDI Receiver MUST have its format attribute set to urn:x-nmos:format:mux.

Receiver Capabilities

An NDI Receiver MUST specify as a minimum the following capabilities:

"caps" : {
    "media_types" : [
       "application/ndi"
    ]
}

Additional capabilities MAY be expressed in the constraint_sets array attribute of the Receiver for the stream of media type application/ndi and for the associated sub-streams.

NDI IS-05 Resources

Transport Type

The IS-05 transport for NDI is urn:x-nmos:transport:ndi.

Sender transport_file

Not used.

Sender Transport Parameters

The IS-05 schemas sender_transport_params_ndi.json and constraints_schema_sender_ndi.json describe the transport parameters associated with the NDI transport.

[
  {
    "server_ip": "10.10.10.10",
    "server_port": 5960,
    "source_name": "ndi-sender-unique-name",
    "group_name": "camera1"
  }
]

server_ip: IP address hosting the NDI server (IP address of interface bound to the server). If the parameter is set to auto the Sender MUST establish for itself which interface it can use, based on its own internal configuration. A null value indicates that the Sender has not yet been configured.

server_port: Port for the NDI server. If the parameter is set to auto the Sender MUST establish for itself which port it can use, based on its own internal configuration.

source_name: The name of the NDI stream as declared by the NDI Sender.

group_name: The NDI group of the source. null indicates the default group.

Although the NDI Advanced SDK does allow NDI Native Devices to specify additional Sender transport parameters, these parameters and properties SHOULD NOT be exposed in NMOS.

Receiver Parameters

The IS-05 schemas receiver_transport_params_ndi.json and constraints_schema_receiver_ndi.json describe the transport parameters associated with the NDI transport.

[
  {
    "interface_ip": "10.10.10.20",
    "server_host": "10.10.10.10",
    "server_port": 5960,
    "source_name": "ndi-sender-unique-name",
    "group_name": "camera1"
  }
]

interface_ip: IP address of the network interface the receiver MUST use. If set to auto the receiver MUST determine which interface to use.

server_host: Hostname or IP hosting the NDI server. A null value indicates that the Receiver has not yet been configured.

server_port: Port for the NDI server. If set to auto the receiver MUST determine which port to use.

source_name: The name of the NDI stream as declared by the NDI sender.

group_name: The NDI group of the source, null indicates the default group.

Although the NDI Advanced SDK does allow NDI Native Devices to specify additional Receiver transport parameters, these parameters and properties SHOULD NOT be exposed in NMOS.

Controllers

A controller MUST be able to discover NDI Senders and NDI Receivers by using the query API.

A controller MUST be able to connect an NDI Receiver to an NDI Sender by using the IS-05 API. See example Use Case 1.

A controller MAY be able to discover Native NDI Senders.

A controller MAY be able to connect an NDI Receiver to a Native NDI Sender by using the IS-05 API. See example Use Case 2.

Use Case Scenarios (Informative)

Use cases are informative only and provided as examples.

NMOS-NDI Model

Use Case 1: NDI Sender to NDI Receiver - IS-05 Connection

NMOS-NDI Model

Use Case 2: Native NDI Sender to NDI Receiver - IS-05 Connection

NMOS-NDI Model

Use Case 3: Native NDI Sender to NDI Receiver - Non-NMOS Connection

NMOS-NDI Model

Use Case 4: NDI Sender to Native NDI Receiver - Non-NMOS Connection

NMOS-NDI Model

Use Case 5: Native NDI Sender to Native NDI Receiver - Non-NMOS Connection

NMOS-NDI Model

Use Case 6: NDI Sender to NDI Receiver - Non-NMOS Connection

NMOS-NDI Model

Index↑

Documentation built at 12:57:54 CDT on 2023-10-04.
(c) AMWA 2023. RAML/JSON licensed under Apache 2.0. Documentation licensed under CC BY-ND 4.0.