Networked Media Open Specifications

AMWA BCP-006-02: NMOS With H.264

Index↑

WORK IN PROGRESS … DOCUMENT UNDER CONSTRUCTION

Introduction

H.264 is a video compression technology standardized in Rec. ITU-T H.264 | ISO/IEC 14496-10. A companion RTP payload format specification was developed through the IETF Payloads working group, IETF RFC 6184 for the transport of an H.264 bitstream over RTP.

The BCP-006-02 specification includes support for H.264 bitstreams that are compliant with the clauses of the main document and annexes A, B, C, D and E of the Rec. ITU-T H.264 specification. It excludes support for bitstreams that are compliant with other annexes of the specification.

Annex F (deprecated), Annex G (scalable video coding), Annex H (multiview video coding), Annex I (multiview and depth video coding) and Annex J (multiview and depth video with enhanced non-base view coding) are not supported.

The Rec. ITU-T H.222.0 ISO/IEC 13818-1 specification and associated amendments describe the embedding of an H.264 stream in an MPEG2-TS transport stream. An RTP payload format specification for MPEG2-TS transport stream was developed through the IETF Payloads working group, IETF RFC 2250 for transport over RTP. Other normative documents describe the requirements for the streaming of an MPEG2-TS transport stream over other non-RTP transports.

The Society of Media Professionals, Technologists and Engineers developed Standard ST 2110-22 of the ST 2110 suite of protocols, which cover the end-to-end application use of constant bit rate compression for video over managed IP networks.

Note that the definition of constant bit rate of ST 2110-22 is very strict. “The video compression or the packetization of the video compression shall produce a constant number of bytes per frame. The packetization shall produce a constant number of RTP packets per frame.” This definition of constant bit rate is hereafter described as strict-CBR, using the H.264 definition of constant bit rate for CBR.

The Video Services Forum developed Technical Recommendation TR-10-11 and TR-10-?? of the IPMX suite of protocols, which cover the end-to-end application use of constant and variable bit rate compression for video, using the SMPTE ST 2110 and IPMX suite of protocols.

TR-10-11 and TR-10-?? mandate the use of the AMWA IS-04 and IS-05 NMOS Specifications in IPMX compliant systems.

AMWA IS-04 and IS-05 have support for various transport protocols and can signal the media type video/H264 as defined in RFC 6184.

Use of Normative Language

The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in RFC 2119.

Definitions

The NMOS terms ‘Controller’, ‘Node’, ‘Source’, ‘Flow’, ‘Sender’, ‘Receiver’ are used as defined in the NMOS Glossary.

The term ‘strict-CBR’ corresponds to the definition of constant bit-rate at section 4 “Video Compression and Packetization” of the ST 2110-22 standard.

The terms CBR and VBR are defined in the Rec. ITU-T H.264 standard.

H.264 IS-04 Sources, Flows and Senders

Nodes capable of transmitting H.264 video streams MUST have Source, Flow and Sender resources in the IS-04 Node API.

Sources

The Source resource MUST indicate urn:x-nmos:format:video for the format attribute.

Flows

The Flow resource MUST indicate video/H264 in the media_type attribute, and urn:x-nmos:format:video for the format attribute. This has been permitted since IS-04 v1.1.

For Nodes implementing IS-04 v1.3 or higher, the following additional requirements on the Flow resource apply.

In addition to those attributes defined in IS-04 for all coded video Flows, the following attributes defined in the Flow Attributes Register of the NMOS Parameter Registers are used for H.264.

These attributes provide information for Controllers and Users to evaluate stream compatibility between Senders and Receivers.

The Flow’s profile and level attributes map to the profile-level-id parameter of the SDP transport file. See the SDP format-specific parameters section.

The Flow’s profile and level attributes map to the members profile_idc, level_idc and constraint_set<N>_flag of the AVC_video_descriptor of an MPEG2-TS transport stream. See the RTP transport based on RFC 2250 section.

Informative note: The Flow’s profile and level attributes are always required. The SDP transport file profile-level-id parameter may be omitted when matching the default value.

Informative note: The maximum bit rate information relates to the codec profile / level limits and the HRD buffering model. The CBR versus VBR mode of operation of the encoder provide essential clues about the coded bitstream produced.

Informative note: For streams compliant with ST 2110-22, the constant bit rate mode is more appropriately described as a strict-CBR mode where “the video compression or the packetization of the video compression shall produce a constant number of bytes per frame. The packetization shall produce a constant number of RTP packets per frame.” For other streams, not compliant with ST 2110-22, it is the constant bit rate definition of the H.264 specification that prevails.

The H.264 encoder associated with the Flow MUST produce an H.264 bitstream that is compliant with the profile and level attributes.

Examples Flow resources are provided in Examples.

Senders

This section applies to a Sender directly associated with an H.264 Flow through the Sender’s flow_id attribute.

Informative note: When an H.264 Flow is not directly associated with a Sender but with a multiplexed Flow through the Flow’s parents attribute, the Sender does not provide H.264 format-specific attributes. A Sender provides such attributes only when the H.264 Flow is directly associated with it. The attributes parameter_sets_flow_mode and parameter_sets_transport_mode associated with an H.264 Flow that are described in this section get their respective default value of dynamic and in_band.

For multiplexed H.264 Flows, the Sender MUST conform to the behaviour dictated by the dynamic and in_band modes described in this specification.

Sender resources provide no indication of media type or format, since this is described by the associated Flow resource.

RTP transport based on RFC 6184

For Nodes transmitting H.264 using the RTP payload mapping defined by RFC 6184, the Sender resource MUST indicate urn:x-nmos:transport:rtp or one of its subclassifications for the transport attribute.

For Nodes implementing IS-04 v1.3 or higher, the following additional requirements on the Sender resource apply.

In addition to those attributes defined in IS-04 for Senders, the following attributes defined in the Sender Attributes Register of the NMOS Parameter Registers are used for H.264.

An example Sender resource is provided in the Examples.

SDP format-specific parameters

The SDP file at the manifest_href MUST comply with the requirements of RFC 6184 in the Usage in Declarative Session Descriptions mode of operation. The SDP Offer/Answer Model described in Section 8.2.2 of RFC 6184 is not supported. The fmtp source attribute as specified in Section 6.3 of RFC 5576 is not supported, i.e. use-level-src-parameter-sets parameter is not present or equal to 0.

Additionally, the SDP transport file needs to convey, so far as the defined format-specific parameters allow, the same information about the stream as conveyed by the Source, Flow and Sender attributes defined by this specification and IS-04, unless such information is conveyed through in-band parameter sets.

Therefore:

The stream’s active parameter sets MUST be compliant with the format-specific parameters, except sprop-parameter-sets, declared in the “fmtp=” attribute of an SDP transport file.

If the Sender meets the traffic shaping and delivery timing requirements specified for ST 2110-22, the SDP transport file MUST also comply with the provisions of ST 2110-22.

Informative note: ST 2110-22 does not require the sampling or depth SDP parameters. RFC 6184 does not define any such SDP parameters. The sampling and depth of the associated uncompressed picture could be derived from the H.264 active parameter sets by a Receiver.

If the Sender meets the traffic shaping and delivery timing requirements specified for IPMX, the SDP transport file MUST also comply with the provisions of IPMX.

An example SDP file is provided in the Examples.

Other transports

For Nodes transmitting H.264 using other transports, the Sender resource MUST indicate the associated urn:x-nmos:transport: label of the transport or one of its subclassifications for the transport attribute.

Sender resources provide no indication of media type or format, since this is described by the associated Flow resource.

The manifest_href attribute MAY be null if an SDP transport file is not supported by the transport. Otherwise the SDP transport file MUST comply with the transport specific requirements. There is no SDP format-specific parameters requirements for transports other than RTP.

H.264 IS-04 Receivers

Nodes capable of receiving H.264 video streams MUST have Receiver resources in the IS-04 Node API.

This section applies to a Receiver directly or indirectly associated with an H.264 stream.

Informative note: When an H.264 stream is directly associated with a Receiver, the Receiver has format set to urn:x-nmos:format:video and media_types of the caps attribute contains video/H264. When an H.264 stream is part of a multiplexed stream and is indirectly associated with a Receiver, the Receiver has format set to urn:x-nmos:format:mux, media_types of the caps attribute does not contains video/H264 and constraint_sets of the caps attribute contains video/H264.

For a Receiver directly associated with an H.264 stream, the Receiver resource MUST indicate urn:x-nmos:format:video for the format attribute and MUST list video/H264 in the media_types array within the caps object. This has been permitted since IS-04 v1.1.

For a Receiver indirectly associated with an H.264 stream part a multiplexed stream, the Receiver resource MUST indicate urn:x-nmos:format:mux for the format attribute, MUST list the mux media type in the media_types array within the caps object and MUST list a constraint set indicating support for the media type video/H264 in the constraint_sets array within the caps object.

If the Receiver has limitations on or preferences regarding the H.264 video streams that it supports, the Receiver resource MUST indicate constraints in accordance with the BCP-004-01 Receiver Capabilities specification. The Receiver SHOULD express its constraints as precisely as possible, to allow a Controller to determine with a high level of confidence the Receiver’s compatibility with the available streams. It is not always practical for the constraints to indicate every type of stream that a Receiver can or cannot consume successfully; however, they SHOULD describe as many of its commonly used operating points as practical and any preferences among them.

The constraint_sets parameter within the caps object MUST be used to describe combinations of frame rates, width and height, and other parameters which the receiver can support, using the parameter constraints defined in the Capabilities register of the NMOS Parameter Registers.

When the H.264 decoder has no restrictions on the value of some parameter, the Receiver can indicate that the parameter is unconstrained, as described in BCP-004-01.

The following parameter constraints can be used to express limits or preferences specifically defined by Rec. ITU-T H.264 for H.264 decoders:

A Receiver MUST be able to decode bitstreams conforming to the profiles and levels declared in the Receiver Capabilities. A Receiver MAY have preferences and more optimal profiles and levels that MAY be declared through Receiver Capabilities. A preferred constraint set MAY indicate such preferences while another constraint set MAY indicate full support of some profiles and levels. A Receiver MAY further constrain the support of a coded bitstream compliant with a profile and level using other constraints in its Receiver Capabilities.

Other existing parameter constraints, such as the following, are also appropriate to express limitations on supported H.264 video streams:

An example Receiver resource is provided in the Examples.

RTP transport based on RFC 6184

For Nodes consuming H.264 using the RTP payload mapping defined by RFC 6184, the Receiver resource MUST indicate urn:x-nmos:transport:rtp or one of its subclassifications for the transport attribute.

The following parameter constraints can be used to express limits or preferences specifically defined for H.264 decoders:

RTP transport based on RFC 2250

For Nodes consuming H.264 using the RTP payload mapping defined by RFC 2250, the Receiver resource MUST indicate urn:x-nmos:transport:rtp or one of its subclassifications for the transport attribute.

The following parameter constraints can be used to express limits or preferences specifically defined for H.264 decoders:

Other transports

For Nodes consuming H.264 using other transports, the Receiver resource MUST indicate the associated urn:x-nmos:transport: label of the transport or one of its subclassifications for the transport attribute.

For Receivers indicating urn:x-nmos:format:video for the format attribute, the following parameter constraints can be used to express limits or preferences specifically defined for H.264 decoders:

For Receivers indicating urn:x-nmos:format:mux for the format attribute, the following parameter constraints can be used to express limits or preferences specifically defined for H.264 decoders:

H.264 IS-05 Senders and Receivers

RTP transport

Connection Management using IS-05 proceeds in exactly the same manner as for any other stream format carried within RTP.

If IS-04 Sender manifest_href is not null, the SDP transport file at the /transportfile endpoint on an IS-05 Sender MUST comply with the same requirements described for the SDP transport file at the IS-04 Sender manifest_href.

A PATCH request on the /staged endpoint of an IS-05 Receiver can contain an SDP transport file in the transport_file attribute. The SDP transport file for a H.264 stream is expected to comply with RFC 6184 and, if appropriate, ST 2110-22 or IPMX. It need not comply with the additional requirements specified for SDP transport files at Senders.

If the Receiver is not capable of consuming the stream described by a PATCH on the /staged endpoint, it SHOULD reject the request. If it is unable to assess the stream compatibility because some parameters are not included PATCH request, it MAY accept the request and postpone stream compatibility assessment.

Other transports

Connection Management using IS-05 proceeds in exactly the same manner as for any other stream format carried within other transports.

If IS-04 Sender manifest_href is not null, the SDP transport file at the /transportfile endpoint on an IS-05 Sender MUST comply with the same requirements described for the SDP transport file at the IS-04 Sender manifest_href.

If the Receiver is not capable of consuming the stream described by a PATCH on the /staged endpoint, it SHOULD reject the request. If it is unable to assess the stream compatibility because some parameters are not included PATCH request, it MAY accept the request and postpone stream compatibility assessment.

Parameter Sets

The active parameter sets of an H.264 stream are made of the active sequence and picture parameter sets. The parameter sets of an H.264 stream are transmitted by a Sender to a number of Receivers either out-of-band through an SDP transport file or other out-of-band transport specific mechanism, or in-band through the coded stream. The active parameter sets of an H.264 coded stream MUST reference valid out-of-band or in-band parameter sets, the stream is invalid otherwise.

The sprop-parameter-sets parameter of an SDP transport file MAY contain a collection of out-of-band parameter sets. Those parameter sets provide initial parameter sets for the H.264 stream before the decoding starts.

An H.264 stream MAY transport in-band parameter sets to update or duplicate parameter sets received out-of-band, to define additional parameter sets or to update or duplicate parameter sets received in-band. The parameter_sets_transport_mode Receiver Capability indicates when set to in_band or in_and_out_of_band that a Receiver supports getting updated, duplicated or additional parameter sets in-band. When set to out_of_band a Receiver only supports getting initial parameter sets from the SDP transport file. In all the cases the Receiver MUST support getting duplicated parameter sets in-band.

The sprop-parameter-sets parameter MAY be used by the Receiver to assert the parameter_sets_transport_mode in use by the Sender. The in-and-out-of-band mode is signaled if sprop-parameter-sets is present and not empty, and terminates by an empty NAL unit (i.e. an empty byte sequence that is base64 encoded). An sprop-parameter-sets parameter terminates by an empty NAL unit if it ends by a comma ‘,’. The terminating colon ‘,’ indicates that more parameter sets will be received in-band, indicating the in-and-out-of-band mode. The out-of-band mode is signaled when sprop-parameter-sets is present and not empty, and not terminated by a comma ‘,’. The in-band mode is signaled when sprop-parameter-sets is not present or empty. A Sender MUST terminate the sprop-parameter-sets parameter by a comma ‘,’ if it operates in in-and-out-of-band mode and MUST not terminate it by a comma ‘,’ in out-of-band mode.

Informative note: It results that an H.264 Sender operating in in-and-out-of-band mode that is not sending parameter sets out-of-band will set the sprop-parameter-sets parameter to a single comma ‘,’ value.

Informative note: The in-and-out-of-band mode implies that the Sender can optionally use the in-band path, the out-band path or both paths to transmit parameter sets. There is no requirement to use both paths simultaneously.

Receivers

A Receiver MUST verify that the active parameter sets comply with the Receiver’s Capabilities. If a Receiver support only out-of-band parameter sets it SHOULD perform the verification when a Controller PATCH the /staged endpoint for activation. In this situation, all the out-of-band parameter sets MUST be compliant with the Receiver Capabilities. Otherwise if a Receiver supports both out-of-band and in-band parameter sets it SHOULD perform the verification of the out-of-band parameter sets when a Controller PATCH the /staged endpoint for activation and it MUST perform the verification of the in-band parameter sets just-in-time as it decodes the stream. In this situation, all the out-of-band and in-band parameter sets MUST be compliant with the Receiver Capabilities.

The parameter_sets_flow_mode Receiver Capability indicates when set to dynamic that a Receiver supports decoding an H.264 stream where the active parameter sets values associated with Flow attributes may changes dynamically. The Flow attributes that are allowed to change correspond to the following attributes of a coded video Flow: grain_rate, frame_width, frame_height, colorspace, interlace_mode, transfer_characteristic, components, profile, level, bit_rate and constant_bit_rate. The static mode is more restrictive, requiring active parameter sets values associated with Flow attributes to be constant, with the exception of the Flow’s bit_rate attribute that MAY change. The strict mode has the further restriction that at most one SPS be used by the coded stream.

Informative note: The Flow bit_rate attribute is not included in the previous criterion to allow changing HRD parameters to adapt to IP usable bandwidth changing conditions in static mode.

A Receiver with the parameter_sets_flow_mode capability set to strict requires that a coded stream uses picture parameter sets that are defined once and associated with at most one Flow and at most one sequence parameter set that is defined once. Such parameter sets MAY be obtained out-of-band or in-band depending on the Receiver parameter_sets_transport_mode capability. When obtained out-of-band the sprop-parameter-sets parameter of an SDP transport file or the parameter sets from a transport specific out-of-band mechanism MUST contain picture parameter sets associated with only one Flow and one sequence parameter set. When obtained in-band the sprop-parameter-sets parameter of an SDP transport file or the parameter sets from a transport specific out-of-band mechanism MUST be empty or omitted and the Sender transmits the picture parameter sets associated with the Flow in-band along with a sequence parameter set. At all time, the Sender MAY transmit in-band parameter sets that are duplicates of the parameter sets obtained either out-of-band or in-band. When obtained in-and-out-of-band, in-band parameter sets have priority over out-of-band parameters sets.

A Receiver with this capability set to static requires that a coded stream uses parameter sets associated with at most one Flow. Such parameter sets MAY be obtained out-of-band or in-band depending on the Receiver parameter_sets_transport_mode capability. When obtained out-of-band the sprop-parameter-sets parameter of an SDP transport file or the parameter sets from a transport specific out-of-band mechanism MUST contain parameter sets associated with only one Flow. When obtained in-band the sprop-parameter-sets parameter of an SDP transport file or the parameter sets from a transport specific out-of-band mechanism MUST be empty or omitted and the Sender transmits the parameter sets associated with the Flow in-band. At all time, the Sender MAY transmit in-band parameter sets that are duplicates of the parameter sets obtained either out-of-band or in-band. When obtained in-and-out-of-band, in-band parameter sets have priority over out-of-band parameters sets.

Informative note: The static mode allows multiple SPS and PPS to be used by the coded bitstream as long as the active values associated with Flow attributes do not change. For example scaling list are allowed to change when the profile allows it.

A Receiver with this capability set to dynamic supports that a coded stream uses parameter sets that MAY be associated with multiple Flows on the Sender. Such parameter sets MAY be obtained out-of-band or in-band according to the Receiver parameter_sets_transport_mode capability. When obtained out-of-band the sprop-parameter-sets parameter of an SDP transport file or the parameter sets from a transport specific out-of-band mechanism MAY contain parameter sets associated with multiple Flows. When obtained in-band the sprop-parameter-sets parameter of an SDP transport file or the parameter sets from a transport specific out-of-band mechanism MUST be empty or omitted and the Sender MAY transmit in-band parameter sets to update parameter sets initially received out-of-band or to add and update new ones. At all time, the Sender MAY transmit in-band parameter sets that are duplicates of the parameter sets obtained either out-of-band or in-band. When obtained in-and-out-of-band, in-band parameter sets have priority over out-of-band parameters sets.

Senders

A Sender MAY, unless constrained by IS-11, produce any H.264 coded stream that is compliant with the profile and level of the associated Flow. Such a Sender MAY use one or multiple active parameter sets as per the H.264 specification. A Sender MAY seamlessly change dynamically the coded stream’s active parameter sets, provided that the Flow associated with the Sender changes accordingly and the content of the SDP transport file does not change. If the content of the SDP transport file changes, the Sender MUST comply with IS-04, IS-05. A Sender indicates its mode of operation with the parameter_sets_flow_mode and parameter_sets_transport_mode attributes.

A Sender MUST transport parameter sets in_band when the H.264 stream is transmitted over an MPEG2-TS based transport as per the H.222.0 specification.

A Sender operating with parameter_sets_flow_mode set to strict MUST produce a coded bitstream using at most one SPS and MAY use a number of PPS associated with at most one Flow. The parameter sets MUST be defined once in-band or out-of-band and MAY be refreshed by in-band duplicates.

A Sender operating with parameter_sets_flow_mode set to static MAY produce a coded bitstream using a number of SPS and PPS that MUST be associated with at most one Flow. The parameter sets MAY be defined in-band or out-of-band and MAY be refreshed by in-band duplicates or updated, replaced or augmented by in-band ones.

A Sender operating with parameter_sets_flow_mode set to dynamic MAY produce a coded bitstream using a number of SPS and PPS that MAY be associated with multiple Flows. The parameter sets MAY be defined in-band or out-of-band and MAY be refreshed by in-band duplicates or updated, replaced or augmented by in-band ones.

H.264 IS-11 Senders and Receivers

RTP transport

Other transports

Controllers

Index↑

Documentation built at 04:50:50 CDT on 2024-08-29.