AMWA BCP-006-01: NMOS With JPEG XS
- Use of Normative Language
- Definitions
- JPEG XS IS-04 Sources, Flows and Senders
- JPEG XS IS-04 Receivers
- JPEG XS IS-05 Senders and Receivers
- Controllers
JPEG XS is a technology standardized in ISO/IEC 21122 for video contribution at low latency with very high video quality. A companion RTP payload format specification was developed through the IETF Payloads working group, IETF RFC 9134.
The Video Services Forum developed Technical Recommendation TR-08, which covers the end-to-end application use of JPEG XS compression for video, alongside uncompressed audio and VANC, using the SMPTE ST 2110 suite of protocols. TR-08 mandates the use of the AMWA IS-04 and IS-05 NMOS Specifications in TR-08 compliant systems.
AMWA IS-04 and IS-05 already have support for RTP transport and can signal the media type video/jxsv as defined in RFC 9134.
Use of Normative Language
The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in RFC 2119.
Definitions
The NMOS terms ‘Controller’, ‘Node’, ‘Source’, ‘Flow’, ‘Sender’, ‘Receiver’ are used as defined in the NMOS Glossary.
JPEG XS IS-04 Sources, Flows and Senders
Nodes capable of transmitting JPEG XS video streams MUST have Source, Flow and Sender resources in the IS-04 Node API.
Nodes MUST support IS-04 v1.3 to implement all aspects of this specification. Partial implementation can be achieved using IS-04 v1.2 and earlier.
Sources
The Source resource MUST indicate urn:x-nmos:format:video for the format.
Source resources can be associated with many Flows at the same time.
The Source is therefore unaffected by the use of JPEG XS compression.
Flows
The Flow resource MUST indicate video/jxsv in the media_type attribute, and urn:x-nmos:format:video for the format.
This has been permitted since IS-04 v1.1.
For Nodes implementing IS-04 v1.3 or higher, the following additional requirements on the Flow resource apply.
In addition to those attributes defined in IS-04 for all coded video Flows, the following attributes defined in the Flow Attributes register of the NMOS Parameter Registers are used for JPEG XS.
These attributes provide information for Controllers and Users to evaluate stream compatibility between Senders and Receivers.
- Components
The Flow resource MUST indicate the color (sub-)sampling using thecomponentsattribute. Thecomponentsarray value corresponds to thesampling,widthandheightvalues in the SDP format-specific parameters defined by RFC 9134. - Profile
The Flow resource MUST indicate the JPEG XS profile, which defines limits on the required algorithmic features and parameter ranges used in the codestream. The permittedprofilevalues are strings, defined as per RFC 9134. The Unrestricted profile is indicated by omitting this attribute. - Level
The Flow resource MUST indicate the JPEG XS level, which defines a lower bound on the required throughput for a decoder in the image (or decoded) domain. The permittedlevelvalues are strings, defined as per RFC 9134. The Unrestricted level is indicated by omitting this attribute. - Sublevel
The Flow resource MUST indicate the JPEG XS sublevel, which defines a lower bound on the required throughput for a decoder in the codestream (or coded) domain. The permittedsublevelvalues are strings, defined as per RFC 9134. The Unrestricted sublevel is indicated by omitting this attribute. - Bit Rate
The Flow resource MUST indicate the target bit rate (kilobits/second) of the codestream. Thebit_rateinteger value is expressed in units of 1000 bits per second, rounding up.
An example Flow resource is provided in the Examples.
Senders
For Nodes transmitting JPEG XS using the RTP payload mapping defined by RFC 9134, the Sender resource MUST indicate urn:x-nmos:transport:rtp or one of its subclassifications for the transport attribute.
Sender resources provide no indication of media type or format, since this is described by the associated Flow resource.
The SDP file at the manifest_href MUST comply with the requirements of RFC 9134.
Additionally, the SDP file needs to convey, so far as the defined parameters allow, the same information about the stream as conveyed by the Source, Flow and Sender attributes (or their defaults, when omitted) defined by this specification and IS-04.
Therefore:
- Each of the
profile,levelandsublevelformat-specific parameters MUST be included with the correct value unless it is Unrestricted. - The correct
sampling,depth,width,height,exactframerate,colorimetryandTCSMUST be included in all cases. - The
interlaceandsegmentedparameters MUST be included or omitted to correctly indicate the scanning/interlace mode.
If the Sender meets the traffic shaping and delivery timing requirements specified for ST 2110-22, the SDP file MUST also comply with the provisions of ST 2110-22.
For Nodes implementing IS-04 v1.3 or higher, the following additional requirements on the Sender resource apply.
In addition to those attributes defined in IS-04 for Senders, the following attributes defined in the Sender Attributes register of the NMOS Parameter Registers are used for JPEG XS.
- Bit Rate
The Sender resource MUST indicate the bit rate (kilobits/second) including the RTP transport overhead. Thebit_rateinteger value is expressed in units of 1000 bits per second, rounding up. The value is for the IP packets, so for the RTP payload format per RFC 9134, that includes the RTP, UDP and IP packet headers and the payload.
Informative note: This definition is consistent with the definition of the bit rate attribute required by ST 2110-22 in the SDP media description. - Packet Transmission Mode
If the Sender is using the slice packetization mode, it MUST include thepacket_transmission_modeattribute. Since the default value of this attribute iscodestream, the Sender MAY omit this attribute when using codestream packetization. - ST 2110-21 Sender Type
If the Sender complies with the traffic shaping and delivery timing requirements for ST 2110-22, it MUST include thest2110_21_sender_typeattribute.
An example Sender resource is provided in the Examples.
JPEG XS IS-04 Receivers
Nodes capable of receiving JPEG XS video streams MUST have a Receiver resource in the IS-04 Node API, which lists video/jxsv in the media_types array within the caps object.
This has been permitted since IS-04 v1.1.
If the Receiver has limitations on or preferences regarding the JPEG XS video streams that it supports, the Receiver resource MUST indicate constraints in accordance with the BCP-004-01 Receiver Capabilities specification. The Receiver SHOULD express its constraints as precisely as possible, to allow a Controller to determine with a high level of confidence the Receiver’s compatibility with the available streams. It is not always practical for the constraints to indicate every type of stream that a Receiver can or cannot consume successfully; however, they SHOULD describe as many of its commonly used operating points as practical and any preferences among them.
The constraint_sets parameter within the caps object can be used to describe combinations of frame rates, width and height, and other parameters which the Receiver can support, using the parameter constraints defined in the Capabilities register of the NMOS Parameter Registers.
The following parameter constraints can be used to express limits specifically defined by ISO/IEC 21122 and RFC 9134 for JPEG XS decoders:
When the JPEG XS decoder supports the Unrestricted profile, level or sublevel, the Receiver can indicate that the parameter is unconstrained, as described in BCP-004-01.
When the decoder does not support Unrestricted but supports a range of profiles, levels or sublevels, the enum Constraint Keyword can be used to indicate the acceptable values.
Other existing parameter constraints, such as the following, are also appropriate to express limitations on supported JPEG XS video streams:
- Frame Width
- Frame Height
- Color Sampling
- Component Depth
- Format Bit Rate
- Transport Bit Rate
- ST 2110-21 Sender Type
An example Receiver resource is provided in the Examples that demonstrates how to represent VSF TR-08 interoperability points using these parameter constraints.
JPEG XS IS-05 Senders and Receivers
Connection Management using IS-05 proceeds in exactly the same manner as for any other stream format carried within RTP.
The SDP file at the /transportfile endpoint on an IS-05 Sender MUST comply with the same requirements described for the SDP file at the IS-04 Sender manifest_href.
A PATCH request on the /staged endpoint of an IS-05 Receiver can contain an SDP file in the transport_file attribute.
The SDP file for a JPEG XS stream is expected to comply with RFC 9134 and, if appropriate, ST 2110-22.
It need not comply with the additional requirements specified for SDP files at Senders.
If the Receiver is not capable of consuming the stream described by the SDP file, it SHOULD reject the request. If it is unable to assess the stream compatibility, for example when the SDP file does not include some of the optional parameters defined by RFC 9134, it MAY accept the request.
An example SDP file is provided in the Examples.
Controllers
Controllers MUST use IS-04 to discover JPEG XS Senders and Receivers and IS-05 to manage connections between them. Controllers MUST support IS-04 v1.3 to implement all aspects of this specification. Partial implementation can be achieved using IS-04 v1.2 and earlier.
Controllers MUST support the BCP-004-01 Receiver Capabilities mechanism and all the parameter constraints listed in this specification in order to evaluate the stream compatibility between JPEG XS Senders and Receivers.