Explanation –Flow
←Explanation - Source · Index↑ · Explanation - Flow Representation→
Please ensure you have read the Summary and Definitions before considering the further detail covered by this page.
Understanding the Flow Entity
The intention is that a Flow (identified by a Flow ID) provides a simple way to identify a particular expression of a Source. A Flow is a timed-data interface that might be implemented in different ways. These different implementations of a Flow are called Flow Representations.
In general, it is important not to think of a Flow as something that is stored in a file or carried in network packets – that is a Flow Representation and is just one practical realisation of the Flow. The Flow ID carried along with, or associated with, this Flow Representation is of importance here. It is this Flow ID that identifies the content and the interface (that is, as well as identifying the content it communicates what can be expected when handling the data). It helps to inform the user about how the data can be handled. For example:
- It might be that a decoder is expected to handle either all the
Flow Representations of a particularFlowor only a subset of theFlow Representations. - It might be that either: each transport used to carry a particular
Flowrequires a differentFlow Representation; or that all transports used can carry any of theFlow Representations.
In each scenario a decision needs to be made about the level at which to set the Flow. The most practical level is chosen: this is the lowest level that is considered interesting for the scenario. A Flow might be set at a low level in which case all associated Flow Representations will be very similar. Alternatively, a Flow can be set at a higher level in which case the associated Flow Representations can be much more varied in how they implement the Flow.
There is a need to work across a very wide range of different scenarios (involving both media and data; that is, all kinds of content Sources). Across these scenarios there are practical reasons for handling data at different levels. In some scenarios, the sequences of bytes used to represent the data are of interest and this is the level we principally want to work at. In other scenarios, these sequences of bytes are not of interest and we principally want to handle the data at a higher (more conceptual) level. So, the lowest level that is interesting is at a different position in each scenario (at least when the most common operations are considered).
The need to handle data at different levels is illustrated in these simple examples:
- Example 1: A time-based signal consists of a numeric value that varies over time. This is expressed by integers associated with
Time Values.- We expect the integers to be carried using different serialisations (such as JSON, XML, plain text, binary) on different transports and APIs. The serialisation is known in each carriage (the serialisation is pre-defined, signalled or requested) and it is straightforward to match the serialisation in each case with a suitable decoder.
- These different serialisations used for the integers are not generally of interest to the consumer of the data – the expectation is that the data will be decoded and all operations carried out on the integers.
- Example 2: A video signal is represented using lossy compression (each video frame is a sequence of bytes generated by a video encoder).
- Many operations need to be performed without decoding the video due to the cost of this. Therefore, the encoded data is of interest.
- Also, there are often complexities and subtleties involved in the video coding process which cannot be fully described in practice (such as with technical metadata) but which do have an impact on how the encoded data can be used or the ability of a device to decode it (for example, not all H.264 coded video can be decoded by all H.264 decoders – the standard provides many options and tools). This too makes the encoded data of interest, and it is useful to be able to identify expressions of the video signal that are encoded in a particular way.
Even if Flow is set (for a particular scenario) at a much higher “level” than the Flow Representations, this does not mean that all levels below Flow are unimportant. The levels below that defined by the Flow are not considered interesting when interfacing with the data but they do need to be taken care of somewhere in the wider system; that is, these lower levels are critical to a successful data transfer but are not of interest once that transfer has completed.
It is typically important for technical metadata to be provided about a Flow. This “Flow metadata” will apply to all Flow Representations associated with the Flow. Some Flow Representations might have extra Flow Representation-specific technical metadata associated with them. Additionally, it is important that the level of the Flow is clearly defined – this level might also need to be communicated (using additional metadata properties) between devices.
Guidance on how Flows can be applied to the handling of video and audio can be found in 3.0. Practical Guidance for Media
Notes on the Formal Definition of Flow
General Notes
A Flow can be regarded as a collection of Entrys. This collection of Entrys does not have any fixed / defined ordering, although the Entrys can of course be ordered by Time Value if required. However, in some real scenarios there could be Time Value-based restrictions on the order in which Entrys can be added to an existing Flow (for example, for a live media Flow). Note: This relates to mutability of the model entities which is commented on in 4.0. Appendix - Commentary. Also refer to notes on the relationship between Grains and Entrys.
There is no constraint that requires the technical properties of a Flow to be static. For example, the dimensions of a video or its framerate could change over time – but these properties would change in the same way across all Flow Representations associated with the Flow.
There is also no constraint that all Flow Representations which match the technical properties of a particular Flow must be part of that Flow – this constraint would not be practical to enforce. However, clearly a Flow is most useful if all eligible Flow Representations are members (for example, it is generally better if one Flow is used rather than two if this is permitted by the scenario and by the definition of Flow).
Interpretation of a Flow (for example, “rendering” of the Flow) will depend on the exact data types etc used. For example:
- A
Data Objectmight still contain some time-based information (e.g. information about how long the effect of theData Objectshould persist for when rendered). - The exact interpretation of the
Time Valuewith which aData Objectis associated will depend on how it is being used:- For example, for an SD video frame captured from SDI the
Time Valuemight be the time at which the frame signal begins (Line 1) or it could be the time at which the visible picture for the frame begins (Line 23). - In some scenarios each
Time Valuewill correspond with the time at which theData Objectcame into existence. However, this is certainly not the case for all scenarios. For example, eachTime Valuemight be used to map theData Objectonto a future point in time.
- For example, for an SD video frame captured from SDI the
Notes on Data Objects
Data Objects might depend on each other (for example, inter-frame coded video containing B/P frames; with AAC coded audio multipleData Objects are needed in order to be able to decode the audio). Any such inter-dependencies are not modelled here.- In practical media applications,
Data Objects will typically contain data for a single media sample but could contain multiple media samples. - Regardless of how each
Data Objectis constructed, multipleData Objects might be packaged together inFlow Representations. For example, a real system might choose to always transportData Objects in packages of ten for practical reasons. This packaging must of course be completely reversible so that the originalData Objects and theirTime Values can be accessed. - In practical media applications, a
Data Objectwill typically be mono-essence, but it could be multi-essence. Essence data might exist in both mono-essence and multi-essenceFlows. Mapping between theseFlows might be necessary. For example, it might be useful to state that one multi-essenceFlow(video + audio) contains all of the data required to (reversibly) create two mono-essenceFlows (one video and one audio). - A
Data Objectcan appear in aFlowmore than once.Data Objects are not owned by aFlowand so aData Objectcan be used by more than oneFlow.
←Explanation - Source · Index↑ · Explanation - Flow Representation→

