Explanation –Flow
←Explanation - Source · Index↑ · Explanation - Flow Representation→
Please ensure you have read the Summary and Definitions before considering the further detail covered by this page.
Understanding the Flow
Entity
The intention is that a Flow
(identified by a Flow
ID) provides a simple way to identify a particular expression of a Source
. A Flow
is a timed-data interface that might be implemented in different ways. These different implementations of a Flow
are called Flow Representation
s.
In general, it is important not to think of a Flow
as something that is stored in a file or carried in network packets – that is a Flow Representation
and is just one practical realisation of the Flow
. The Flow
ID carried along with, or associated with, this Flow Representation
is of importance here. It is this Flow
ID that identifies the content and the interface (that is, as well as identifying the content it communicates what can be expected when handling the data). It helps to inform the user about how the data can be handled. For example:
- It might be that a decoder is expected to handle either all the
Flow Representation
s of a particularFlow
or only a subset of theFlow Representation
s. - It might be that either: each transport used to carry a particular
Flow
requires a differentFlow Representation
; or that all transports used can carry any of theFlow Representation
s.
In each scenario a decision needs to be made about the level at which to set the Flow
. The most practical level is chosen: this is the lowest level that is considered interesting for the scenario. A Flow
might be set at a low level in which case all associated Flow Representation
s will be very similar. Alternatively, a Flow
can be set at a higher level in which case the associated Flow Representation
s can be much more varied in how they implement the Flow
.
There is a need to work across a very wide range of different scenarios (involving both media and data; that is, all kinds of content Source
s). Across these scenarios there are practical reasons for handling data at different levels. In some scenarios, the sequences of bytes used to represent the data are of interest and this is the level we principally want to work at. In other scenarios, these sequences of bytes are not of interest and we principally want to handle the data at a higher (more conceptual) level. So, the lowest level that is interesting is at a different position in each scenario (at least when the most common operations are considered).
The need to handle data at different levels is illustrated in these simple examples:
- Example 1: A time-based signal consists of a numeric value that varies over time. This is expressed by integers associated with
Time Value
s.- We expect the integers to be carried using different serialisations (such as JSON, XML, plain text, binary) on different transports and APIs. The serialisation is known in each carriage (the serialisation is pre-defined, signalled or requested) and it is straightforward to match the serialisation in each case with a suitable decoder.
- These different serialisations used for the integers are not generally of interest to the consumer of the data – the expectation is that the data will be decoded and all operations carried out on the integers.
- Example 2: A video signal is represented using lossy compression (each video frame is a sequence of bytes generated by a video encoder).
- Many operations need to be performed without decoding the video due to the cost of this. Therefore, the encoded data is of interest.
- Also, there are often complexities and subtleties involved in the video coding process which cannot be fully described in practice (such as with technical metadata) but which do have an impact on how the encoded data can be used or the ability of a device to decode it (for example, not all H.264 coded video can be decoded by all H.264 decoders – the standard provides many options and tools). This too makes the encoded data of interest, and it is useful to be able to identify expressions of the video signal that are encoded in a particular way.
Even if Flow
is set (for a particular scenario) at a much higher “level” than the Flow Representation
s, this does not mean that all levels below Flow
are unimportant. The levels below that defined by the Flow
are not considered interesting when interfacing with the data but they do need to be taken care of somewhere in the wider system; that is, these lower levels are critical to a successful data transfer but are not of interest once that transfer has completed.
It is typically important for technical metadata to be provided about a Flow
. This “Flow
metadata” will apply to all Flow Representation
s associated with the Flow
. Some Flow Representation
s might have extra Flow Representation
-specific technical metadata associated with them. Additionally, it is important that the level of the Flow
is clearly defined – this level might also need to be communicated (using additional metadata properties) between devices.
Guidance on how Flow
s can be applied to the handling of video and audio can be found in 3.0. Practical Guidance for Media
Notes on the Formal Definition of Flow
General Notes
A Flow
can be regarded as a collection of Entry
s. This collection of Entry
s does not have any fixed / defined ordering, although the Entry
s can of course be ordered by Time Value
if required. However, in some real scenarios there could be Time Value
-based restrictions on the order in which Entry
s can be added to an existing Flow
(for example, for a live media Flow
). Note: This relates to mutability of the model entities which is commented on in 4.0. Appendix - Commentary. Also refer to notes on the relationship between Grains and Entry
s.
There is no constraint that requires the technical properties of a Flow
to be static. For example, the dimensions of a video or its framerate could change over time – but these properties would change in the same way across all Flow Representation
s associated with the Flow
.
There is also no constraint that all Flow Representation
s which match the technical properties of a particular Flow
must be part of that Flow
– this constraint would not be practical to enforce. However, clearly a Flow
is most useful if all eligible Flow Representation
s are members (for example, it is generally better if one Flow
is used rather than two if this is permitted by the scenario and by the definition of Flow
).
Interpretation of a Flow
(for example, “rendering” of the Flow
) will depend on the exact data types etc used. For example:
- A
Data Object
might still contain some time-based information (e.g. information about how long the effect of theData Object
should persist for when rendered). - The exact interpretation of the
Time Value
with which aData Object
is associated will depend on how it is being used:- For example, for an SD video frame captured from SDI the
Time Value
might be the time at which the frame signal begins (Line 1) or it could be the time at which the visible picture for the frame begins (Line 23). - In some scenarios each
Time Value
will correspond with the time at which theData Object
came into existence. However, this is certainly not the case for all scenarios. For example, eachTime Value
might be used to map theData Object
onto a future point in time.
- For example, for an SD video frame captured from SDI the
Notes on Data Object
s
Data Object
s might depend on each other (for example, inter-frame coded video containing B/P frames; with AAC coded audio multipleData Object
s are needed in order to be able to decode the audio). Any such inter-dependencies are not modelled here.- In practical media applications,
Data Object
s will typically contain data for a single media sample but could contain multiple media samples. - Regardless of how each
Data Object
is constructed, multipleData Object
s might be packaged together inFlow Representation
s. For example, a real system might choose to always transportData Object
s in packages of ten for practical reasons. This packaging must of course be completely reversible so that the originalData Object
s and theirTime Value
s can be accessed. - In practical media applications, a
Data Object
will typically be mono-essence, but it could be multi-essence. Essence data might exist in both mono-essence and multi-essenceFlow
s. Mapping between theseFlow
s might be necessary. For example, it might be useful to state that one multi-essenceFlow
(video + audio) contains all of the data required to (reversibly) create two mono-essenceFlow
s (one video and one audio). - A
Data Object
can appear in aFlow
more than once.Data Object
s are not owned by aFlow
and so aData Object
can be used by more than oneFlow
.
←Explanation - Source · Index↑ · Explanation - Flow Representation→