TTML Profiles for Internet Media Subtitles and Captions 1.3

This specification defines two profiles of [[ttml2]]: a text-only profile and an image-only profile. These profiles are intended to be used across subtitle and caption delivery applications worldwide, thereby simplifying interoperability, consistent rendering and conversion to other subtitling and captioning formats.

This specification improves on [[ttml-imsc1.2]] by supporting contemporary practices, while retaining compatibility with [[ttml-imsc1.2]] documents. Relative to [[ttml-imsc1.2]], any addition or deprecation of features are summarized at .

Overview

This Section specifies the Hypothetical Render Model illustrated in .

The purpose of the model is to limit Document Instance complexity. It is not intended as a specification of the processing requirements for implementations. For instance, while the model defines a glyph buffer for the purpose of limiting the number of glyphs displayed at any given point in time, it neither requires the implementation of such a buffer, nor models the sub-pixel character positioning and anti-aliased glyph rendering that can be used to produce text output.

The model operates on successive intermediate synchronic documents obtained from an input Document Instance, and uses a simple double buffering model: while an intermediate synchronic document E_n is being painted into Presentation Buffer P_n (the "front buffer" of the model), the previous intermediate synchronic document E_n-1 is available for display in Presentation Buffer P_n-1 (the "back buffer" of the model).

The model specifies an (hypothetical) time required for completely painting an intermediate synchronic document as a proxy for complexity. Painting includes drawing region backgrounds, rendering and copying glyphs, and decoding and copying images. Complexity is then limited by requiring that painting of intermediate synchronic document E_n completes before the end of intermediate synchronic document E_n-1.

Whenever applicable, constraints are specified relative to Root Container Region dimensions, allowing subtitle sequences to be authored independently of Related Video Object resolution.

To enable scenarios where the same glyphs are used in multiple successive intermediate synchronic documents, e.g. to convey a CEA-608/708-style roll-up (see [[CEA-608]] and [[CEA-708]]), the Glyph Buffers G_n and G_n-1 store rendered glyphs across intermediate synchronic documents, allowing glyphs to be copied into the Presentation Buffer instead of rendered, a more costly operation.

Similarly, Decoded Image Buffers D_n and D_n-1 store decoded images across intermediate synchronic documents, allowing images to be copied into the Presentation Buffer instead of decoded.

General

The Presentation Compositor SHALL render in Presentation Buffer P_n each successive intermediate synchronic document E_n using the following steps in order:

clear the pixels, except for the first intermediate synchronic document E₀ for the which the pixels of P₀ SHALL be assumed to have been cleared;
paint, according to stacking order, all background pixels for each region;
paint all pixels for background colors associated with text or image subtitle content; and
paint the text or image subtitle content.

The Presentation Compositor SHALL start rendering E_n:

at the presentation time of E₀ minus Initial Painting Delay (IPD), if n = 0; or
at the presentation time of E_n-1, if n > 0.

The duration DUR(E_n) for painting an intermediate synchronic document E_n in the Presentation Buffer P_n SHALL be:

DUR(E_n) = S(E_n) / BDraw + DUR_T(E_n) + DUR_I(E_n)

where

S(E_n) is the total normalized drawing area for intermediate synchronic document E_n, as specified in ;
BDraw is the normalized background drawing performance factor;
DUR_T(E_n) is the duration, in seconds, for painting the text subtitle content for intermediate synchronic document E_n, as specified in Section ; and
DUR_I(E_n) is the duration, in seconds, for painting the image subtitle content for intermediate synchronic document E_n, as specified in Section .

The contents of the Presentation Buffer P_n SHALL be transferred instantaneously to Presentation Buffer P_n-1 at the presentation time of intermediate synchronic document E_n, making the latter available for display.

It is possible for the contents of Presentation Buffer P_n-1 to never be displayed. This can happen if Presentation Buffer P_n is copied twice to Presentation Buffer P_n-1 between two consecutive video frame boundaries of the Related Video Object.

It SHALL be an error for the Presentation Compositor to fail to complete painting pixels for E_n before the presentation time of E_n.

It SHALL be an error if the total retrieved font size exceeds 10 MiB.

Unless specified otherwise, the following table SHALL specify values for IPD and BDraw.

Parameter	Initial value
Initial Painting Delay (IPD)	1 s
Normalized background drawing performance factor (BDraw)	12 s^-1

BDraw effectively sets a limit on fillings regions - for example, assuming that the Root Container Region is ultimately rendered at 1920×1080 resolution, a BDraw of 12 s^-1 would correspond to a fill rate of 1920×1080×12/s=23.7×2²⁰pixels s^-1.

IPD effectively sets a limit on the complexity of any given intermediate synchronic document.

Paint Regions

The total normalized drawing area S(E_n) for intermediate synchronic document E_n SHALL be

S(E_n) = CLEAR(E_n) + PAINT(E_n )

where CLEAR(E₀) = 0 and CLEAR(E_{n | n > 0}) = 1, i.e. the Root Container Region in its entirety.

To ensure consistency of the Presentation Buffer, a new intermediate synchronic document requires clearing of the Root Container Region.

PAINT(E_n) SHALL be the normalized area to be painted for all regions that are used in intermediate synchronic document E_n according to:

PAINT(E_n) = ∑_{R_i∈R_p} NSIZE(R_i) ∙ NBG(R_i)

where R_p SHALL be the set of presented regions in the intermediate synchronic document E_n.

NSIZE(R_i) SHALL be given by:

NSIZE(R_i) = (width of R_i ∙ height of R_i ) ÷ (Root Container Region height ∙ Root Container Region width)

For a region R_i in with tts:extent="250px 50px" within a Root Container Region with tts:extent="1920px 1080px", NSIZE(R_i) ≈ 0.00603.

NBG(R_i) SHALL be the total number of tts:backgroundColor attributes associated with the given region R_i in the intermediate synchronic document. A tts:backgroundColor attribute is associated with a region when it is explicitly specified (either as an attribute in the element, or by reference to a declared style) in the following circumstances:

it is specified on the region layout element that defines the region; or
it is specified on a div, p, span or br content element that is to be flowed into the region for presentation in the intermediate synchronic document (see [[!ttml2]] for more details on when a content element is followed into a region); or
it is specified on a set animation element that is to be applied to content elements that are to be flowed into the region for presentation in the intermediate synchronic document (see [[!ttml2]] for more details on when a set animation element is applied to content elements).

Even if a specified tts:backgroundColor is the same as specified on the nearest ancestor content element or animation element, specifying any tts:backgroundColor SHALL require an additional fill operation for all region pixels.

Paint Images

The Presentation Compositor SHALL paint into the Presentation Buffer P_n all visible pixels of presented images of intermediate synchronic document E_n.

For each presented image, the Presentation Compositor SHALL either:

if an identical image is present in Decoded Image Buffer D_n, copy the image from Decoded Image Buffer D_n to the Presentation Buffer P_n using the Image Copier; or
if an identical image is present in Decoded Image Buffer D_n-1, i.e. an identical image was present in intermediate synchronic document E_n-1, copy using the Image Copier the image from Decoded Image Buffer D_n-1 to both the Decoded Image Buffer D_n and the Presentation Buffer P_n; or
otherwise, decode the image using the Image Decoder the image into the Presentation Buffer P_n and Decoded Image Buffer D_n.

Two images SHALL be identical if and only if they reference the same encoded image source.

The duration DUR_I(E_n) for painting images of an intermediate synchronic document E_n in the Presentation Buffer SHALL be as follows:

DUR_I(E_n) = ∑_{I_i ∈ I_c} NRGA(I_i) / ICpy + ∑_{I_j ∈ I_d} NSIZ(I_j) / IDec

where

I_c is the set of images copied when painting intermediate synchronic document E_n;
I_d is the set of images decoded when painting intermediate synchronic document E_n;
IDec is the image decoding rate; and
ICpy is the normalized image copy performance factor.

NRGA(I_i) is the Normalized Image Area of presented image I_i and SHALL be equal to:

NRGA(I_i)= (width of I_i ∙ height of I_i ) ÷ ( Root Container Region height ∙ Root Container Region width )

NSIZ(I_i) SHALL be the number of pixels of presented image I_i.

The contents of the Decoded Image Buffer D_n SHALL be transferred instantaneously to Decoded Image Buffer D_n-1 at the presentation time of intermediate synchronic document E_n.

The total size occupied by images stored in Decoded Image Buffers D_n or D_n-1 SHALL be the sum of their Normalized Image Area.

The size of Decoded Image Buffers D_n or D_n-1 SHALL be the Normalized Decoded Image Buffer Size (NDIBS).

Unless specified otherwise, the following table SHALL specify ICpy, IDec, and NDBIS.

Parameter	Initial value
Normalized image copy performance factor (ICpy)	6
Image Decoding rate (IDec)	1 × 2²⁰ pixels s^-1
Normalized Decoded Image Buffer Size (NDIBS)	0.9885

Paint Text

In the context of this section, a glyph is a tuple consisting of (i) one character and (ii) the computed values of the following style properties:

tts:color
tts:fontFamily
tts:fontSize
tts:fontStyle
tts:fontWeight
tts:textDecoration
tts:textOutline
tts:textShadow

While one-to-one mapping between characters and typographical glyphs is generally the rule in some scripts, e.g. latin script, it is the exception in others. For instance, in arabic script, a character can yield multiple glyphs depending on its position in a word. The Hypothetical Render Model always assumes a one-to-one mapping, but reduces the performance of the glyph buffer for scripts where one-to-one mapping is not the general rule (see GCpy below).

For each glyph associated with a character in a presented region of intermediate synchronic document E_n, the Presentation Compositor SHALL:

if the font associated with the character, as determined using the Font Matching Algorithm, corresponds to an external data resource that was not previously retrieved, retrieve the external data resource and add its size (in bytes) to total retrieved font size; and
if an identical glyph is present in Glyph Buffer G_n, copy the glyph from Glyph Buffer G_n to the Presentation Buffer P_n using the Glyph Copier; or
if an identical glyph is present in Glyph Buffer G_n-1, i.e. an identical glyph was present in intermediate synchronic document E_n-1, copy using the Glyph Copier the glyph from Glyph Buffer G_n-1 to both the Glyph Buffer G_n and the Presentation Buffer P_n; or
otherwise render using the Glyph Renderer the glyph into the Presentation Buffer P_n and Glyph Buffer G_n.

In the case of external data resource retrieved using [[HTTP11]], the size in bytes of the external data resource is the entity-length of the message.

Example of Presentation Compositor Behavior for Text Rendering

The duration DUR_T(E_n) for rendering the text of an intermediate synchronic document E_n in the Presentation Buffer is as follows:

DUR_T(E_n) = ∑_{g_i ∈ Γ_r} NRGA(g_i) / Ren(g_i) + ∑_{g_j ∈ Γ_c} NRGA(g_j) / GCpy

where

Γ_r is the set of glyphs rendered into the Presentation Buffer P_n using the Glyph Renderer in intermediate synchronic document E_n;
Γ_c is the set of glyphs copied to the Presentation Buffer P_n using the Glyph Copier in intermediate synchronic document E_n;
Ren(g_i) is the text rendering performance factor for glyph g_i; and
GCpy is the normalized glyph copy performance factor.

The Normalized Rendered Glyph Area NRGA(g_i) of a glyph g_i SHALL be equal to:

NRGA(g_i) = (fontSize of g_i as percentage of Root Container Region height)²

NRGA(G_i) does not take into account decorations (e.g. underline), effects (e.g. outline) or actual typographical glyph aspect ratio. An implementation can determine an actual buffer size needs based on worst-case glyph size complexity.

The contents of the Glyph Buffer G_n SHALL be copied instantaneously to Glyph Buffer G_n-1 at the presentation time of intermediate synchronic document E_n.

It SHALL be an error for the sum of NRGA(g_i) over all glyphs Glyph Buffer G_n to be larger than the Normalized Glyph Buffer Size (NGBS).

Unless specified otherwise, the following table specifies values of GCpy, Ren and NGBS.

Normalized glyph copy performance factor (GCpy)
Script property (see Standard Annex #24 at [[!UNICODE]]) for the character of g_i	GCpy
latin, greek, cyrillic, hebrew or base	12
any other value	3
Text rendering performance factor Ren(G_i)
Block property (see [[!UNICODE]]) for the character of g_i	Ren(G_i)
CJK Unified Ideograph	0.6
any other value	1.2
Normalized Glyph Buffer Size (NGBS)
1

The choice of font by the presentation processor can increase rendering complexity. For instance, a cursive font can generally result in a given character yielding different typographical glyphs depending on context, even if latin script is used.