Media Accessibility User Requirements

2.1 Described video

Described video contains descriptive narration of key visual elements designed to make visual media accessible to people who are blind or visually impaired. The descriptions include actions, costumes, gestures, scene changes or any other important visual information that someone who cannot see the screen might ordinarily miss. Descriptions are traditionally audio recordings timed and recorded to fit into natural pauses in the program, although they may also briefly obscure the main audio track (see the section on extended descriptions for an alternative approach).

As with captions, descriptions can be open or closed.

• Open descriptions are merged with the program-audio track and cannot be turned off by the viewer.
• Closed descriptions can be turned on and off by the viewer. They can be recorded as a separate track containing descriptions only, timed to play at specific spots in the timeline and played in parallel with the program-audio track.
• Some descriptions can be delivered as a separate audio channel mixed in at the player.
• Other options include a computer-generated ‘text to speech’ track, also known as text video descriptions. This is described in the next subsection.

Described video provides benefits that reach beyond blind or visually impaired viewers; e.g., students grappling with difficult materials or concepts. Descriptions can be used to give supplemental information about what is on screen—the structure of lengthy mathematical equations or the intricacies of a painting, for example.

Described video is available on some television programs and in many movie theaters in the U.S. and other countries. Regulations in the U.S. and Europe are increasingly focusing on description, especially for television, reflecting its priority with citizens who have visual impairments. The technology needed to deliver and render basic video descriptions is in fact relatively straightforward, being an extension of common audio-processing solutions. Playback products must support multi-audio channels required for description, and any product dealing with broadcast TV content must provide adequate support for descriptions. Descriptions can also provide text that can be indexed and searched.

Systems supporting described video where the descriptions are available as independent file or channel resources must:

2.2 Text video description

Described video that uses text for the description source rather than a recorded voice creates specific requirements.

Text video descriptions (TVDs) are delivered to the client as text and rendered locally by assistive technology such as a screen reader or a Braille device. This can have advantages for screen-reader users who want full control of the preferred voice and speaking rate, or other options to control the speech synthesis.

Text video descriptions are provided as text files containing start times for each description cue. Since the duration that a screen reader takes to read out a description cannot be determined during authoring of the cues, it is difficult to ensure they don't obscure the main audio or other description cues. This is likely to be caused by at least three reasons:

• An author of text video descriptions does not have a screen reader. This means s/he cannot check if the description fits within the time frame. Even if s/he has a screen reader, a user's screen reader will be set to a different reading speed and may take longer to read the same sentence.
• Some screen-reader users (e.g., those who are elderly or have learning disabilities) may slow down the speech rate.
• A visually complicated scene (e.g., figures on a blackboard in an online physics class) may require more description time than is available in the program-audio track.

People with low-vision may also benefit from having access to text video descriptions.

Systems supporting text video descriptions must:

1. start time, text per description cue (the duration is determined dynamically, though an end time could provide a cut point)
2. possibly a speech-synthesis markup to improve quality of the description (existing speech synthesis markups include SSML and CSS 3 Speech Module)
3. accompanying metadata providing labeling for speakers, language, etc. and
4. visual style markup (see section on Captioning).

2.3 Extended video descriptions

Video descriptions are usually provided as recorded speech, timed to play in the natural pauses in dialog or narration. In some types of material, however, there is not enough time to present sufficient descriptions. To meet such cases, the concept of extended description was developed. Extended descriptions work by pausing the video and program audio at key moments, playing a longer description than would normally be permitted, and then resuming playback when the description is finished playing. This will naturally extend the timeline of the entire presentation. This procedure has not been possible in broadcast television; however, hard-disk recording and on-demand Internet systems can make this a practical possibility.

Extended video description (EVD) has been reported to have benefits for cognitive disabilities; for example, it might benefit people with Asperger's Syndrome and other Autistic Spectrum Disorders, in that it can make connections between cause and effect, point out what is important to look at, or explain moods that might otherwise be missed.

Systems supporting extended audio descriptions must:

2.4 Clean audio

A relatively recent development in television accessibility is the concept of clean audio, which takes advantage of the increased adoption of multichannel audio. This is primarily aimed at audiences who are hard of hearing, and consists of isolating the audio channel containing the spoken dialog and important non-speech information that can then be amplified or otherwise modified, while other channels containing music or ambient sounds are attenuated.

Using the isolated audio track may make it possible to apply more sophisticated audio processing such as pre-emphasis filters, pitch-shifting, and so on to tailor the audio to the user's needs, since hearing loss is typically frequency-dependent, and the user may have usable hearing in some bands yet none at all in others.

Systems supporting clean audio and multiple audio tracks must:

2.5 Content navigation by content structure

Most people are familiar with fast forward and rewind in media content. However, because they progress through content based only on time, fast forward and rewind are ineffective particularly when the content is being used for purposes other than entertainment. People with disabilities are also particularly disadvantaged if forced to rely solely on time-based fast forward and rewind to study content.

Fortunately, most content is structured, and appropriate markup can expose this structure to forward and rewind controls:

• Books generally have chapters and perhaps subsections within those chapters. They also have structures such as page numbers, side-bars, tables, footnotes, tables of contents, glossaries, etc.
• Short music selections tend to have verses and repeating choruses.
• Larger classical-music works have movements which can be divided into components such as exposition, development and recapitulation, or theme and variations.
• Operas, theatrical plays, and movies have acts and scenes within those acts.
• Television programs generally have clear divisions; e.g., newscasts have individual stories usually wrapped within larger structures called news, weather, or sports.
• A lecturer may first lay out a topic, then consider a series of approaches or illustrative examples, and finally draw a conclusion.

This is, of course, a hierarchical view of content. However, effective navigation of a multi-level hierarchy will require an additional control not typically available using current media players. This mechanism, which we are calling a "granularity-level control," will allow the user to adjust the level of granularity applied to "next" and "previous" controls. This is necessary because next and previous are too cumbersome if accessing every node in a complex hierarchy, but unsatisfactorily broad and coarse if set to only the top level of the hierarchy. Allowing the user to adjust the granularity level that next and previous apply to has proven very effective—hence the adjustable granularity level control.

Two examples of granularity levels

1. In a news broadcast, the most global level (analogous to <h1>) might be the category called "news, weather, and sports." The second level (analogous to <h2>) would identify each individual news (or sports) story. With the granularity control set to level 1, "next" and "previous" would cycle among news, weather, and sports. Set at level 2, it would cycle among individual news (or sports) stories.

2. In a bilingual audiobook-plus-e-text production of Dante Alighieri's "La Divina Commedia," the user would choose whether to listen to the original medieval Italian or its modern-language translation—possibly toggling between them. Meanwhile, both the original and translated texts might appear on screen, with both the original and translated text highlighted, line by line, in sync with the audio narration.

• The most global (<h1>) level would be each individual book— "Inferno," "Purgatorio," and "Paradiso."
• The second (<h2>) level would be each individual canto.
• The third (<h3>) level would be each individual verso.
• The fourth (<h4>) level would be each individual line of poetry.

With granularity set at level 1, "next" and "previous" would cycle among the three books of "La Divina Commedia." Set at level 2, they would cycle among its cantos, at level 3 among its versos, and at level 4 among the individual lines of poetry text.

Navigating ancillary content

There is a kind of structure, particularly in longer media resources, which requires special navigational consideration. While present in the media resource, it does not fit in the natural beginning-to-end progression of the resource. Its consumption tends to interrupt this natural beginning-to-end progression. A familiar example is a footnote or sidebar in a book. One must pause reading the text narrative to read a footnote or sidebar. Yet these structures are important and might require their own alternative media renditions. We have chosen to call such structures "ancillary content structures."

Commercials, news briefs, weather updates, etc., are familiar examples from television programming. While so prevalent that most of us may be inured to it, they do function to interrupt the primary television program. Users will want the ability to navigate past these ancillary structures—or perhaps directly to them.

E-text-plus-audio productions of titles such as "La Divina Commedia," described above, may well include reproductions of famous frescoes or paintings interspersed throughout the text, though these are not properly part of the text/content. Such illustrations must be programmatically discoverable by users. They also need to be described. However, the user needs the option of choosing when to pause for that interrupting description.

Additional note

Media in HTML5 will be used heavily and broadly. These accessibility user requirements will often find broad applicability.

Just as the structures introduced particularly by nonfiction titles make books more usable, media is more usable when its inherent structure is exposed by markup. Markup-based access to structure is critical for persons with disabilities who cannot infer structure from purely presentational queues.

Structural navigation has proven highly effective in various programs of electronic book publication for persons with print disabilities. Nowadays, these programs are based on the ANSI/NISO Z39.86 specifications. Z39.86 structural navigation is also supported by e-publishing industry specifications.

The user can navigate along the timebase using a continuous scale, and by relative time units within rendered audio and animations (including video and animated images) that last three or more seconds at their default playback rate.

The user can navigate by semantic structure within the time-based media, such as by chapters or scenes, if present in the media

Systems supporting content navigation must:

2.6 Captioning

For people who are deaf or hard-of-hearing, captioning is a prime alternative representation of audio. Captions are in the same language as the main audio track and, in contrast to foreign-language subtitles, render a transcription of dialog or narration as well as important non-speech information, such as sound effects, music, and laughter. Historically, captions have been either closed or open. Closed captions have been transmitted as data along with the video but were not visible until the user elected to turn them on, usually by invoking an on-screen control or menu selection. Open captions have always been visible; they had been merged with the video track and could not be turned off.

Ideally, captions should be a verbatim representation of the audio; however, captions are sometimes edited for various reasons— for example, for reading speed or for language level. In general, consumers of captions have expressed that the text should represent exactly what is in the audio track. If edited captions are provided, then they should be clearly marked as such, and the full verbatim version should also be available as an option.

The timing of caption text can coincide with the mouth movement of the speaker (where visible), but this is not strictly necessary. For timing purposes, captions may sometimes precede or extend slightly after the audio they represent. Captioning should also use adequate means to distinguish between speakers as turn-taking occurs during conversation; this has in the past been done by positioning the text near the speaker, by associating different colors to different speakers, or by putting the name and a colon in front of the text line of a speaker.

Captions are useful to a wide array of users in addition to their originally intended audiences. Gyms, bars, and restaurants regularly employ captions as a way for patrons to watch television while in those establishments. People learning to read or learning the language of the country where they live as a second language also benefit from captions: research has shown that captions help reinforce vocabulary and language. Captions can also provide a powerful search capability, allowing users and search engines to search the caption text to locate a specific video or an exact point in a video.

Formats for captions, subtitles or foreign-language subtitles must:

Further, systems that support captions must:

2.7 Enhanced captions/subtitles

Enhanced captions are timed text cues that have been enriched with further information - examples are glossary definitions for acronyms and other initialisms, foreign terms (for example, Latin), jargon or descriptions for other difficult language. They may be age-graded, so that multiple caption tracks are supplied, or the glossary function may be added dynamically through machine lookup.

Glossary information can be added in the normal time allotted for the cue (e.g., as a callout or other overlay), or it might take the form of a hyperlink that, when activated, pauses the main content and allows access to more complete explanatory material.

Such extensions can provide important additional information to the content that will enable or improve the understanding of the main content to users of assistive technology. Enhanced text cues will be particularly useful for those with restricted reading skills, to subtitle users, and to caption users. Users may often come across keywords in text cues that lend themselves to further in-depth information or hyperlinks, such as an e-mail contact or phone number for a person, a unfamiliar term that needs a link to a definition, or an idiom that needs comments to explain it to a foreign-language speaker.

Systems that support enhanced captions must:

2.8 Sign translation

Sign language shares the same concept as captioning: it presents both speech and non-speech information in an alternative format. Note that due to the wide regional variation in signing systems (e.g., American Sign Language vs British Sign Language), sign translation may not be appropriate for content with a global audience unless localized variants can be made available.

Signing can be open, mixed with the video and offered as an entirely alternative stream or closed (using some form of picture-in-picture or alpha-blending technology). It is possible to use quite low bit rates for much of the signing track, but it is important that facial, arm, hand and other body gestures be delivered at sufficient resolution to support legibility. Animated avatars may not currently be sufficient as a substitute for human signers, although research continues in this area and it may become practical at some point in the future.

Acknowledging that not all devices will be capable of handling multiple video streams, this is a SHOULD requirement for browsers where hardware is capable of support. Strong authoring guidance for content creators will mitigate situations where user-agents are unable to support multiple video streams (WCAG) - for example, on mobile devices that cannot support multiple streams, authors should be encouraged to offer two versions of the media stream, including one with signed captions burned into the media.

Selecting from multiple tracks for different sign languages should be achieved in the same fashion that multiple caption/subtitle files are handled.

Systems supporting sign language must:

2.9 Transcripts

While synchronized captions are generally preferable for people with hearing impairments, for some users they are not viable – those who are deaf-blind, for example, or those with cognitive or reading impairments that make it impossible to follow synchronized captions. And even with ordinary captions, it is possible to miss some information as the captions and the video require two separate loci of attention. The full transcript supports different user needs and is not a replacement for captioning. A transcript can either be presented simultaneously with the media material, which can assist slower readers or those who need more time to reference context, but it should also be made available independently of the media.

A full text transcript should include information that would be in both the caption and video description, so that it is a complete representation of the material, as well as containing any interactive options.

Systems supporting transcripts must:

3.2 Granularity level control for structural navigation

As explained in "Content navigation" above, a real-time control mechanism must be provided for adjusting the granularity of the specific structural navigation point next and previous. Users must be able to set the range/scope of next and previous in real time.

3.3 Time-scale modification

While all devices may not support the capability, a standard control API must support the ability to speed up or slow down content presentation without altering audio pitch.

The user can adjust the playback rate of prerecorded time-based media content, such that all of the following are true

3.4 Production practice and resulting requirements

One of the biggest challenges to date has been the lack of a universal system for media access. In response to user requirements various countries and groups have defined systems to provide accessibility, especially captioning for television. However these systems are typically not compatible. In some cases the formats can be inter-converted, but some formats — for example DVD sub-pictures — are image based and are difficult to convert to text.

Caption formats are often geared towards delivery of the media, for example as part of a television broadcast. They are not well suited to the production phases of media creation. Media creators have developed their own internal formats which are more amenable to the editing phase, but to date there has been no common format that allows interchange of this data.

Any media based solution should attempt to reduce as far as possible layers of translation between production and delivery.

In general captioners use a proprietary workstation to prepare caption files; these can often export to various standard broadcast ingest formats, but in general files are not inter-convertible. Most video editing suites are not set up to preserve captioning, and so this has typically to be added after the final edit is decided on; furthermore since this work is often outsourced, the copyright holder may not hold the final editable version of the captions. Thus when programming is later re-purposed, e.g. a shorter edit is made, or a ‘directors cut’ produced, the captioning may have to be redone in its entirety. Similarly, and particularly for news footage, parts of the media may go to web before the final TV edit is made, and thus the captions that are produced for the final TV edit are not available for the web version.

It is important when purchasing or commissioning media, that captioning and described video is taken into account and made equal priority in terms of ownership, rights of use, etc., as the video and audio itself.

This is primarily an authoring requirement. It is understood that a common time-stamp format must be declared in HTML5, so that authoring tools can conform to a required output.

Systems supporting accessibility needs for media must:

3.5 Discovery and activation/deactivation of available alternative content by the user

As described above, individuals need a variety of media (alternative content) in order to perceive and understand the content. The author or some web mechanism provides the alternative content. This alternative content may be part of the original content, embedded within the media container as 'fallback content', or linked from the original content. The user is faced with discovering the availability of alternative content.

Alternative content must be both discoverable by the user, and accessible in device agnostic ways. The development of APIs and user-agent controls should adhere to the following UAAG guidance:

The user agent can facilitate the discovery of alternative content by following these criteria:

3.6 Requirements on making properties available to the accessibility interface

Often forgotten in media systems, especially with the newer forms of packaging such as DVD menus and on-screen program guides, is the fact that the user needs to actually get to the content, control its playback, and turn on any required accessibility options. For user agents supporting accessibility APIs implemented for a platform, any media controls need to be connected to that API.

On self-contained products that do not support assistive technology, any menus in the content need to provide information in alternative formats (e.g., talking menus). Products with a separate remote control, or that are self-contained boxes, should ensure the physical design does not block access, and should make accessibility controls, such as the closed-caption toggle, as prominent as the volume or channel controls.

3.7 Requirements on the use of the viewport

The video viewport plays a particularly important role with respect to alternative-content technologies. Mostly it provides a bounding box for many of the visually represented alternative-content technologies (e.g., captions, hierarchical navigation points, sign language), although some alternative content does not rely on a viewport (e.g., full transcripts, descriptive video).

One key principle to remember when designing player ‘skins’ is that the lower-third of the video may be needed for caption text. Caption consumers rely on being able to make fast eye movements between the captions and the video content. If the captions are in a non-standard place, this may cause viewers to miss information. The use of this area for things such as transport controls, while appealing aesthetically, may lead to accessibility conflicts.

3.8 Requirements on secondary screens and other devices

Multiple secondary user devices must be directly addressable. This functionality is increasingly also known by the new term, "Second Screen," even though there may be more than two screens in any given viewing environment, and even though not all secondary devices are video displays. It must be assumed that many users will have at least one additional display device (such as a tablet), and/or at least one additional audio output device (such as a Bluetooth headset) attached to a primary video display device, an individual computer, or locally addressable on a LAN. It must be possible to configure certain types of media for presentation on specific devices, and these configuration settings must be readily overwritable on a case-by-case basis by users.

Systems supporting secondary devices must: