Multimodal Presentation of HTML documents for Blind Using Extended Cascading Style Sheets

Philippe Truillet, Nadine Vigouroux
118, Route de Narbonne — 31062 TOULOUSE Cedex — France
{truillet, vigourou} / URL:

1. Introduction

Public World-Wide-Web users are accustomed to visual presentation of HTML (Hyper Text Markup Language) documents on a screen. Alternatives to visual presentation are to be found to consult these documents when they are accessed through an interactive voice server or by the visually impaired persons.
The goal is to provide blind readers with maximum information (both text and material layout wise) against minimum cognitive effort on their part.
CARE [Coutaz 95] was defined as a formal framework to characterize and to assess the relations between several modalities. These relations concern: the Complementarity, Assignment, Redundancy and Equivalence between several modalities (visual, aural, ...) which may be used sequentially, concurrently, independently or combined synergistically. The CARE properties rely on the notions of modality and temporal relationships.
We want to study if the CARE properties can be conveyed within the CSS formalism proposed by the World Wide Web Consortium [W3C 98]. Our work hypothesis is that this formalism is supposed to be flexible and adaptable for information presentation strategies.
Using CSS, the HTML document representation and the document presentation are independent. This approach could provide improved accessibility of document for disabled people without compromising the visual design of the document. In this way, T.V. Raman [W3C 97] proposed an extension of the CSS model for an aural presentation (Aural Cascading Style Sheet) by adding audio, spatial audio and TTS properties [W3C 98b].

2. CARE properties in CSS formalism

CSS defines presentation property classes associated on both HTML tags and media. The @media rule allows to extend the HTML document presentation to other media. For each used media, CSS specifies a possible presentation which can be modified by user on line regarding its capabilities [RNIB 98].
Assignment and Redundancy properties can be easily modeled in the CSS formalism with for example, the @media rule for redundancy property.
Nevertheless, problems still occur in order to model cooperation between modalities. First, the Complementarity property is not modeled. For instance, a CSS does not allow to present typographic information on a Braille display and synthesize simultaneously the information associated on a Text-to-Speech synthesis system. Then, the definition of the media use and its properties have to be refined and structured. We think that it will be more easy to model modalities uses than media uses: we'll extend the CSS formalism to cooperation between modalities in the CARE meaning.

3. The Extension of the CSS formalism

We refined the different media into the modality classes. Our major criterion is to stay independent of the device used. We defined three classes of modalities: visual, aural and tactile (Cf. Figure 1). Each class can be decomposed in several sub-classes of modalities (e.g. "verbal" and "non verbal" for aural, braille and force feedback for tactile).
Figure 1
Figure 1 - Modality Types in our Extended CSS formalism

This proposition is not limited only to these modalities: this model can be extended to other presentation modalities.
Nevertheless, the presentation rules are not specified necessarily even if the presentation properties have been defined. For example, how a Title (H1) has to be presented both to Braille display and aurally? In consequence, we added some operators in CSS like:
  1. synchronization operators between modalities; two operators were added: parallel noted # or sequential noted ;. Note that synchronization and buffer-size harmonization between the Braille display and the TTS system are not easy to achieve,
  2. binary operators for multimodal cooperation ("and" noted && for synergy and "or" noted || for independence)
  3. and another operator for user’s preferences. This operator is defined by the order of presentation rules. For example, @braille.H1 || @TTS.H1 || @Recorded_Speech.H1, the Braille modality is used firstly. By default, the TTS modality will be used and by default, recorded speech will be used. These preferences can be specified for each typographic attribute of the electronic document (title, bold, italic, etc.).
In this formalism, a redundant presentation (in the CARE meaning) of a title for TTS and Braille modality is written #(@TTS.H1 && @braille.H1). Equivalence is written like @TTS.H1 || #(@braille.H1 && @Recorded_Speech.H1).
To present "H1" style, Text-To-Speech synthesis will be used. By default, a cooperation between Braille and recorded speech will be used. This cooperation is supposed to be equivalent to TTS presentation. Finally, the Complementarity property can be written ;( && @TTS.H1.text). The "Title" style is restituted by two modalities: Braille and TTS. Braille modality displays the style (e.g. "Title") and TTS synthesis modality the ASCII text in relation with the style.

4. Conclusion

The implementation of this formalism is in progress within the Smart-Net project [Truillet 97]. After this step, users will have the possibility to personalize the presentation style of documents in regards to its behavior and perceptual model.

5. References

  1. [Coutaz 95] J. Coutaz, L. Nigay, D. Salber, A. Blandford, J. May, R. Young, Four Easy Pieces for Assessing The Usability of Multimodal Interaction: The CARE properties, InterAct’95, Lillehammer (Norway), June 1995, pp. 115-120.
  2. [RNIB 98] Royal National Institute for Blind,
  3. [Truillet 97] Ph. Truillet, B. Oriola, N. Vigouroux, Multimodal Presentation as a Solution to Access a Structured Document, Access Review, Spring 1997, Vol. II, N° 1, pp. 17-22.
  4. [W3C 97] Aural Cascading Style Sheet, W3C Working Draft 30-June-1997,
  5. [W3C 98] World Wide Web Consortium,
  6. [W3C 98b] World Wide Web Consortium, "HTML 4.0 Specification",