What's new, new media?

The everyday experiences of every individual are multimodal: we see, we hear, we touch, we smell, and we taste. Our experience of the world comes to us through the multiple modes of communication to which each of our senses is attuned.

The definition of mode[]

Mode – a category of signification through meaning is encoded or decoded. It includes spoken language, written language, still image, motion image, music, non-verbal sound, smell, taste, touch, etc.

According to Multimodal Discourse: The Modes and Media of Contemporary Communication by Gunther Kress and Theo Van Leeuwen

Mode is a material resource which is used in recognizably stable ways as a means of articulating discourse.[1]

Mode is a semiotic resource which allows the simultaneous realization of discourses and types of (inter)action.[2]

Modes can be realized in more than one production medium. Narrative is a mode because it allows discourse to be formulated in particular ways (ways which ‘personify’ and ‘dramatize’ discourses, among other things), because it constitutes a particular kind of interaction, and because it can be realized in a range of different media.[3]

The relation between mode and discourse[]

In the context of multimodal environments, spatial and temporal information has to be represented, exchanged, and processed between different components using various modes. Therefore, in brief, modes are available as means of the articulation of discourses. An integrated discourse, as magazine or newspaper, consists of lots of modes, text (mode: written language), photo (mode: still image), diagram, and layout, etc., which are used dependently to articulate the discourse itself.

Language is a mode used broadly in both the academic realm and popular discussion. In the case of magazine or newspaper, the mode of writing (written language) is clearly used to set the tone, to indicate how the other modes and the objects and structures which they articulate are to be read and understood.

Color is a well-articulated mode for the expression of discursive meanings. In the case of the website, The Coke Side of Life, color takes on the functions of a mode and is used to articulate aspects of a discourse on the enterprise culture of Coca-Cola. In fact, blazing with colors, the website is hustle and bustle. Although the site is more than colorful, one color, red, is present throughout the whole site. The navigation set on the left side is purely red; the visual identity, Coca-cola, is red; the icons of ‘happiness factory’, ‘coke football’, ‘coke music’ and ‘the coke side of life’ are red; the flying butterfly, the balloon and the man kicking a football are all red. Color here acts as the carrier of discourses about the image and the visual identity of Coca-Cola. Although the site is multimodal: it consists of flash animation, text, 2D graphics, music and sound effects; color as mode is entirely distinct from other modes here. Therefore, discourse, realized through the mode of color, expresses and articulates how Coca-Cola aimed their target audience, how they hope to be thought about, and of what values attach to the brand.

Screen拷貝

The homepage of 'Coke Side of Life'

The relation between mode and design[]

What mode to use? The most critical part of design is mode selection. design involves issues such as what modes to use for what segment of the curricular content; how to arrange the content, for instance whether to devise a (largely) sequential structure for it; how to arrange the ensemble of modes in the structure; and, as we said, the initial decision as to the rhetorical and epistemological starting point.[4]

When planning a TV show, the producer has to select the modes which are most effective in relation for the purpose of the text, expectations from the audience, and the types of discourse to be articulated. There are similar circumstances regarding this process such as: a teacher planning a lecture or and architect designing a house, etc.

From medium to mode[]

According to Multimodal Discourse: The Modes and Media of Contemporary Communicationby Gunther Kress and Theo Van Leeuwen

Some medium, original feelings, meanings and experiences became modes because they were developed into more abstract, more explicit and more systematic forms of knowledge, like sounds. People can find the sounds of closing car doors are designed to be meaningful and pleasurable, like connoisseur, class, solid, rather than just a mechanical and meaningless by-product of closing the door. Technology plays an increasing role in changing media into modes, and hence in controlling how meanings can be made.[5]

Likewise, technology creates synthetic materials, like plastic, which is a good example to illustrate the notion how medium becomes mode. Plastic was originally developed as a surrogate for various comparatively rare luxury materials. It imitates some qualities of the materials, but lacks the same texture. It also lacks of some of the most essential qualities of physical materiality, but it can imitate any of materials' qualities because of this very ‘meta’-physicality. Plastic can be made to almost anything. Plastic is already a ‘language’, a system to signify material qualities, a material ‘that adapts itself to the syntax of the design in the same way that the words of a language adapt themselves to the syntax of a text’.[6]

Clearly plastic can be used to articulate differences, and hence meanings. These meanings are perceived not visually, but physically, by touching the texture.

External Links[]

  1. Kress and Van Leeuwen, "Multimodal Discourse." page 25. 2001.
  2. Kress and Van Leeuwen, "Multimodal Discourse." page 21. 2001.
  3. Kress and Van Leeuwen, "Multimodal Discourse." page 22. 2001.
  4. Kress and Van Leeuwen, "Multimodal Discourse." page 51. 2001.
  5. Kress and Van Leeuwen, "Multimodal Discourse." page 79. 2001.
  6. Kress and Van Leeuwen, "Multimodal Discourse." page 80. 2001.