Jay L. Lemke
City University of New York
“Travels in Hypermodality” playfully echoes Umberto Eco’s title Travels in Hyperreality, which critically examined our fascination with imitation realities that are somehow more appealing than the everyday real world. As new technologies have enabled artifice to build new relationships to the givenness of the everyday world, we have moved from oral narratives to written literature, from drawing and painting to photography, film and video, and from interaction with fixed texts to new modes of participation in the material systems that make texts dynamically responsive to our readings of them. Eco wondered why anyone would prefer a Disneyland village to a real one, as we might wonder why a child prefers a doll to a playmate or a cartoon to a home movie. Some scholars today still wonder why anyone would prefer an illustrated text to purely verbal one, or an interactive hypertext to a printed page.
Scholars are professionally obtuse. Designers on the other hand know very well that simplicity gives us welcome respite from the demanding complexity of everyday life and the less we’re constrained by realism, the more engaged our imaginations become. In many moods, people seek to actively construct meaning with whatever signs or sensory means are at hand. Both verbal text and visual images can be built to be more constraining of the meanings a reader makes or more enabling of the reader as a co-conspirator. Semiotic products can be designed to be more passive objects of contemplation or more active resources for the creation of further meaning. They can invite us to follow, or they can invite us to lead. Good design builds in both functions, in varying combinations, depending on the known purposes of designer and client and the imagined, or fantasized, purposes of a prospective user (cf. Kress & van Leeuwen 2001).
Hypermodality is one way to name the new interactions of word-, image-, and sound- based meanings in hypermedia, i.e. in semiotic artifacts in which signifiers on different scales of syntagmatic organization are linked in complex networks or webs. I will propose here that one useful way to understand the design resources afforded by hypermodality is to consider multiplicative combinations of the presentational, orientational, and organizational resources of each semiotic mode (language, depiction/imagery/graphics, and soundforms).
Hypermodality is more than multimodality in just the way that hypertext is more than plain text. It is not simply that we juxtapose image, text, and sound; we design multiple interconnections among them, both potential and explicit. In the simplest form of hypertext, we might have a web of “pages” (or paragraphs, sentences, or even single words) in which the whole or some part of the text of the page was linked to the whole or some part of another page (or even the same page) in some way other than by the default sequential convention of ordinary reading. The links might be invisible, discoverable by exploring some technology that actuates them. They might be partially explicit (e.g. a unit is marked visually as the source of a linking vector), but the target destination of the link, the nature of the meaning relation between source or anchor and target, and whether the link is reversible might not be explicit. Or all these features could be visible before activating the link. Links make hypertexts multi-sequential (cf. Aarseth 1997). There are many possible trajectories, or traversals, through the web of a hypertext. Meaning on a time- and text-scale long compared to the typical scale of linked anchor units (e.g. paragraphs or pages) becomes a creation of the user/reader that is far less predictable to the designer than in the case of a printed book whose narrative or argument has a single conventional sequence.
It’s important to be clear from the beginning in what sense hypertextuality differs from textuality. The difference is, first of all, one of medium. Typical meaning differences then arise because people exploit the affordances of one medium differently from those of another. A printed text is not itself truly linear or sequential as a medium in the sense that, say, spoken monologue is. Imagine a technology in which written words were presented to us visually one at a time. That would be a linear or singly-sequenced written textual medium. We don’t do it that way. We present written texts almost always, whether in handwriting, print, or on the computer screen as a two-dimensional array, and we exploit that visual medium in many ways. We distinguish headers and sidebars from main text. We read clusters of words at least in a single horizontal line, and to varying degrees also up and down a page. We are visually aware of paragraphing and sectioning.
And our eyes wander. There are many sources of visual salience on a page, and just as our eyes traverse a painting or diagram according to salient features and vectors linking them (Arnheim 1956), we may look away from nearby words on a page to more distant words that are salient because of typeface (italic, bold, small caps), or by their recognizability (e.g. proper names or key words of interest to us), or because they happen to sit in a header or sidebar, or are the initial or final words of a paragraph or section. So do our interests wander, too. We do not always start a printed text at the title page, or the first paragraph of the main text. We may leaf through a book, glancing at this page or that; we may turn to an index, we may follow the page numbers in a table of contents, we may look from a line to a footnote, to a bibliography of references, to an author index, and back to another page. This is a traversal in the print medium, using the technology of the book, both materially (turning pages) and by means of its genre elements (page numbers, index, etc.). But it differs from the hypertext medium not simply because the technology is different – one could use the technology of hypertext to simulate a book in all these respects – but because the web of connectivity of a hypertext activates our expectations that there will be links out from any present text unit and that there will be no single default reading sequence of a main text to return to, or against which we should be reading the content of an excursus. In hypertext there is only excursus – trajectories and loops on different scales without a single unifying narrative or sequential development of a thesis are not available in the print medium.
Conversely, the print medium does afford genuine hypertextual traversals. One could indicate at the link anchors of a printed hypertext instructions for finding the target elsewhere in the printed book, and some experimental literary texts do just this, as do scholarly works with multiple internal cross-references. It’s just rather cumbersome, and slow. Hypertext technology makes for instant sequencing. The sequence generated by activating links (and not activating them) is the text-as-read. Unless there is also some default sequence indicated by the designer, there is no other complete text, there are only the text-units in no particular order. Hypertext makes us aware of the importance of text-scales. We do not necessarily make the same kinds of meanings with a text of hundreds of paragraphs that we make within a single paragraph, just as we do not make the same kinds of meanings with complex sentences that we make with single words or phrases.
Hypermodality is the conflation of multi-modality and hypertextuality. Not only do we have linkages among text-units of various scales, but we have linkages among text-units, visual elements, and sound units. And these go beyond the default conventions of traditional multimodal genres. Even on a single printed page of a magazine, newspaper, or scholarly article in the sciences, we know to connect certain graphical images with certain verbal units (via labels, captions, explanatory text) and vice versa (illustrations of narrative events, figures cited in the text). Organizational devices such as bounding boxes and nearness or juxtaposition combine with semantic content to indicate to us what goes with what across the modal divide between text and image. In hypermedia, there are more kinds of connection than those provided for in print genres.
The hypertextual features of hypermedia also include new forms of intertextuality. Hypertext, as originally envisioned (Nelson 1965/1974, Bush 1945), provides not just for links within a text, but also between texts. In fact the notion of a “whole text” must be re-examined in the hypertext medium. There may be minimal presentational units (called “lexias” by Landow 1997) that are clearly wholes, but on text-scales much larger than this, the only semiotic objects are the traversals, the user-made occasion-specific sequences of lexias. In fact, in many hypertexts (e.g. those written with Storyspace from Eastgate Systems) the designer/author does creates default and optional, but clearly marked trajectories or pathways: sequences of lexias with explicitly indicated links. A user-traversal may follow such a pathway, or it may branch off from it onto another path, or chart its own course. Designer-provided paths in hypertexts however are rarely or never complete; they do not pass through every lexia in the web. They are partial connections, a compromise between author-guided sequencing and reader-selected sequences.
The hypertext medium also permits connections not just from one unit to another within “the same web” (i.e. one recognized by various unifying features, see below) but between units within that web and either another whole web or units within another web. And this feature of the hypertext medium applies in hypermedia to links among text-, image-, and sound- units across different multimedia webs.
The resulting universe of texts is not a seamless web. There are still organizational scales, both those inferable from internal homogeneities and heterogeneities (so that we know when we have crossed from one insitutional website to another, even if the topic and images are very similar) and from external ones (e.g. economic and political boundaries, such as password barriers). The universal web is a fantasy, an updated version of the utopian world in which all national barriers could be passed freely (by the rich; the poor could scarcely walk into the nearest bourgeois neighborhood unaccosted by the police). There are barriers of language, which are not removed by translation (which changes meanings even as it gives partial access to them), barriers of proprietary property, privacy and secrets, barriers of nationalism and politics (foreign URLs and domains blocked by governments), and barriers of culture and values (from simple incomprehensibility to the blocking of access to “immoral” or “irreligious” sites or pages).
If we could follow the traversals of myriad users of the WorldWideWeb, we would see these barriers; most users will wander again and again in familiar territories, and very few will pass elsewhere and then only rarely. It may be useful, as I have proposed elsewhere (Lemke 2001, in press-c), to reserve the term traversal for a trajectory through hypertext (or through life) that crosses the divides between radically culturally separated domains, producing at least the possibility of some hybridity of meaning made across their disjunctions.
A Guided Sequence
I have tried in the previous section to give a rough idea of what I mean by traversals and hypermodality. I have not done more than suggest a number of technical distinctions and terminological niceties that a more systematic analysis would provide in detail. If you want to get a sense of these before proceeding, skip ahead to the Appendix and then return here. There you will find some distinctions among: the medium of hypertext, the technologies for implementing that medium, and the informational content of a particular hypertext web; the sequence of signifiers that constitute a trajectory through a hypertext web and the meanings made with those signifiers that constitute the traversal as such; the various scales or units of signifiers and meaning-making practices both extensionally (size) and in time.
The sequence of topics I have planned, if you read what follows in its default order, will first distinguish three kinds of meaning made by every semiotic act (presentation, orientational, and organizational), and then consider how meanings based on signifiers in different sign systems (language, depiction, music, etc.) can be combined or integrated to produce more specific and new kinds of meanings not otherwise available. I will illustrate these principles with analyses of some website pages and then return to issues of the politics of hypermedia design.