For some time now I have been trying to develop a discourse about literate social practices that is informed by a theoretical perspective often referred to as social semiotics (e.g. Lemke 1989, 1995; Halliday 1978, Hodge & Kress 1988). The basic principle of social semiotic analysis is that meanings are made by selective contextualization: each entity which we take to be a sign we make meaningful by considering its syntagmatic, paradigmatic, situational and intertextual contexts, both actual and potential (Lemke 1985). We do this in relatively automated ways that represent the typical and repeated meaning-making practices of the communities to which we belong, and in ways that are specific to cultures and subcultures, topics, participants, and settings. Making meaning is a process that takes place in material systems that include but extend beyond ourselves as biological organisms; these systems include the material texts, tool, and artifacts of a community, as well as other persons and nonhuman participants, and extend over multiple spatial and temporal scales from the local setting and immediate moment to the whole history of a widespread community. The meanings we make on any occasion are both uniquely emergent and culturally typical; they depend both on local contexts and on other meanings made in other times and places.
In this view the broadest sense of ‘literacy’ is identical to meaning-making or semiosis in general. A narrower definition of literacy may be constructed by focusing on meaning-making in which complex configurations of artifacts or natural structures play a critical role, as ‘text’, in the meanings we make on some occasion. In this sense a geologist may ‘read’ the Grand Canyon as a text, just as s/he may ‘read’ a photo-montage of the canyon, a geological map of its strata and topography, or a verbal account of its stratigraphy. In fact the meanings made by such a geologist, and the typical genres of diagrammatic and verbal texts in geology, presume that verbal accounts are typically made sense of in relation to relevant maps, diagrams, photographic records, and personal field experience. Each of these is in turn to be interpreted in relation to all the others, including a large intertextual web of verbal and mutlimedia texts. Actual scientific texts are almost never in fact purely verbal (Lemke 1998a, Roth et al. 1999).
The narrowest definition of literacy would focus solely on verbal literacy with written media, but this definition is intellectually untenable today. Efforts to say what distinguishes ‘writing’ (e.g. Harris 1995 , Lemke 1997), unless they arbitrarily restrict themselves to signs interpreted only in terms of the semantics of the linguistic system, find that the boundaries between written text and mathematical or chemical symbolism are hard to declare or justify; indeed, both of these latter instances arguably descend from language historically and remain partially interpreted linguistically. From text to table, table to chart, chart to graph, graph to diagram, diagram to picture there are historical continuities and contemporary unities in practice (Lemke 1998a). Developmentally, speech and gesture derive from common motor routines, pictures descend from the lasting traces of gestures, and writing is a differentiated form of expressive speech-accompanying gesture, not initially separated from depiction. Semiotically, we never in fact make meaning with only the resources of one semiotic system: words conjure images, images are verbally mediated, writing is a visual form, algebra shares much of the syntax and semantics of natural language, geometric diagrams are interpreted verbally and pictorially, even radio voices speak to us of individuality, accent, emotional state and physical health through vocal signs not organized by the linguistic code. All semiotics is multimedia semiotics; all meaning is made in the integration of resources from only analytically separable semiotic resource systems.
In the perspective of social semiotics, meaning making is social, and material, and semiotic, and so therefore is literacy. Because it is material, no actual phenomenon of literate practice can ever be exhaustively analyzed by specifying the formal relations by which it instances some one semiotic, or even all known semiotics (the consequence here has long been recognized by phenomenology, the cause is a bit more mysterious; Lemke, in press -b). Because it is semiotic, our accounts of what and how it means must consider the state of affairs it presents, how it orients itself in the system of intertextual alternatives, and what unifies it as a ‘text’ (Halliday’s 1978, 1994 ‘ideational, interpersonal, and textual’ linguistic meta-functions, and see generalizations in Lemke 1998a, Kress & van Leeuwen 1996; O’Toole 1990, 1994). Because it is social, we must explicate its social functions, both local and immediate and larger-scale and longer-term, to understand its meanings in the widest sense.