MATHEMATICS IN THE MIDDLE:
MEASURE, PICTURE, GESTURE, SIGN, AND WORD
J L Lemke
City University of New York
Brooklyn College School of Education
Brooklyn, NY 11210 USA
JLLBC@CUNYVM.CUNY.EDU
ABSTRACT
Argues that mathematics can best be learned and taught as an integral component of a larger sense-making resource system which also includes natural language and visual representation. Formal and social semiotic perspectives are used to show how natural language, mathematics, and visual representations form a single unified system for meaning-making in which mathematics extends the typological resources of natural language to enable it to connect to the more topological meanings made with visual representations. Concludes that the mathematics curriculum and education for mathematics teaching need to give students and teachers much greater insight into the historical contexts and intellectual development of mathematical meanings, as well as the intimate practical connections of mathematics with natural language and visual representation.
KEYWORDS
mathematics education, social semiotics, visual semiotics, history of mathematics, typological meaning, topological meaning, multiple representations, semantics, natural science, science education
Why should a semiotic perspective matter to teachers and students of mathematics? In this chapter I will argue that semiotics helps us understand how mathematics functions as a tool for problem-solving in the real world, and how this function may have played a key role in the historical evolution of mathematics. What matters to teachers and students is the conclusion of this analysis: that mathematics is used and can only be learned and taught as an integral component of a larger sense-making resource system which also includes natural language and visual representation. A semiotic perspective helps us understand how natural language, mathematics, and visual representations form a single unified system for meaning-making.
I want to begin by asking what sort of semiotic beast mathematics might be? that is, what sorts of meaning relationships has it evolved to make sense of? and how does it help us do so? In what sense is mathematics "a language"? or a part of language? and in what sense does it go beyond language in its resources for making meaning? I will try to answer these questions in two ways, historically and semiotically. My conclusion will be that mathematics has evolved historically to help us make a kind of meaning ("topological" meaning) that natural language is not very good at, and that mathematics always evolves and typically functions in close conjunction with natural language and with other semiotic resources, such as visual representations, that are also good at making this important kind of meaning. Along the way I will need to develop and use some semiotic notions to define and contrast topological meaning with more usual "typological" meaning, which is also found in mathematics and visual representations, but which language is particularly good at. I will need to refer at least a little to a social semiotics of action as a unifying framework for understanding the practical integration of verbal language, visual representation, and mathematical symbolics. Finally, I will try to draw out some of the implications of my analysis for teachers and students of mathematics.
Semiotic theories come in many flavors, each useful as a tool for specific sorts of analyses. I am going to make use here of C.S. Peirce's basic notions about signs (see Buchler 1955, Houser & Kloesel 1992), as I interpret him, and link these into a "social semiotics" (Halliday 1978, Hodge and Kress 1988, Lemke 1995), which I will also describe very briefly. Peirce is useful for giving us a very general and abstract set of terms for talking about signs of different sorts; social semiotics will be helpful both for its account of language and for its ways of talking about meaning-making as social practice.
What sort of a semiotic beast is mathematics? There are many possible approaches to this question. We could ask, with Peirce, what sorts of "signs" mathematical symbols are, but by itself that would be a very limited perspective. What we really want to know, especially as educators, is what kinds of meanings we make with mathematics and how people learn and can be more effectively taught to make such meanings. Peirce recognized that signs do not operate in isolation; they are always part of an on-going process of "semiosis" or meaning-making. People solve problems, communicate with one another, or get some practical task done, and in the process mobilize semiotic resources, which are not simply isolated signs, but as Saussure (1959, see also Thibault 1997), the other founder of semiotic theory, emphasized, always part of organized systems of signs, in which each sign has some specific meaning relationship to each other sign in the system. Social semiotics takes a functional approach to meaning-making: it asks how our semiotic resources (e.g. language, diagrams, mathematics, etc.) have evolved to enable us to do things by making particular kinds of meanings.
To answer our first question, it is not enough to ask what a "sign" is, we aIso have to be clear what we mean by "mathematics". In my view, mathematics is a system of related social practices, a system of ways of doing things. Historically, the oldest mathematical texts are lists of problems with their solutions (Cajori 1928, Neugebauer 1975, Peet 1925); they are "how-to" handbooks with no theory. What makes them mathematical handbooks, rather than handbooks of surveying or accounting, is that problems are grouped together by the mathematical methods used to solve them and by their degree of mathematical complexity. Today we are accustomed to finding textbooks, treastises, and whole library shelves labeled "mathematics," but most mathematics in use is not found in such places; it is embedded in writing about physics, engineering, accounting, surveying, etc. What makes it mathematics, wherever we find it, is its characteristic ways of doing things: calculating, symbolizing, deriving, analyzing, etc. It is perfectly possible to have mathematics with no mathematical symbols at all, and there was a long tradition in which mathematicians, particularly in geometry, from Euclid to the early 18th century (the "rhetoricians" as opposed to the "symbolists", Cajori 1928) avoided symbols and wrote their arguments out in words, accompanied by diagrams (which may have had a few symbolic labels in them). Mathematics cannot be identifed by the use of specialized mathematical symbolisms or any unique type of signs.
Mathematics can be identified by the kinds of meanings it makes: meanings about addition, subtraction, multiplication, and division; about numerical difference and equality; about geometrical relationships of parallelism, orthogonality, similarity, congruence, tangency, etc., and many more in mathematical history. It is distinguished by these kinds of meanings, whether they are made by writing natural language, by drawing diagrams, or by formulating symbolic expressions. In most mathematical writing before modern times, symbolic expressions were rare; they were integrated into the running verbal text, and they were clearly meant to be read out in words as part of complete sentences that also included ordinary words. In fact mathematical symbolism originated almost entirely as abbreviations for Greek, Latin, and modern European words and phrases (Cajori 1928). But even in words, or abbreviations for words, mathematical sentences were about kinds of meanings that natural language has trouble articulating. The history of mathematical speaking and writing is a history of the gradual extension of the semantic reach of natural language into new domains of meaning.
In fact it is often difficult to point to this or that sign and say whether it is mathematical or linguistic, mathematical or diagrammatic. Some linguistic signs are also mathematical, and many mathematical signs are also linguistic ones. Some diagrams are mathematical and some mathematical signs are diagrammatic. Indeed we tend to forget that in writing all linguistic signs are also visual, and visual organization is important for reading (e.g. paragraphs, headers and footers, footnotes, marginalia, etc.). Even pure mathematical symbolic expressions are visually grouped into factor clusters, numerators and denominators, fraction expressions, left- and right-sides of equalities, etc. that play a role in how we read and interpret them (Kirshner 1989). It is the meanings, not the forms or even the systems of signs, that determine what is "mathematical".
How do mathematical signs work? in the same ways, by and large, as all signs do. Using one set of terms from Peirce, we can say that there is a (usually visible, but remember that we can "talk mathematics" too) material signifier, the "representamen" (R), that we actually encounter on the page; and then there is the "object" (X), known otherwise as the "signified", that may be (according to one's philosophical inclinations) a real object in the world, a concept, a quantity, an abstraction, or another sign; and finally there is the "interpretant" (I), which is the means by which R gets connected to a particular X. Sometimes in Peirce, "interpretant" means just our interpretation of R, or the on-going process of interpreting R, not just as X, but as some sort of meaning of (R-as-sign-of-X). The important idea is that there has to be some system of interpretance (SI) in the context of which, or by means of which, R's get interpreted as X's. As Peirce says, we have a sign when something (R) stands for something else (X) for somebody in some context (SI). In this respect all signs work in the same way. I wish to argue here that mathematical signs (including a whole equation, or a paragraph of mathematical argumentation in words, or in words with a diagram) are not different in kind from most linguistic signs, though they may represent a special case in some respects, nor from many diagrammatic signs. What is different about mathematical signs is the kinds of meanings they present to us.
Social semiotics is a functional, rather than a purely formal, approach to the analysis of meaning-making. It is less about the nature of signs and more about how people use signs to make meaning. It begins with a form-follows-function assumption about the evolution of sign systems: every system of meaning-related signs and the conventions for using them has evolved to enable us to make certain kinds of meanings.
Language is in many ways the most complex of the known semiotic resource systems. It enables us to make, always and simultaneously in every linguistic sign, three kinds of meanings: (a) Presentational meanings, which are presentations of states-of-affairs, of relations among (abstract) "participants" (or "actants") and processes (doings and happenings) involving such participants; (b) Orientational meanings, which index the stance that the meaning-maker is taking to real and potential audiences and interlocutors, and to the presentational "content" (e.g. indications of speaker evaluations of its desirability, importance, warrantability, usuality, etc.); and (c) Organizational meanings, which define relations of whole-part and part-part on multiple scales of organization in the linguistic "text".
It seems likely that all meaning-making has this tri-functional character, and in any case, for humans who have once caught the language disease, these elements of linguistic meaning-making are never absent from the "system of interpretance" for any sort of sign. You can't really interpret the meaning of a picture or a diagram or an equation in a way that totally suppresses the meaning-making potential of the semantics of your verbal language. A mathematical equation or diagram can relatively autonomously present a state of affairs, especially a relationship between abstract "participants", largely though its own semiotic resources, but for its orientational meanings (is it an assertion or an instruction? presented as important or trivial?) and its organizational meanings (how does it relate to the preceding and following equation or diagram?), it is much more dependent on being embedded in the context of natural language commentary.
In social semiotics every material sign is the product of an action or interaction, or it is a participant ("actant") in the process of that action or interaction. This helps avoid the problems of Platonic Idealism, more severe in mathematics than elsewhere in intellectual culture today (cf. Rotman 1988). Semiotic resource systems in the abstract can be represented as systems of purely formal relations among purely formal signs (words, numbers, shapes), but this is our abstraction as analysts from the reality that signs (actually representamina, R) are always material entities in some real process, and semiosis itself (the process by which an SI reads an R as an X) is itself also equally always a concrete material process in a real social and ecological system where the relationships are not just formal ones (equality, identity, difference) but physical ones (involving the exchange and transformation of matter and energy). Thus mathematics is about what real people do when using mathematical meanings.
All this theoretical framework (for more details see Lemke 1993, 1995) has been sketched out to enable me to make some much more specific arguments about mathematical meanings. I want to argue that they have evolved historically to allow us to integrate two fundamentally different kinds of meaning-making: meaning-by-kind and meaning-by-degree. Mathematical meaning enables us to mix and to move smoothly back and forth between meaning-by-kind, in which natural language specializes, and which I will call categorial or "typological" meaning, and meaning-by-degree, which is more easily presented by means of motor gestures or visual figures -- the meaning of continuous variation or "topological" meaning.
Our basic problems, questions, and concepts are formulated most often in natural language. It is in the semantics of natural language that we do most of our reasoning and informal logic. No mathematical treatise entirely avoids the connective tissue of verbal language to link mathematical symbolic expressions, to comment on the process of development of arguments, and so on. All our applications of mathematics, in the context of which most of our present commonly used mathematics evolved historically -- in the natural sciences, engineering and design, commerce and computing -- require verbal language to link mathematical tools to specific real-world things and events. But the semantics of natural language, its system of possible meanings, is primarily a categorial contrast system, a system of formal "types" or equivalence classes. Every common noun and verb is an abstract type, defined by the availability of alternative categories that stand in contrast to it. How do I describe this movement? as "sauntering" or as "capering"? This thing? as "booklet" or "pamphlet"?
Because our experience in the world does not fall into neatly categorizable types, language has acquired great flexibility in this regard: there is substantial overlap among linguistic categories for naming, and indeed one can probably argue convincingly that a "fuzzy logic" (Zadeh 1965, Klir and Folger 1988, Kosko 1997) applies to these categorizations (i.e. many linguistic categories admit degrees of membership on various criterial dimensions). But at the core of linguistic meaning, the most slowly changing and unconsciously influential part, we find the basic grammatical categories, and these are much more strictly typological. A subject is either singular or plural, there are no degrees of in-between; a verb has one of several distinct tenses, there are no further degrees of intermediate tense. Even for lexical categories (the common nouns, verbs, etc. simply as words), there are only a finite number of words for a "walking-like" activity; you have to pick one of them. In natural language we do not have a standard way to express further, indefinitely more precise, intermediate degrees of meaning in between the existing word categories. (For a situation where people try to do this and fall back on gestures, see Lemke in press.)
As a result, natural language is very bad at giving precise and useful descriptions of natural phenomena in which matters of degree, or quantitative variation, are important. Try to describe the exact shape of a mountain range or a cloud; try to describe the precise difference between two colors, tastes, or smells. Try to describe in words the exact movement through space of a fly, both the shape of the path and the changing rate of movement. Try to describe precisely in words alone the spatial or quantitative relationship between two irregular but continuous curves on a sheet of graph paper. Of these tasks, you may notice that one that is more nearly possible is to note quantitative differences in the ordinates of the curves at corresponding abscissas -- but even then the possibility that some difference might be an irrational number shows the final limitation of natural language, even extended by fractions or decimal numbers.
How do we normally describe motions in space and irregular shapes? not so much with words as with gestures, or we say: let me draw you a picture. Gestures, and more generally actional movements in space and time, are a primary meaning-making resource for imitative, or iconic, representation of meaning-by-degree as opposed to meaning-by-kind. Of course gestures can also mean-by-kind, categorially and typologically, as words do, and function as "symbolic" signs in Peirce's division of signs into icons (by similarity of properties), indices (by cause-effect or co-participation in material events), and symbols (by arbitrary social convention). But a "bow" for example, means not just by being interpretable as "a bow" (vs., say, just bending down) but also by the degree of how low you go. An action counts not just as that action, but also by its timing and pacing: was it done hastily? or tardily? etc. Pictures are in some sense originally traces of gestures (in the sand, on the paper, on the bedroom wall with crayon), but with elaborate typological conventions. They are hardly purely iconic (except, say, for rubbings) either. But in the interpretation of pictorial signs, especially say in painting, matters of degree in color, shape, texture, quality of line (which is really quantity of line!) are very important to salience, pathways of seeing, esthetic effect (which is certainly a kind of meaning), visual organization, etc. (see e.g. Arnheim 1956, Lemke 1998a, O'Toole 1990).
There are many important kinds of meaning that depend as much or more on matters of degree (size, shape, distance, proportion, intensity, loudness, pitch, color, duration, speed, temperature, pressure, voltage, concentration, density, rates of change, etc.) than on matters of kind. These are the meanings in terms of which we represented measured phenomena, and the theory and practice of measurement takes notions of number beyond the counting integers to the threshold of continuous variation. Natural language is poor in semantic resources for giving sufficiently precise descriptions of mensurable phenomena, and the more so as they co-vary with respect to more than one parameter (i.e. require representation in 3- or higher-dimensional spaces). Our solution to this problem historically has been to supplement gestural and pictorial resources for meaning-by-degree with extensions to natural language in the forms we recognize as mathematics.
Figure 1 presents the principal contrasts between typological and what I will call "topological" semiotics. Topological semiotics makes meaning by degree. In the simplest case, both R and X elements of the Peircean sign-relation are capable of continuous or quasi-continuous variation, and this variation matters. We represent continuous variability in something of interest (size, shape, color, temperature, etc.) by continuous variation in something that is convenient as a representamen. As noted in the examples in Figure 2, this can be done iconically, indexically, and symbolically. By comparison, in "typological" semiotics, we represent types or categories by other types or categories. Most of classical semiotics, especially in the Saussurean tradition, has been heavily influenced by the most studied example of a semiotic resource system: natural language. Our well-developed theories of semiosis are mostly about typological semiosis, and topological semiotics, though clearly included in Peirce's view of the sign, has been much less well developed.
Figure 1
I hope the use of the agnate pair "typological" and "topological" will not confuse mathematicians who are used to thinking of topology as a very "qualitative" and not very "quantitative" approach to mathematics. I am using the term here in a very broad sense as connoting the sorts of continuous deformations of manifolds, and the emphasis on issues of nearness and density, associated with real analysis, as opposed to the more discrete, point-like nature of typological categories.
Neither language nor mathematics are purely typological or topological in their semiotic strategies. Language, particularly speech, has its topological resources, in such matters as loudness for emphasis, or the many (but not unlimited) degrees of variation for evaluations (how good, how true). Linguistics has tended to marginalize these, and allowed only the typological elements to be counted as true parts of language. Thus prosody and intonation are very important resources for meaning-making with language, but linguistics tends to recognize them only to the extent that there are identifiable types. Mathematics, on the other hand, as originally a direct extension of the semantics of natural language (see historical discussions below), has some very typological properties. Arithmetic operations, as opposed to numbers, are limited to strict types; there are no intermediate operations between addition and multiplication. Algebraic variables are discrete: values are assigned either to "x" or to "y". Relations such as equality, congruence, similarity, or set-membership are all-or-nothing relations (though fuzzy set theory has introduced meaning-by-degree here recently). In general mathematical expressions are constructed by typological systems of signs, but the values of mathematical expressions can in general vary by degree within the topology of the real numbers.
Figure 3 shows that it is of course in principle possible to have semiotic strategies that mix typological and topological elements not just in the overall meaning-making but even within the sign relation itself. We do, in some very important cases, represent continuously variable objects (X) by categorial signifiers (R). Language itself does this in analyzing the continuous variability of the sound-stream into discrete sound-patterns (phonemes) that ignore most of the acoustic variability in order to produce a simple set of potentially meaningful contrasting types. Thus between the sound of "good" and the sound of "goad" there are many possible sounds, but they will get heard as either one or the other of these two, or as ambiguous and needing to be clarified and resolved as one or the other. And all the quantitative differences in how I say this word in different moods, when sick or well, or how I say it and how you say it, are ignored, so long as the basic contrasts needed to identify distinct words are maintained.
An important example in mathematics is the way in which ratios, which as meanings can be construed in terms of spatial-visual proportions more readily than in words or symbols (see below), are represented as fractions. A fraction (p/q) is semantically an instruction to take (p) multiples of the result of dividing something into (q) parts (think: five one-sevenths), or as a linguistic nominalization of this process, may be interpreted as standing for the numerical result of doing so, regarded as a thing-like quantity (a number). Now fractions, as rational numbers, do not have the full continuum topology of the real numbers, but they certainly can vary significantly by arbitrarily small degree. Nevertheless, in a fraction such quantitative-meanings are represented quasi-linguistically by two numbers, each of which can be regarded as a discrete counting type or category (the integers as cardinals), and by the instruction to consider some relation between them (ratio, or multiple of a part, to be evaluated by the algorithm of division). All of these elements are typological, but the meanings which fractions represent as ratios are topological. If I give you a set of fractions: 13/19, 11/17, 4/6, 9/13; you know that there is no simple way to tell from these typological representations what even the order of sizes of these ratios is, without performing the divisions. But if I presented these same ratios visually, you would have a much better idea of their relationships.
Fractions, and more particularly, complex ratios, represent a major step of mathematics beyond natural language: the first successful effort to represent topological variation by typological means. Mathematics' extension of the meaning resources of natural languages begins with the growth of the system of natural numbers (the counting integers) from the first few, to the first few dozen, to hundreds, thousands, and higher. Historically it was also necessary to learn to combine natural language's use of numeratives as multipliers (five chairs, ten horses) with an extension of the notion of (unit-) fractional parts: from one-half and one-third (which, along with two-thirds and in Old Bablyonian even 5/6, were separate words with their own etymological roots not derived from the words for the corresponding whole numbers) to one-N-th parts (possibly an Egyptian innovation), paving the way for the ancient Greek mathematicians to analyze ratios in detail. It is perhaps not surprising that this is also usually the first topic in elementary mathematics that students find un-intuitive. Visually and manipulatively they have no problem with concrete ratios, but their representation and manipulation by fractions is confusing. Not, I think, because fractions are more abstract (the usual visual diagrams are very abstract compared to photographs), but because they represent something which is not present in the semantics of natural language: a means of representing, quasi-linguistically (i.e. with typological sign resources), what is essentially a topological meaning: significant differences of potentially arbitrarily small degree.
Irrational numbers and the topology of the reals are not topics of elementary mathematics for good reason, but they are certainly also sticking points for more advanced students. These meanings came much, much later in mathematical history, and represent further divergences from the typological logic of natural language. There are a number of other, intermediate topics in which mathematics again represents topological meanings by typological means to the confusion of many students. Basic algebraic notation takes "literals" as symbolic variables which are incorporated into the grammar of mathematical sentences both as if they were words or concepts and as if they were numbers or quantities. This is a great advance of the semantics of mathematics over that of natural language. Pure numbers know no units of measure; if a line has length 4 centimeters, what are the units of its square root? this was meaningless for the Greeks (though not for the Babylonians, who were less geometrically oriented and seemed not to care; they did number theory), and a puzzle until the Renaissance. The root of a number is a number; there are no units of measure. This is not possible in Indo-European, and many other, languages; there is no such meaning as "five", only "five [things]" or "five [units] of a [substance]". To read an algebraic expression both (typologically) as a relationship among distinct variables or a set of arithmetic instructions -- kinds of quasi-linguistic, typological sentences -- and then also (topologically) as a set of potentially continuously variable quantitative relationships is far from an obvious matter for students.
This difficulty is compounded in the case of functions. Functional variation is co-variation; there is always more than one quantity involved, and each has potentially continuous "topological" variability. But our notation for functions is again algebraic, and so quasi-linguistic and typological in its resources. This is in fact another of the great semantic achievements of mathematics: to allow us to represent at least some sorts of continuous covariation in terms of typological operations on typological variables. All the so-called elementary functions, and even many of the special functions that appear in the solutions of partial differential equations, and their algebraic combinations, are represented in algebraic-linguistic this way. Of course, arbitrary functions cannot be so represented; they require infinite sets of Fourier coefficients or the like, reminding us that ultimately typological semiotics is inherently unable to cope with the full range of variability in meaning-by-degree.
Students have a great deal of trouble typically in understanding functional notation and its meaning in terms of quantitative co-variation. They are greatly aided, of course, by that further achievement of historical mathematics: the Cartesian graph. It is not surprising that a visual representation should be able to more clearly present the topological meaning of continuous co-variation, but to use these graphs to understand functions, students must learn to synthesize representations in three different semiotics: verbal language, graphical diagrams, and algebraic expressions. It is only the fact that the last of these shares common features with each of the others (typological strategies with language, topological ones with graphs) that makes this possible at all. It is just this dual nature of mathematical symbolism as a semiotic that makes it both so powerful and so difficult to understand -- unless the connections to more familiar natural language and to visual-gestural semiotics are made clear.
Why has mathematics historically tended to represent quantitative variation in quasi-linguistic ways? There has certainly always been a "geometrical" tradition in mathematics, which has relied far more on visual representations and visual-spatial relationships, even for the representation of quantity. But until modern times there have not been successful purely visual logics (Spencer-Brown 1969, Zellweger 1997). Only natural language provided the means to reason with relative certainty about relations among quantities (or among anything). Natural language logical operators of course required some extension and more careful restriction of sense to avoid logical errors, but this was all done in quasi-linguistic ways (cf. modern symbolic logics). The reason mathematics had to reason about quantitative relationships in quasi-linguistic ways was most fundamentally so that mathematics could embed itself in the logical semantics of natural language reasoning, and historically, because the advance of mathematics was driven to such a large degree by its practical applications. Only natural language could bridge between mathematical meanings and those to be found in the domains of application.
Social semiotics tries to look at meaning-making as an aspect of a whole activity, i.e. some on-going process of inter-related doings in which meaning-making plays a key role. These extended activities tend to take place in complex networks involving many participants, animate and inanimate (cf. Latour 1987, 1996), and they occur on multiple, sometimes intersecting time-scales (cf. Lemke 1998b). People make mathematical meanings as an integral part of activities such as building and operating a power plant, sending an expedition to Mars, creating global climate models, or trying to find more efficient compression algorithms for digital video. Perhaps they are just trying to learn mathematical methods, or theories of physics and chemistry, in the most difficult of all contexts: no other context at all.
Recently, I was asked to analyze a video-tape of one day in the life of a student in an Australian high school (Lemke 1997). In the last year of advanced work in science, this student, John, was taking, in one day, a chemistry and a physics class, with lunch and some mathematics study in between. What was unusual about this videotape was that it showed the student's view of these classrooms; the camera was located next to and a little behind the student, at the end of one row of seats, and it showed his activity and his view out over the rest of the classroom. The point of this research study was to look through the student's eyes, to take the student's role perspective on classroom interaction and learning. In addition to the videotape, I had access to the relevant pages of the textbooks, to the teachers' overheads, handouts and notes, and to the student's own notes from his notebook.
When I began this analysis, I focussed on what different media and channels of information were conveying scientific information to John, and what media and tools John was using to interpret the information. I could see John reading from a textbook, writing in his notebook, looking at a diagram on the chalkboard, using a calculator, talking to a friend in the next seat, listening to the teacher and other classmates ... and even I was astonished at how many different semiotic systems John had to integrate and make use of in every few minutes of time in the classroom.
In every typical few minutes of his work in these science classes, John was listening to the teacher's spoken words, but also looking at diagrams and lists and tables and calculations and equations written both on the chalkboard and displayed on an overhead projector screen. He was listening to his classmate's answers to the teacher's questions, and to her evaluations of those answers (for only the three together make a complete meaning). He was consulting a copy of the textbook and relating what he found there to statements, to questions and answers, to the tables on the board and screen. As he followed the explanation of a method of problem-solving, he was getting ahead of the teacher by using his own calculator, and then comparing his result with hers. He was writing in his notebook, sometimes copying from board or screen, sometimes copying from textbook, sometimes transcribing teacher talk, sometimes formulating his own version of a conclusion of a question-answer-evaluation discussion. His notes integrate all these sources into a reasonably coherent exposition, and they too contain not just the words, but also the tables, the diagrams, the equations and calculations -- John's versions of these, not identical to the teacher's, sometimes more correct than hers, sometimes less.
In the physics class there were also demonstrations of emission spectra and laser light, and the teacher did a visual pantomime of coherent emission and amplification in an imaginary laser crystal, all invisible, but clearly followed by most of the students as they visualized the crystal and the photons from his verbal narration, his gestures, his movements through the (large) imaginary crystal. They relied on memory of other diagrams and pictures that they had seen in the textbook and perhaps in another lesson. And they drew in their notebooks real drawings of invisible, imagined objects created solely by the narration and pantomime of the teacher and never drawn at all. This visual-verbal integration transforms gestures into drawings through the joint work of teachers and students, mediated by verbal language cues and mathematical expressions.
The point of these observations is that the total activity is an integrated whole with respect to meaning-making. Again and again it would not be possible to get a complete and correct meaning just from the verbal language in the activity, nor just from the mathematical expressions written and calculations performed, nor just from the visual diagrams, overheads, and chalkboard cues, nor just from the gestures and motor actions of the participants. It is only by cross-referring and integrating these thematically, by operating with them as if they were all component resources of a single semiotic system, that meanings actually get effectively made and shared in real life.
This is certainly also well-attested in the work of professional scientists (e.g. Ochs 1996, Lynch and Woolgar 1990) and in other classroom research in science education (e.g. Lemke 1987, Roth in press) and in mathematics education (e.g. O'Halloran 1996). It is also very evident in the products of scientific work, such as research articles and advanced treatises. In an informal survey (Lemke 1998a), I examined the most significant of the general English-language scientific journals, Science published by the American Association for the Advancement of Science, and its British counterpart, Nature. I looked across the whole range of science, from molecular biology to field ecology, from particle physics to cosmology. What I saw on the pages of these books and journals was never words alone; on every page there were also visual-graphical representations of many kinds, as well as mathematical equations, and charts and tables.
For example, in the prestigious Physical Review Letters, there were an average of at least one and often two or three graphical figures (tables, charts, graphs, photographs, drawings, maps, and more specialized visual presentations) per page, as well as at least 2-3 and often as many as 6-7 equations per page. In Science, with more experimental and few theoretical articles, there were fewer equations, but on average one table, one data graph, and a total of 4-5 visual graphics of one kind or another per article. In the broader survey as well, there was usually at least one graphic per page and, if equations were used, 1-2 of these per page as well.
The normal mode of technical communication in our society, like the normal mode of technical work, integrates verbal, visual-graphical, and mathematical semiotic resources at every turn. In much of this work, as I have already argued, mathematics is the medium that binds natural language's typological-conceptual meanings to the topological-quantitative meanings we need to make about natural phenomena and which we often also represent visually.
To take one example from this study, which I have analyzed in more detail (Lemke 1998), consider Figure 4, which reproduces a figure from from Berge et al. (1984: 84). The fine horizontal dotted lines visually integrate (organizational function) the abstract graph (i.e. it is not a data graph, but represents only conceptual relationships) at right with the abstract diagram (of a convection apparatus) at left. This is possible here because one of the variables, z, in the graph is a measure of spatial position, and the scale of the graph has been set to correspond exactly to the visual representation of the same spatial distance in the diagram. But otherwise we are still in two semiotically different visual worlds on the two sides of this Figure, a mathematical-geometrical and quantitative-topological world at right vs. an abstracted pictorial-spatial world at the left. There the bold arrow of "g" points down to indicate the pull of gravity and the horizontal dimension is also spatial. At the right, "z" on its vertical axis is only an arbitrary, scalable measure of spatial position, oriented to "real space" only by the visual arrangement here; the horizontal direction now represents the value of the temperature of places with vertical position "z", it is not spatial at all, except metaphorically.
Figure 4
The abstract graph portion of Figure 4 is itself a visual metaphor, showing a relation between position and temperature as if it were a shape in space, something that might be gestured and understood as a relationship among possible positions or motions. It brings to problems of continuous co-variation in all sorts of phenomena our intuitions and perceptual pattern-recognition capacities for spatial-motor phenomena. Those intuitions are supplemented by a mathematization in the form of the abstract graph, itself an extension of an original, textualizable description of numerical data to a pure mathematical abstraction of pattern (i.e. historically, data-graphs have antecedents in data tables; Tufte 1983, Lemke 1998a). Mathematics is more powerful than visualization, even though it is less intuitive, because it can represent patterns that cannot be visualized, and allow them to be compared, manipulated, combined, etc.
The abstract graph in this Figure is not mathematized in the text because it is much too simple. Any competent reader of this advanced textbook could do the algebraic functional mathematization immediately, but does not even need to do so because this graphical pattern is so familiar that it can be directly translated into verbal language, e.g. "The temperature rises linearly with depth in the fluid from To to To+dT." Or any one of many thematically equivalent textualizations, such as: "The temperature is directly proportional to depth in the fluid," or "The temperature decreases linearly with height in the fluid," or "There is a linear vertical temperature gradient in the fluid, with the bottom at higher temperature." Conversely, any competent reader could draw a substantially identical graph, or write the equivalent equation, from any of these verbal statements. There are mathematical equations in the accompanying text, but they deal with more complex matters of the convectional dynamics of a fluid element in a medium of a particular viscosity, density, and coefficient of thermal expansion. The point is that once again we see that competent deployment of mathematical meaning in context is typically interdependent with both verbal language and visual representations.
What lessons for teaching should we draw from this brief analysis of the formal and social semiotics of mathematical meaning?
The great historical achievement of mathematics, in terms of meaning-making, is its development of ways to bridge between the typological meaning strategies of natural language and the topological meanings associated with motor action, visual representation, and the co-variations of mensurable properties of the material world. Too much opportunity for gaining mathematical understanding and intuition, too much practice at learning how to use mathematical meaning in real situations, is lost if mathematics is not taught, particularly at the introductory level, as a co-equal partner with language and visual representation in the analysis of natural and social phenomena. This means that:
(a) we should at all stages make repeatedly explicit for students just how mathematical expressions and mathematized visual representations can be partially but never completely translated into natural language statements and questions, and into one another's forms, and how natural language can be partially translated into mathematical symbolisms, both algebraic and geometric;
(b) we should embed the uses of mathematics in the application contexts that either gave rise to the mathematics historically, or are among the most typical or interesting uses of the mathematics today;
(c) where possible, we should expose students to real, out-of-school settings and practices, in which real people are using mathematics for practical (and theoretical) purposes as part of normal activities in actual institutional contexts (cf. Lemke 1994), including occasionally pure research mathematics; students should observe the practices, simulate them in simplified fashion, gain a sense of their real contexts and complexity, and handle the important artifacts associated with the practices (such as seeing real technical and mathematical research papers, journals and books, operating computers, etc.)
I believe these strategies are always useful, but are especially needed when particular levels and topics are dealt with: those where mathematics historically has extended the semantic meaning potential of natural language in new ways, such as fractions as ratios, algebraic notation, functional relationships, irrational and real numbers, and principles of real analysis. No doubt there are many other such topics that could be identified by a careful semiotic comparison of natural language semantics and the historical development of mathematics and its symbolisms and visual representations in different periods. The history of mathematics, especially as seen in facsimile copies of original mathematical works, with the original notations and diagrams, can be a treasure trove of insights for teachers into why students stumble at certain key conceptual points, what are reasonable alternative approaches to such topics, and even where many less-than-obvious moves in the development of a topic or the strategy of a proof had their original motivation.
The mathematics curriculum and education for mathematics teaching need to give students and teachers much greater insight into the historical contexts and intellectual development of mathematical meanings, as well as the intimate practical connections of mathematics with natural language and visual representation. I hope that I have shown how a semiotic perspective can contribute to reconceptualizing mathematics not simply as a system of signs, but as an integral component of a much larger system of semiotic resources for making mathematical meaning in real, historical contexts.
Arnheim, R. (1956). Art and Visual Perception. London: Faber.
Berge, P., Pomeau, Y., and Vidal, C. (1984). Order Within Chaos: Towards a Deterministic Approach to Turbulence. New York: Wiley.
Buchler, J. (Ed.) (1955). Philosophical Writings of Peirce. New York: Dover.
Cajori, F. (1928). A History of Mathematical Notations. Chicago: Open Court Publishing.
Halliday, Michael A.K. (1978). Language as Social Semiotic. London: Edward Arnold.
Hodge, R. and Kress, G. (1988). Social Semiotics. Ithaca: Cornell University Press.
Houser, N. and Kloesel, C. (1992). The Essential Peirce. Bloomington: Indiana University Press.
Kirshner, David. (1989). The Visual Syntax of Algebra. Journal for Research in Mathematics Education 20 (3), 274-287.
Klir, George J. and Folger, T.A. (1988). Fuzzy Sets, Uncertainty, and Information. New Jersey: Prentice-Hall.
Kosko, Bart. (1997). Fuzzy Engineering. New Jersey: Prentice-Hall.
Latour, Bruno. (1987). Science in Action. Cambridge, MA: Harvard University Press.
Latour, Bruno. (1996). On Interobjectivity. Mind, Culture, and Activity 3(4): 228-245.
Lemke, Jay L. (1987). Strategic Deployment of Speech and Action: A Sociosemiotic Analysis" In J.Evans and J.Deely, Eds., Semiotics 1983: Proceedings of the Semiotic Society of America `Snowbird' Conference (pp.67-79). University Press of America.
Lemke, Jay L. (1993). Discourse, Dynamics, and Social Change. Cultural Dynamics 6(1), 243-275.
Lemke, Jay L. (1994). The Missing Context in Science Education: Science. Paper presented at American Educational Research Association annual meeting, Atlanta, GA (April (1992). Arlington VA: ERIC Documents Service (ED 363 511), (1994.
Lemke, Jay L. (1995). Textual Politics: Discourse and Social Dynamics. London: Taylor and Francis.
Lemke, Jay L. (1997). Multimedia Literacy Demands of the Scientific Curriculum. In Joy Cumming and Claire Wyatt-Smith, (Eds.), Report of the Griffith University (Australia) Project on Literacy Demands of the Post-compulsory Curriculum.
Lemke, Jay L. (1998a). Multiplying Meaning: Visual and Verbal Semiotics in Scientific Text. In James R. Martin and Robert Veel, Eds., Reading Science. London: Routledge.
Lemke, Jay L. (1998b). Across the Scales of Time. Paper presented at ISCRAT Congress of Cultural Research and Activity Theory, Aarhus, Denmark.
Lemke, Jay L. (in press). Typological and Topological Meaning in Diagnostic Discourse. To appear in Discourse Processes (special issue on Discourse in Problem-based Learning, T. Koschmann, Ed.)
Lynch, M. and Woolgar, S. (Eds.) (1990). Representation in Scientific Practice. Cambridge, MA: MIT Press.
Neugebauer, O. (1975). Introduction to the History of Ancient Mathematical Science. New York: Springer Verlag.
Ochs, E., Gonzales, P. and Jacoby, S. (1996). "When I come down I'm in the domain state ...": Grammar and Graphic Representation in the Interpretive Activity of Physicists. In E.Ochs., E. Schegloff, and S. Thompson (Eds.), Interaction and Grammar. New York: Cambridge University Press.
O'Halloran, Kay. (1996). The Discourses of Secondary School Mathematics. (Ph.D. thesis, Murdoch University, Western Australia).
O'Toole, L. Michael. (1990). A Systemic-Functional Semiotics of Art. Semiotica 82: 185-209.
Peet, T.E. (1925). The Rhind Mathematical Papyrus. London: Hodder and Staughton.
Rotman, Brian. (1988). Towards a Semiotics of Mathematics. Semiotica 72 (1-2), 1-35.
Roth, Wolf-Michael and Bowen, G.M. (in press). Complexities of Graphical Representations during Lectures. Cognition and Instruction.
Saussure, Ferdinand de (1959). Course in General Linguistics. New York: McGraw-Hill.
Spencer-Brown, George. (1969). Laws of Form. London: George Allen and Unwin.
Thibault, Paul J. (1997). Re-reading Saussure: The Dynamics of Signs in Social Life. London: Routledge.
Tufte, Edward. (1983). The Visual Display of Quantitative Information. Cheshire, Connecticut: Graphics Press.
Zadeh, Lotfi. (1965). Fuzzy Sets. Information and Control 8, 338-353.
Zellweger, Shea (1997). Untapped Potential in Peirce's Iconic Notation for the Sixteen Binary Connectives." In Houser, N., Roberts, Don D., and Van Evra, J. (Eds.) Studies in the Logic of Charles Sanders Peirce. Bloomington: Indiana University Press, pp. 334-386.