Multimedia Genres of Scientific Print Publications

 In another recent study (Lemke 1998a), I examined the semiotic forms found in the standard genres of research articles and advanced treatises of professional scientific publication. In a diverse corpus, across disciplines and publication venues, the clear finding was that there is typically at least one and often more than one graphical display and one mathematical expression per page of running text in typical scientific print genres. There can easily be 3-4 each of graphics displays and mathematical expressions separate from verbal text per page.

In one prestigious journal of the physical sciences, each typical 3-page article integrated four graphical displays and eight set-off mathematical expressions. Some had as many as three graphical displays per page of double-column text, or as many as seven equations per page. In another journal, in the biological sciences, each typical page had two non-tabular visual-graphical representations integrated with the verbal text, and each short (average length 2.4 pages) article typically had six graphics, including at least one table and one quantitative graph.

To appreciate the absolutely central role of these non-verbal textual elements in the genres being characterized, it may help to ponder a few extreme (but hardly unique) cases:

                    In one advanced textbook chapter, a diagram was included in a footnote printed at the bottom of the page.

                    In one 7-page research report, 90% of a page (all but 5 lines of main text at the top) was taken up by a complex diagram and its extensive figure caption.

                    The main experimental results of a 2.5-page report were presented in a set of graphs occupying one-half page and a table occupying three-fourths of another. The main verbal text did not repeat this information but only referred to it and commented on it.

                    In most of the theoretical physics articles, the running verbal text would make no sense without the integrated mathematical equations, which could not in most cases be effectively paraphrased in natural language, even though they can be, and are normally meant to be read out as if part of the verbal text (in terms of semantics, cohesion, and frequently grammar).

 A more detailed analysis in this study showed how absolutely normal and necessary it is to interpret the verbal text in relation to these other semiotic formations, and vice versa. It is not the case that they are redundant, each presenting the complete relevant information in a different medium; rather the nature of the genre presupposes close and constant integration and cross-contextualization among semiotic modalities.

 Why? Why is science not content with verbal linguistic expression? Why have the forms of verbal expression it does use co-evolved so as to mainly make sense only when interpreted in close association with mathematics, diagrams, graphs, tables, maps, etc.? One reasonable hypothesis (for more discussion see Lemke 1998a, in press -d) is that in attempting to describe the quantitative covariation of continuously variable phenomena (shape, temperature, velocity, angle, color, voltage, concentration, mass, etc., etc.) scientific discourse came up against the limitations of language as a semiotic resource. The semantics of natural language specializes in categorization -- in discretely nameable things and processes and in classifying their relationships. Language is not very good at describing complex shapes, shades of color, or degrees of temperature. For these purposes visual and spatial-motor representations are much better suited. To achieve precision however in relating one representation to another, the semantics of natural language also had to be extended historically to describe quantitative variation and relationships. Language remained the main tool of conceptualization and classification, but it was of use only when integrated with mathematical and visual representation. All these representations were and are themselves useful primarily as adjuncts to practical, technical, and experimental activity (see for example Lynch and Woolgar, 1990). The operational conventions of scientific procedures themselves constitute yet another semiotic modality to be integrated with the others.