ISFC 28
Systemic Functional Linguistics Summer Institute
Carleton University, Ottawa
July 2001
Institute Course Workshop
Jay
L. Lemke
City University of New York
http://academic.brooklyn.cuny.edu/education/jlemke
http://web.gc.cuny.edu/urbaneducation
Guide to Multimedia Analysis - NEW
Systemic-functional linguistics provides conceptual tools for specifying the semantic resources available for taking and presenting evaluative stances toward our own and others’ discourses and media productions. An analysis of value orientations in verbal texts such as newspaper editorials can be extended to the mixed visual-verbal medium of the political cartoon or the online website.
The workshop will present basic tools for doing evaluative analysis of texts and images and will consider some of the fundamental issues raised by analyzing multimedia, hypertext, and interactive genres. Application of basic principles will be illustrated from research on editorials, cartoons, magazine ads and features, hypertexts and websites. Evaluative stances toward warrantability, expectedness, normativity, desirability, importance, comprehensibility, and seriousness will be considered.
Participants will be encouraged to bring texts and media from their own areas of interest and to collaboratively explore how the tools and principles of the workshop may help explicate the value systems and heteroglossic contexts which are implicit or explicit in their materials, as well as to articulate difficulties which help identify needed areas for further development of the methods and theory.
Participants will be better able to learn from the Workshop activities and discussion if they familiarize themselves in advance with at least the following readings:
"Resources
for Attitudinal Meaning: Evaluative Orientations in Text Semantics."
Functions of Language 5(1): 33-56, 1998.
"Multiplying Meaning: Visual and Verbal Semiotics in Scientific Text" in J.R. Martin & R. Veel, Eds., Reading Science. London: Routledge. (pp.87-113). 1998.
"Typological
and Topological Meaning in Diagnostic Discourse." Discourse
Processes 27(2), 173-185. 1999.
Note: The items above are linked to manuscript drafts, which may differ from the published versions.
In addition, the following three items on this website provide important background for the Workshop (see also the Notes to Participants, below):
Multimedia Genres for Science
Education and Scientific Literacy -- NASA websites analysis.
Will appear in
M. Schleppegrell & M.C. Colombi, Eds. Developing Advanced Literacy in First and Second
Languages. Mahwah, NJ: Erlbaum.
Visual and Verbal Resources for Evaluative Meaning in Political Cartoons. Unpublished Work; Do not cite or re-distribute without written permission.
Hypertext Semantics -- Work in progress.
Depending on your particular
interests, you may find the following items useful. All items relate directly to
the approach of the Workshop. (If no author is listed, the author is J.L.
Lemke.)
Some items link to manuscript versions of the work.
Mathematics in the Middle: Measure, Picture, Gesture, Sign, and Word. To appear in Anderson, M., Cifarelli, V., Saenz-Ludlow, A., & Vile, A., (Eds) Semiotics Perspectives on Mathematics Education. Erlbaum.
"Metamedia
Literacy: Transforming Meanings And Media"
In D. Reinking, L. Labbo, M. McKenna, & R. Kiefer (Eds.), Handbook
of Literacy and Technology: Transformations in a Post-Typographic World.
Hillsdale, NJ: Erlbaum. pp.283-301. 1998.
"Semantics
and Social Values." WORD 40(1-2): 37-50. 1989.
"Multimedia Demands of the Scientific Curriculum". Linguistics and Education 10(3): 1-25. 2000.
----------------------------------------
Arnheim,
R. (1956). Art and Visual Perception. London: Faber.
Bolter,
J. D. (1991).Writing Space: The computer, hypertext, and the history of
writing. Hillsdale, N.J.:
Lawrence Erlbaum and Associates.
Kress,
G. & van Leeuwen, T. 1996. Reading
images.
London: Routledge.
Landow,
George P. (Ed.). 1994. Hyper Text Theory.
Baltimore: Johns Hopkins U Press.
Landow, George P. 1997. Hypertext 2.0. Baltimore: Johns Hopkins U Press.
Martin,
J.R. (2000). Beyond Exchange: Appraisal systems in English. In S. Hunston and G.
Thompson (Eds.), Evaluation in Text. Oxford: OUP.
O’Toole,
M. (1990). “A systemic-functional semiotics of art.” Semiotica 82:
185-209.
-- . (1994). The Language of Displayed Art. London: Leicester University Press.
Thibault,
P. (2000). 'The multimodal transcription of a television advertisement." In
A. Baldry et al., Multimodality and Multimediality. Campobasso, IT:
Palladino.
Tufte, E. (1983). The Visual Display of Quantitative Information. Cheshire, Connecticut: Graphics Press.
I will expand this bibliography before and after the workshop.
The workshop aims to raise questions about theory and method at the intersection of three topics:
Resources for attitudinal-evaluative meaning
Analysis of multimedia, particularly complex verbal-visual media
Semantics of hypertext
The approach to evaluative meaning will follow my 1998 paper in Functions of Language (Readings). An earlier version of this stance is found in my 1989 WORD paper. The core idea is that propositions and proposals are evaluated and themselves express evaluations along a small number of semantic dimensions. This provides a useful tool for analyzing the implicit and explicit values in text and multimedia. The 1998 paper uses newspaper editorials for examples. The extension to visual-verbal media is found in the unpublished paper on political cartoons (Readings, online).
The analysis of multimedia is based on the extension of Halliday's metafunctional semantics (Ideational, Interpersonal, and Textual meanings) to visual media and to multimedia semiotics in general (Presentational, Orientational, and Organizational meanings). The strong thesis is that every meaningful act or sign must simultaneously make (or is construed in the community where it is meaningful as making) each of these three kinds of meaning. It is then possible to see how the text and the visual media contribute separately and conjointly to meaning of each kind, and how the three kinds of meaning interact (co-contextualize) each other. The best account of this model is given in my 1998 chapter for Reading Science (see Readings). Earlier versions are found in the WORD paper and elsewhere.
Evaluative meaning is an aspect of Orientational meaning: taking a stance toward interlocutors and one's own text. Its verbal component is an aspect of Attitudinal meaning, within the domain of the Interpersonal metafunction. (Cf. J.R. Martin's closely related work on Appraisal, which looks at evaluations of people and things as well as propositions, proposals, and text itself.)
My approach to the semantics of hypertext was presented at the ISFC in Cardiff. The guiding research question is: What kinds of meanings are made at each scale of a traversal or trajectory through a hypertext? What kinds of meanings are made at every scale? what kinds are made along longer trajectories that are not made within a single hypertext unit (page, lexia)? and what semantic resources does hypertext provide or make salient that are unavailable or not salient in ordinary text? This part of the workshop will be closely connected to my Plenary Presentation at ISFC28 on July 26. We will take it up in the workshop mainly when analyzing websites, which are hypertexts, but it will not be a primary emphasis.
More information about all these topics and my work on
them can be found at:
http://academic.brooklyn.cuny.edu/education/jlemke
See particularly the links leading from the homepage for Multimedia Semiotics
and Discourse Analysis, as well as items listed on the Recent Additions page.
The kinds of texts we will consider will include:
Newspaper editorials
Political cartoons
Scientific print publications
NASA websites
We will also take up relevant examples from the texts which participants bring with them. There may also be another workshop, part of the Congress rather than of the Institute, which will consider issues of multimedia analysis, organized by Paul Thibault and myself.
The basic structure of the workshop will be (a) in the morning, presentation of the theoretical tools, issues, and primary examples; (b) in the afternoon, discussion of application of the tools to various texts and examples, including those brought by participants, and of the methodological and theoretical issues raised.
If you wish to contribute texts or examples for discussion in the afternoon session, please (1) send me a copy in advance, with some basic contextual information; (2) contact the conference organizers or me nearer to the time of the workshop to find out how many participants are registered and how many copies of materials to bring. We will probably have at least the ability to display overhead transparencies. If your textual materials include video or webpages, I recommend you prepare one or two static screen-shots to duplicate or project as transparencies. We may be able to arrange viewing of video or display of interactive websites during the lunch break, but in my experience the time and technical risks involved make it unwise to do so during the session itself.
Materials can be sent to me by email or post:
J. L. Lemke
Urban Education Ph.D. Program
CUNY Graduate Center
365 Fifth Avenue
New York, NY 10016
USA
jlemke@gc.cuny.edu