Multimedia Semiotic Analysis -- Focal Questions

The following questions are designed to focus the analyst's attention on a variety of meaning types and functions during the analysis of multimodal semiotic artifacts. The initial questions apply to (nearly?) all media; some of the more advanced ones only in more specialized cases (indicated). Not all meaning types are equally salient in all genres. And not all questions provided here are equally relevant for all analytical purposes or research questions. But they are a good starting point.

I imagine the typical semiotic artifact to which these questions are addressed to be a graphical-textual display such as a page from a print publication or a webpage. Typically included are text headers and labels, cohesive text, and visual images and design elements. Many questions also apply, mutatis mutandis, to plain text and its typography and page layout, and to more complex artifacts such as hypertexts, hypermedia, multi-page spreads, etc. Different meanings are made in more extended texts and multimedia than in shorter ones or constituent units. Interactivity adds additional meaning dimensions, as do dynamic media such as audio and video/film or animations. Concrete objects, such as machines and toys, require special consideration on some points, as do immersive environments, whether material (architecture, streetscapes) or virtual.

Organizational Functions and Meanings

Basic and General:

What constitutes the whole text or multimedia artifact under analysis? how do you know what is and what is not a part of it? Does it have a definite boundary or frame in space? in time? Is it part of a larger-scale semiotic entity? Which internal features indicate its role in the larger unit?

What are the largest-scale visual elements or entities within the artifact? the longest-distance and longest-time separated elements that appear to have a definite link or relationship? What is the basis of the link/relationship? what type of meaning relation does it create (e.g. part-part within a sub-maximal whole, similarity or category membership; cf. co-meronymic and co-hyponymic semantic relationships)? What are the longest chains of elements? on the basis of similarity? of visual linkage within a larger structure (spatial-geometrical relationship, graphical linkage)?

What is the most salient visual element? Where is your eye drawn to first? What visual pathways or vectors lead to, and from this element? What other salient elements lies along these pathways? How is salience realized graphically? How are the vectors visually defined in terms of path and directionality?

What are the largest-scale regions? what defines their (fuzzy) boundaries? what are their geometrical relationships to one another? what other meaning-relationships does this global geometry support? For each region, what are its largest-scale constituents? (Iterate the questions until a minimal visual unit is reached.)

What are the visual roles of text in the artifact? How is text associated with other visual elements? Do visual patterns create relationships between distinct text units? Do textual-cohesion relationships create relationships between other distinct visual units? What are the semantics and meaning-functions of these relationships?

Within textual units on each text-scale, how is cohesion achieved? What are the structural units and their semantic relationships to one another at the scales of phrase or group, clause, clause-complex or sentence, rhetorical structure, and genre structure? How does cohesion operate within and across these structural units?

Advanced or Specialized:

What are the typical sequences of events or actions? what are the functional meaning relations of neighboring events? of more distant events along a chain or traversal? What are the options for alternative actions? What differences in types of sequences do different action-choices produce? on shorter sequence-scales? longer?

How do the material affordances of the object constrain organizational relationships of included elements? what are the available dimensions of spatial relationships? of sequential actions/events?

What kinds of meanings are made over longer timescales or space-scales that are not made on shorter ones?

How are organizational relationships created by presentational meanings? (e.g. similar or category-contrasting actors, processes, etc. distributed in different parts of the same text)

How are organizational relationships created by orientational meanings (e.g. similar or contrasting evaluations? source-user stances?)

Orientational Meanings

Presentational Meanings