Call Me What You Want's Bookmarks tagged Downloadable Essays on Visual Language → View Popular
You are here: Diigo Home > Call Me What You Want's Bookmarks
Downloadable Essays on Visual Language
Downloadable Essays by Neil Cohn
Tags: visual language, Neil Cohn, Downloadable Essays on Visual Language, culture, comics, manga on 2008-04-26 and saved by2 people -All Annotations (0) -About
in list: Drawing (Comics)
more fromwww.emaki.net
-
InteractiveAdd Sticky Note
Comics? (February
2005) - Usually, when
people think about language, there is an aspect of face to
face intaction and exchange that springs to mind. This piece
addresses how social interactivity factors into visual language
structure and use- In our own context, the main media of expression matches that of a print culture—pens and paper or computers. Most interactive graphic use in our culture comes in restricted forms, such as drawings by teachers on classroom blackboards, or in the game Pictionary. These instances remain lacking the syntactic qualities found in VL syntax though. While there have been some attempts to bring visual language into everyday use through technology (such as Microsoft's Comic Chat and the experimental Japanese ComicDiary) the results have yet to develop a system where the full allowance of creative expression and grammaticality is paired with real-time interactivity.posted by moshler on 2008-04-26
- Linguist David Wilkins has documented that the Central Australian community of the Arrernte (pronounced "Ar-un-da") create narratives by drawing in the sand, in what could be interpreted as a visual language. These narratives are sequential expressions with highly developed features. Notably, in comparison to our own culture the Arrernte have no writing system, but call their VL "writing" when asked. Their VL seems to have developed without the influence of writing, which perhaps shows in its structure—it does not feature spatially juxtaposed images like in comics. Rather, it is drawn temporally in one location with structurally simple icons. More interestingly though, these narratives are made in real-time exchanges, drawn into the culturally accessible media of sand.posted by moshler on 2008-04-26
Part of what makes the Arrernte system so effective is that their landscape provides them with the resource for expression—the sand allows for easy real-time interactivity, without the need for extraneous tools (such as pens or paper). Without that accessible sand, their VL would be forced to take on different structural features to adapt to a different ecological environment or might not have even developed as language at all. - However, a print culture alone does not limit widespread regularity. Take for example the generalized style that permeates most Japanese comics, with facial features like big eyes, pointy noses and slender chins. Originally, that style stemmed from the "God of Comics" Osamu Tezuka (who himself emulated Walt Disney). First, his single popular "voice" influenced the styles of several others. In time though, it spread to so many people that it no longer could be identifiable as the way a small group of individuals drew, but fossilized as a "manga style" permeating a culture. At this point, new learners (such as the American children now reading manga) become more interested in learning the generalized system, regardless of the individual authors associated to it.posted by moshler on 2008-04-26
In contrast, American comics authors by and large have styles that slightly resemble those of other authors, but not to the degree of allowing for a complete generalized style. Widespread regularity would have difficulty emerging in a culture emphasizing originality of style. For instance, recall the many Jim Lee clone artists from the early and mid-1990s. These people started out like those who originally imitated Tezuka—they all shared common styles derived from an individual influence. However, unlike the Japanese example, most Jim Lee clones that have survived continued to develop their own individual styles, using his as a foundation for broader personal development. As a result, they might be systematic in their own work, but have only with tenuous relations to the rest of the language group. Thus, though the print culture might play a role in the exposure that individual "voices" have on the language users, it alone does not determine how the learners of the visual language might develop.
At the same time, the print culture could also have contributed to development of a more widespread visual language. The advantage of print is its potential to reach a large quantity of people not in the same time - A narrow field of speakers is probably one reason why popular authors have such vast influence on other people's styles. If the "voices" are limited to a select group, learners can thus consciously select specific styles they wish to imitate (as opposed to acquiring the general "style" of the group). That is, of course, if learners decide to imitate at all, given the emphasis Art has in on our culture for innovation and individuality.posted by moshler on 2008-04-26
- the print culture also limits how many speakers the learner comes into contact with. In order for most people to gain a "voice" in the culture, one first needs to be published. Compare this to the extremely democratic use of spoken language, where everybody can partake in the culture's language simply by coming into contact with another person.posted by moshler on 2008-04-26
- Studies have shown that children excel in their image-making ability far more when they are able to see the actual sequential process of drawing, rather than just imitating the style of the final product.posted by moshler on 2008-04-26
- By comparison, this highlights the features of our own modern VL's structure, which mimics the spatial juxtaposition employed in written language to circumvent such real-time interactivity. By having a sequence of panels, VL in comics takes on the spatial characteristics of writing, making it understandable even when preserved temporally. Interestingly, with digital tools, the preservation of a temporally unfurling sequence has become possible, such as in Scott McCloud's The Right Number.posted by moshler on 2008-04-26
Our cultural views on the medium could also have contributed to the lack of the interactivity in our visual language as well. The current perception grounds VL users to the roles of a print culture—a static state emerging from an "artist" and interpreted by a "reader," most often in a separate time and place than that of the creation. This is a far cry from an interactive exchange using visual material merely as a tool for expression rather than the expression itself. Culturally, such a position comes from the perception of the form as some sort of "Art" as opposed to "Language," again resulting in inhibiting the social interactivity of VL.
Interactivity in visual creation has also been shown to aid learning as well. - Historical findings of graphic representation such as cave paintings, drawings by native cultures, and other such visual expressions could be instances of visual language use, though overlooked because our analysis of them sees only the finality of their static preservation (not to mention our own inability to decode their visual grammar). Our perception of them is lacking in both temporal sequence and understanding of them as potentially multimodal communicative acts. Forms such as these might very well be examples of visual languages, though unrecognizable because of our limited categorization of them. Granted, this does not prove that such fully developed structures existed, though pictures that we consider as static works of "art" may have been part of a multimodal interactive process, accompanied by speech and or gestures.posted by moshler on 2008-04-26
- When manual expression uses systematized signs with sequential structures—a grammar—it becomes a "sign language," found most often in deaf communities, such as those that use American Sign Language. Likewise, communication with individual drawings occurs frequently, but when put into a grammatical sequence those drawings take on the qualities of a visual language—as found in the social objects of our culture's "comics." Despite the shift towards the interactive poverty of our print culture, it has not restricted multimodality. In print, comics feature both written and visual language, both of which can take on fully grammatical forms—evidenced in their structured sequences.posted by moshler on 2008-04-26
The sequential aspect to language could in fact play a role in the inability to recognize older artifacts of visual language as well. Because VL is by nature visual, any temporally made sequential production of its form will be preserved statically. Thus, if a person were to "visually converse" in real-time (as we do in spoken language) but drawing only in one physical area, those images would lose their sequential characteristics. The temporal aspect of that succession would be preserved merely as a static representation: only the aftermath of that sequence would be seen. Furthermore, the accompaniment of any spoken words or gesture might also be lost because they are not physically fossilized, negating any semblance of a multimodal act. - The gestural code provides a wealth of information in addition to speech, yet it often goes unnoticed because of our cultural perceptions of the dominance of spoken language (which has resulted in pushing public speakers to stand coldly static, or use only artificial looking hand motions). Co-speech gesture shows that language is naturally a "multimodal" act, with meaning created through the combination of modalities' expressions, a predisposition so strong that people often gesture when on the phone, when their communicative partner can't even see them!posted by moshler on 2008-04-26
- As humans' primary means of communication, language serves as an important connection between people. Given that most language is produced dynamically in real-time, while drawings usually are not, it is no wonder that the prospect of a visual language might seem perplexing in light of language's social aspects. However, such concerns can be answered by considering where our modern VL finds most of its exposure: comics, a recognizable print culture. In print, VL faces the same restrictions as written language, separating the "encoder" from the "decoder" across both space and time.posted by moshler on 2008-04-26
This was not always the case. Some early comic authors actually performed their works. In the early 1900's, many well-known cartoonists had vaudeville acts where they would speedily draw pictures on a sketch-pad, canvas, or blackboard, often while performing voices and commentary. These "chalk talks" or "lightning sketches" emerged from Victorian parlor acts in the late nineteenth century, and quickly produced several celebrity performers both in America and Europe.
In fact, it was the connections to these sorts of real-time performances that inspired Winsor McCay (1869-1934) to create his famous cartoon Gertie the Dinosaur, which opened the door for animated features in movie theatres. While performance is not entirely the same as the interactive use of language, it does provide a middle ground between static print and interactive exchange. Moreover, this mixing of verbal and visual expression in real-time reflects a common practice of storytelling that has pervaded human history across cultural boundaries. - Sometimes, when people hear my proposal that the "comics medium" is literally a visual language (VL), I receive a response of disbelief stating something like, "What, do you expect people to carry around little pads of paper so they can 'talk' in comic form?" Statements like this bring up an important aspect of language that is essential to address in visual language studies: the social and interactive role of language.posted by moshler on 2008-04-25
- Links to Scott McCloud's Online Web Comic "The Right Mind"posted by moshler on 2008-04-24
-
Early
Writings on Visual Language
More writings are included in this 120 page
book which collects, refines, and expands on the earliest of
writings on visual language theory. It covers topics such as
the muddled definition of "comics" and its relation
to visual language, how sequences of images communicate, and
child visual language learning, plus a lot more. Want
one? -
A
Time Frame of Mind • Spring
2002
Written as my undergraduate honors thesis at UC
Berkeley, this work has been published in the Spring 2003 edition
of the Berkeley
Undergraduate Journal. This somewhat esoteric piece offers a
look at the relationship of the Dharma Theory of ancient Buddhist
philosophy to the understanding of visual language temporal mapping.
(42 pages, -
Visual
Syntactic Structures
• Spring 2003
Straightforwardly, communication made by a sequence
of images might be viewed in terms of linear "panel transitions."
However, while intuitive, a linear approach ultimately has many
problems with it.
Part
1 addresses
the problems found in a panel transition model (22 pages,
452K)
Part 2 proposes
an alternative method of analysis in the form of hierarchic
rules, and can be found in my book, Early
Writings on Visual Language -
Art
vs. Language
(July 2004)
- This pieces discusses how the cultural conceptions
of "Art" and "Language" might affect the
structure and usage of visual language in American society. -
Interfaces
and Interactions • Fall 2003
Besides sequential images, "comics"
are often lauded for their union of text and image. This paper
examines the ways in which visual and written language connect
to each other, and how expression of meaning exchanges in such
multimodal communication. Notable topics include how 'speech'
and 'thought balloons' derive meaning, and the integration of
text and image into unified increments. (48 pages, 720K) -
¡Eye
græfIk
Semiosis! • Summer
2005
Written as my Masters Thesis for the University
of Chicago, this piece challenges the common classifcation
between "sound" and "idea" based writing systems.
I argue that all graphic signs lie on a cognitive continuum,
the ramifications of which beg for reconsidering their analysis
as homogenous systems, the conception of their invention, the
nature of their relationship to other visual signs, and the universality
of the category of "writing" in the first place. (61
pages, 1.5MB) Cognitive Map -
A
Force of Change
• Summer 2005
Various visual techniques can create meaning
across sequences of images in interesting and effective ways.
This piece examines metonymy, conceptual metaphors, and blending
across a three-panel pattern used in strips from an advertising
campaign by the Chicago Tribune newspaper. (13 pages,
755K) -
A
Visual Lexicon
• Summer 2005
The most recognized unit in visual language is the
"panel," though meaningful units do exist that are both
smaller and larger than panels. This is similar to spoken languages,
where lexical items can be both above or below the level of the
"word." This paper will address these varying levels of
representation in visual language to lead toward a general understanding
of what it means to have "visual lexical items." *An
expanded version of this essay appears in the Public
Journal of Semiotics (19 pages, 725K) -
Cross-Cultural
Space •
Fall 2005
Comparisons between Japanese manga and American
comics have often been made, yet only a few studies have actually
tried to quantify these differences. This study examines the
"spatial" qualities of panels in a variety of American
and Japanese books (17 pages, 952K) Blog
Thread -
Time
Frames... Or Not
• Fall 2006
The
juxtaposition of two images often produces the illusory sense
of time passing in the visual language used in modern comic books.
While this linear sequence may seem to present a succession of
moments, the understanding of graphic narrative is hardly so simple.
This paper explores various assumptions about sequential images
to show why panels are not moments in time. (16
pages, 744K) Blog
Thread -
Japanese
Visual Language
• Fall 2007
Over the past two decades, manga has exploded in readership
beyond Japan, and its style has captured the interest of young
artists all over. But, what exactly are the properties of this
"style" beyond the surface of big-eyes and "backward"
reading? This paper explores the structural properties of the
visual language underlying the "manga style,"
how it works, and how it differs from the visual languages in
comics from other parts of the world. (21 pages, 1.4MB)
Blog
Thread -
NEW
Navigating Comics
• Spring 2008
Most people
believe that the reading of comic pages moves along the same order
as text: the "z-path" of left-to-rigth and down.
However, what happens when layouts are more complicated than simple
grids? This paper reports the findings of a psychology experiment
which found that readers follow a far more complex process of page
layout navigation than the z-path. (30 pages, 1.6MB) Blog
Thread
Interactive Comics
Tags: Downloadable Essays on Visual Language, Neil Cohn, Visual Language, comics, history on 2008-04-26 -All Annotations (0) -About
in list: Drawing (Comics)
more fromcomixtalk.com
-
In our own context, the main media of expression matches that of a print culture—pens and paper or computers. Most interactive graphic use in our culture comes in restricted forms, such as drawings by teachers on classroom blackboards, or in the game Pictionary. These instances remain lacking the syntactic qualities found in VL syntax though. While there have been some attempts to bring visual language into everyday use through technology (such as Microsoft's Comic Chat and the experimental Japanese ComicDiary) the results have yet to develop a system where the full allowance of creative expression and grammaticality is paired with real-time interactivity.
-
Linguist David Wilkins has documented that the Central Australian community of the Arrernte (pronounced "Ar-un-da") create narratives by drawing in the sand, in what could be interpreted as a visual language. These narratives are sequential expressions with highly developed features. Notably, in comparison to our own culture the Arrernte have no writing system, but call their VL "writing" when asked. Their VL seems to have developed without the influence of writing, which perhaps shows in its structure—it does not feature spatially juxtaposed images like in comics. Rather, it is drawn temporally in one location with structurally simple icons. More interestingly though, these narratives are made in real-time exchanges, drawn into the culturally accessible media of sand.
Part of what makes the Arrernte system so effective is that their landscape provides them with the resource for expression—the sand allows for easy real-time interactivity, without the need for extraneous tools (such as pens or paper). Without that accessible sand, their VL would be forced to take on different structural features to adapt to a different ecological environment or might not have even developed as language at all.
-
At the same time, though print might excel at mass distribution, the issue of "media of transmission" could also play a role in limiting our personal interactivity as well. This harkens back to the opening statement about carrying around pens and paper to draw to each other. Practically speaking: what form must VL take in our culture for people to use it interactively? Comparison to another culture's VL might help illuminate the issue further.
-
Think of this in contrast to ancient methods of image-making. Many instances of sequential narratives in the past (at least those that have been found archeologically) take the form of carvings into wood or stone, or paintings onto walls or everyday items like pottery. Speculatively, while other decomposed artifacts might have not survived en masse for our inspection, if these forms were the primary media of use (and not drawn in real time to an audience), it hardly could have served to initiate any sort of mass fluency. Carving visual expressions would require too much time, effort, and labor intensity to be effective across a wide populace, much less in real-time exchange. And, putting visual representation onto walls (often inaccessible like tombs) or everyday items (such as pottery) seems to imply the usage of the form less for direct communicative means and more for ritual or aesthetics (though the categories are admittedly not mutually exclusive).
-
At the same time, the print culture could also have contributed to development of a more widespread visual language. The advantage of print is its potential to reach a large quantity of people not in the same time and space as the producer. Though it may have had constraining effects on the actual structure of the VL, mass distribution has offered many people the exposure necessary for the language as a whole to flourish. Learners may need to contribute more to the language's structure during their development than in a real-time exchange, but they are still provided with stimuli that they can use to develop their own fluency.
-
However, a print culture alone does not limit widespread regularity. Take for example the generalized style that permeates most Japanese comics, with facial features like big eyes, pointy noses and slender chins. Originally, that style stemmed from the "God of Comics" Osamu Tezuka (who himself emulated Walt Disney). First, his single popular "voice" influenced the styles of several others. In time though, it spread to so many people that it no longer could be identifiable as the way a small group of individuals drew, but fossilized as a "manga style" permeating a culture. At this point, new learners (such as the American children now reading manga) become more interested in learning the generalized system, regardless of the individual authors associated to it.
In contrast, American comics authors by and large have styles that slightly resemble those of other authors, but not to the degree of allowing for a complete generalized style. Widespread regularity would have difficulty emerging in a culture emphasizing originality of style. For instance, recall the many Jim Lee clone artists from the early and mid-1990s. These people started out like those who originally imitated Tezuka—they all shared common styles derived from an individual influence. However, unlike the Japanese example, most Jim Lee clones that have survived continued to develop their own individual styles, using his as a foundation for broader personal development. As a result, they might be systematic in their own work, but have only with tenuous relations to the rest of the language group. Thus, though the print culture might play a role in the exposure that individual "voices" have on the language users, it alone does not determine how the learners of the visual language might develop.
-
A narrow field of speakers is probably one reason why popular authors have such vast influence on other people's styles. If the "voices" are limited to a select group, learners can thus consciously select specific styles they wish to imitate (as opposed to acquiring the general "style" of the group). That is, of course, if learners decide to imitate at all, given the emphasis Art has in on our culture for innovation and individuality.
-
the print culture also limits how many speakers the learner comes into contact with. In order for most people to gain a "voice" in the culture, one first needs to be published. Compare this to the extremely democratic use of spoken language, where everybody can partake in the culture's language simply by coming into contact with another person.
-
Studies have shown that children excel in their image-making ability far more when they are able to see the actual sequential process of drawing, rather than just imitating the style of the final product.
-
By comparison, this highlights the features of our own modern VL's structure, which mimics the spatial juxtaposition employed in written language to circumvent such real-time interactivity. By having a sequence of panels, VL in comics takes on the spatial characteristics of writing, making it understandable even when preserved temporally. Interestingly, with digital tools, the preservation of a temporally unfurling sequence has become possible, such as in Scott McCloud's The Right Number.
Our cultural views on the medium could also have contributed to the lack of the interactivity in our visual language as well. The current perception grounds VL users to the roles of a print culture—a static state emerging from an "artist" and interpreted by a "reader," most often in a separate time and place than that of the creation. This is a far cry from an interactive exchange using visual material merely as a tool for expression rather than the expression itself. Culturally, such a position comes from the perception of the form as some sort of "Art" as opposed to "Language," again resulting in inhibiting the social interactivity of VL.
Interactivity in visual creation has also been shown to aid learning as well.
-
Historical findings of graphic representation such as cave paintings, drawings by native cultures, and other such visual expressions could be instances of visual language use, though overlooked because our analysis of them sees only the finality of their static preservation (not to mention our own inability to decode their visual grammar). Our perception of them is lacking in both temporal sequence and understanding of them as potentially multimodal communicative acts. Forms such as these might very well be examples of visual languages, though unrecognizable because of our limited categorization of them. Granted, this does not prove that such fully developed structures existed, though pictures that we consider as static works of "art" may have been part of a multimodal interactive process, accompanied by speech and or gestures.
-
When manual expression uses systematized signs with sequential structures—a grammar—it becomes a "sign language," found most often in deaf communities, such as those that use American Sign Language. Likewise, communication with individual drawings occurs frequently, but when put into a grammatical sequence those drawings take on the qualities of a visual language—as found in the social objects of our culture's "comics." Despite the shift towards the interactive poverty of our print culture, it has not restricted multimodality. In print, comics feature both written and visual language, both of which can take on fully grammatical forms—evidenced in their structured sequences.
The sequential aspect to language could in fact play a role in the inability to recognize older artifacts of visual language as well. Because VL is by nature visual, any temporally made sequential production of its form will be preserved statically. Thus, if a person were to "visually converse" in real-time (as we do in spoken language) but drawing only in one physical area, those images would lose their sequential characteristics. The temporal aspect of that succession would be preserved merely as a static representation: only the aftermath of that sequence would be seen. Furthermore, the accompaniment of any spoken words or gesture might also be lost because they are not physically fossilized, negating any semblance of a multimodal act.
-
The gestural code provides a wealth of information in addition to speech, yet it often goes unnoticed because of our cultural perceptions of the dominance of spoken language (which has resulted in pushing public speakers to stand coldly static, or use only artificial looking hand motions). Co-speech gesture shows that language is naturally a "multimodal" act, with meaning created through the combination of modalities' expressions, a predisposition so strong that people often gesture when on the phone, when their communicative partner can't even see them!
-
As humans' primary means of communication, language serves as an important connection between people. Given that most language is produced dynamically in real-time, while drawings usually are not, it is no wonder that the prospect of a visual language might seem perplexing in light of language's social aspects. However, such concerns can be answered by considering where our modern VL finds most of its exposure: comics, a recognizable print culture. In print, VL faces the same restrictions as written language, separating the "encoder" from the "decoder" across both space and time.
This was not always the case. Some early comic authors actually performed their works. In the early 1900's, many well-known cartoonists had vaudeville acts where they would speedily draw pictures on a sketch-pad, canvas, or blackboard, often while performing voices and commentary. These "chalk talks" or "lightning sketches" emerged from Victorian parlor acts in the late nineteenth century, and quickly produced several celebrity performers both in America and Europe.
In fact, it was the connections to these sorts of real-time performances that inspired Winsor McCay (1869-1934) to create his famous cartoon Gertie the Dinosaur, which opened the door for animated features in movie theatres. While performance is not entirely the same as the interactive use of language, it does provide a middle ground between static print and interactive exchange. Moreover, this mixing of verbal and visual expression in real-time reflects a common practice of storytelling that has pervaded human history across cultural boundaries.
-
Sometimes, when people hear my proposal that the "comics medium" is literally a visual language (VL), I receive a response of disbelief stating something like, "What, do you expect people to carry around little pads of paper so they can 'talk' in comic form?" Statements like this bring up an important aspect of language that is essential to address in visual language studies: the social and interactive role of language.
The Right Number
Visual Flash-based comic by Scott McCloud. Notable for the way it presented the plot as well as both the artwork and the story of the plot though as a whole it does come off like a cliche.
Tags: Scott McCloud, comics, flash, foundation, water, Downloadable Essays on Visual Language on 2008-04-24 and saved by11 people -All Annotations (0) -About
in list: Drawing (Comics)
more fromwww.scottmccloud.com
Notation: * = Private bookmark and comment|… = Clipping [?] | … = Public highlight [?]


