One thing I learned from this week’s reading is the relationship between semiotics and linguistics. In his Semiotics: The Basics, Daniel Chandler briefly introduced the history behind these two interconnected disciplines, including the impact to semiotics from Saussurean Linguistics and C.S. Peirce’s triadic innovation[i]. Semiotics borrows many concepts and methods from linguistics, and extends to much broader sign systems. This week I’d like to use the works of Xiaogang Zhang, a Chinese surrealist painter best known for his Bloodline series, as examples to illustrate how we understand meanings from paintings.
Bloodline in Pierce’s model
As you can see, Xiaogang’s paintings have very distinct characteristics, reminiscent of Chinese family portraits from decades ago. The faces in the painting are nearly identical, expressionless, and sad, though they have different genders, clothes and hair styles. They look alike one another partly because they are family. Moreover, there are other meanings.
Decades ago, China went through a very hard time, both materially and psychologically. During that time, individualism was opposed while collectivism was favored by government. People regardless of age and gender tended to wear similar drab clothes—mostly dark green reminiscent of military uniform—to avoid other people’s attention. In addition, pop culture was highly restricted, only a very limited number of songs and movies being allowed to be released. I don’t want to talk about politics, but honestly, this period of time was really difficult for average people, including my family. Taking a family portrait was a big event for most families, so everyone would put on their best clothes and the same grave expressions, almost identical. These portraits are real epitomes of that period of time.
What makes Xiaogang’s works so special is the sign system subtly hiding in his usage of color, shape, and shade. Consider the painting below. I will use Peirce’s triadic system (representamen, interpretant, and object) to discuss. For simplicity, interpretant will be left out.
- Sign System 1
- Representamen 1: a zigzag red line connecting the three people
- Object 1: a symbol for family, where many Chinese traditional values reside, including collectivism.
- Sign System 2
- Representamen 2: identical faces
- Object 2: lack of self-identity, excessive collectivism.
- Sign System 3
- Representamen 3: gloom color
- Object 3: depression.
- Sign System 4
- Representamen 4: red scarf
- Object 4: an icon for Young Pioneer
- Sign System 5
- Representamen 5: the boy’s face being retouched to brown
- Object 5: an oppressed desire to be free.
- Sign System 6
- Representamen 6: the boy with a unique brown face wearing a red scarf around his neck, which is the only colorful thing in the whole painting.
- Object 6: the desire of young people to be different, but ending up with the same institution—red scarf is the index of Young Pioneer, in turn, a symbol of institutionalization. Here, the process of interpretant of last two signs (red scarf and retouched brownness) becomes the representamen of this sign, demonstrating the difference between Saussure’s signified and Peirce’s interpretant, which itself is a “sign in the mind of the interpreter[i]”.
- Sign System 7
- Representamen 7: stains on the faces
- Object 7: psychological scars. In China, people always put family portraits under a glass pane on the table. Sometimes, tea would somehow get under the pane and stain the photo. Notice the shape of the stains is sharp like a blade.
Strictly speaking, however, I don’t think the “Parallel Architecture” paradigm can always be extended to sign systems other than language, in which phonological structures, syntactic structures, and conceptual formation structures are interconnected with interface rules[ii].
In paintings, lines and shapes are counterparts of linguistic phonological structures that have no meanings. In Xiaogang’s paintings, the “syntactic structures” are meaningful and interpretable shapes made up of lines, such as the people, the red scarf, and the red line. But the “syntactic structures” of paintings are not universally necessary. Abstractionism has abandoned this intermediate layer between “phonological structures” and “conceptual structures”. For example, Convergence by Jackson Pollock is totally a mess at first glance. No recognizable shapes can be found in the painting. It uses simple “phonological structure” to achieve a conceptual meaning of freedom, let alone Mark Rothko’s works without any specific objects but able to tranquilize a disturbed soul.
Just as philosopher Susanne Langer said that the law governs their articulation “are altogether different from the laws of syntax that govern language[i]”. Nevertheless, I believe understanding the rules behind sign systems will give us new insights into the rules governing our cognition.
Are We Bayesian Learners?
After reading Prof. Irvine’s In-class group exercise on first steps in semiotic analysis, I got some thoughts and questions. Prof. Irvine mentions that we are all pattern recognizers, and we all have the ability to generalize individual patterns to genres. I think this pattern recognizing ability is exactly where our powerful learning abilities reside. We don’t need a lot of examples to learn the common patterns of a genre. For example, we can recognize a watermelon in a supermarket just after several encounters of pictures of watermelon, even if we never saw a real watermelon before and they all look somehow different in size, color and pattern. We know it is a watermelon at the first glance. But it is really hard for computers to learn a new genre of things. Computer scientists have to label hundreds of thousands of pictures as in-put data for an algorithm to learn what a house or a dog looks like. After tons of hours of data-learning, they even can’t distinguish a dog from a cat. This so-called “supervised learning” definitely is not what our brains use to form conceptions.
However, there’s a research on Science last year, in which the researchers use “Bayesian Program Learning” (BPL) to teach a computer program to learn new written languages. The program captured the features of new characters and learned how to write them only through very few examples, achieving “human-level performance while outperforming recent deep learning approaches[iii].” I wonder whether this “Bayesian Learning” method is the secret of our brains in terms of pattern recognizing and learning.
[i] Chandler, Daniel. 2007. Semiotics: The Basics. 2nd ed. Basics (Routledge (Firm)). London ; New York: Routledge.
[ii] Jackendoff, Ray. 2002. Foundations of Language: Brain, Meaning, Grammar, Evolution. OUP Oxford.
[iii]Lake, Brenden M., Ruslan Salakhutdinov, and Joshua B. Tenenbaum. 2015. “Human-Level Concept Learning through Probabilistic Program Induction.” Science 350 (6266): 1332–38. doi:10.1126/science.aab3050.