Apps that we use in our smart phones are integral to the functioning of smartphones, and are the major elements for the interface design and functionality. While smartphone users intuitively use Apps, there are multiple symbol-making processes that take place as one is interacting with a smartphone application. To start with: clicking on a single button on a smart phone accesses apps. This process is made intuitive through a text with a name of the app, and a graphic design that further communicates the functionality of an app. The apps are tiled in the smartphone’s interface. This tile design suggests a modular structure: Apps are modules for accessing software.

In terms of functionality, Apps provide access to software that was designed by employment of human cognition. Therefore, “Apps” are interfaces that allow access. Framing Apps as interfaces allows us to further discern the software technology behind them. The interface stands for access to the app. Another dimension of apps is that they are embedded in the smartphones regardless of Internet access: They are mobile technologies, and are characterized by their integration to multiple devices.

Considering the softwares of Applications, defining the symbolic interaction as the interaction between the mobile phone’s screen to a person’s cognition will be limited, and won’t cover all aspects of the symbolic interaction. Pierce mentions an “object” (Chandler, 2007) when he describes the semiotic process, and employs a directional understanding to explore how meaning is generated. Similarly, Saussure mentions a “signified.” (Saussure, 2011) Saussure’s idea of the signified is more explanatory in terms of referring to the mental image and abstraction that takes place during symbol-making, yet these two models of sign making are not comprehensive enough to cover all aspects. Apps are interfaces that enable access to employed human cognition, and extends human cognition by providing cognitive tools for functioning.

De Saussure, F. (2011). Course in general linguistics. Columbia University Press.

Chandler, D. (2007). Semiotics: the basics. Routledge.

This week I re-read the Jackendoff’s parallel architecture piece. I have to admit that there are still parts of it that I cannot understand, but reading it one more time resolved questions that I had from last semester. Jackendoff and his students work outside of linguistic field as well, such as comics and music. I found the research on comic is very interesting. It is also very rewarding by reading Jackendoff’s original research on parallel architecture on linguistics and the research based on this original one.

In Jackendoff’s article, he proposes one basic and important point about human being’s language competence, that is the f-knowledge. He thinks that f-knowledge of language requires two components: (1) finite list of structural elements, such as lexicon; (2) finite set of combinatorial principles, such as grammars. As oppose to generative grammar which consider phonology and semantics as secondary components of language under syntax, Jackendoff argues that meaning is not secondary and the generatively of language doesn’t completely come from generatively of grammars and syntax. I found a very good quote from his article where he explains what is “meaning”: “It is the locus for the understanding of linguistic utterances in context, incorporating pragmatic considerations and ‘world knowledge’… it is the cognitive structure in terms of which reasoning and planning take place. That is, the hypothesized level of conceptual structure is intended as a theoretical counterpart of what common sense calls ‘meaning’.”

Here I want to talk about one more interesting research his student Neil Cohn did on sequential image comprehension. I think it is a very good illustration about how semantics and syntax work independently but correlates to each other. In his article The Limits of Time and Transitions: Challenges to Theories of Sequential Image Comprehension, Cohn talks about when juxtaposing two images, it often produces the illusory sense of time passing, as found in the visual language used in modern comic books. He found out that any linear panel-to-panel analysis or loosely defined principles of connection between sequential images are inadequate to explain people’s understanding, “Sequential image comprehension must be thought of the union of conceptual information that is grouped via unconscious hierarchic structures in the mind.”

It’s still hard to understand entirely his research but I got several interesting take away from his research. One of them is to see how he breaks down the elements of comics:

(1) moment-to-moment— between small increments of time

(2) action-to-action— between full ranges of actions

(3) subject-to subject— between characters or objects in a scene

(4) aspect-to-aspect— between aspects of a scene or an environment

(5) scene-to- scene— between different scenes

(6) non-sequitur— have no apparent meaningful relation

If you want to learn more about his research, here is his website, http://www.visuallanguagelab.com/vitae.html It is a good way to understand Jackendoff.

Warhol’s Marilyn Diptych, via the Tate’s website.

Andy Warhol’s celebrity prints are clearly icons (explicitly representing real people), yet he uses them as symbols for…what? “[D]eath and the cult of celebrity” according to the Tate website where I found the above (ahem, digitized) version of the (ahem, iconic) Marilyn Diptych. In other words, The Marilyn Diptych is an argument (“a sign which is apprehended to be a symbol” in Parmentier’s words) by Pierce’s standards. Parmentier explains in Foundations of Piercean Semiotics, “a proposition… determines its interprétant to represent it as being merely an index of its object. Now this is not to deny that the interprétant still represents both a term and a proposition to be conventionally related to their objects; the claim being made is that, in addition to this level of representation, interprétants have the power to apprehend semiotic grounds as being other than they are.” Here Warhol’s purpose isn’t to depict an individual – we are not, for example, seeing her humanity – his purpose is to evoke our cultural encyclopedia and examine “her” meaning within it.

Dr. Irvine’s “The Grammar of Meaning Making: Sign Systems, Semiosis, and Cognitive Semiotics,” poses the question “Is language the main, or sufficient, modelling system for all human symbolic activity?” Assuming this means spoken or written language, I think the Marilyn Diptych is a clear example of why words aren’t always the strongest symbols. Could the word “Marilyn” printed a million times have achieved a similar effect to the photo? It could absolutely represent something powerful, perhaps even a related “message,” but it would be no substitute for the impact of a familiar image. A face has a singular visual impact – and the depiction of faces in two dimensions is part of a long tradition of portraiture. An instantly recognizable face, simplified, rendered in a range of colors and shades, printed 50 times edge to edge, evokes another layer of meaning for a modern audience. 

Warhol created visual impact with both mundane and grand subjects by applying a blend of high art and commercial design principles. In fact, the symbolic function of his grand subjects (pop culture icons) is mundane and the symbolic function of his mundane subjects (Campbell’s Soup can) is grand.

Warhol’s “Shadows” series, via the Hirshhorn Museum website.

Which brings me to the “Shadows” series. This is my personal favorite body of work by Warhol, yet I wonder if its semiotics are too ambiguous for collective interpretation. This series has had mixed reviews and Warhol himself famously referred to it as “disco decor.” Here he has chosen a subject that is unquestionably mundane yet perhaps a bit grand in its eternal mystery – a shadow. He has treated it with a similar technique to previous works, experimenting with color and repetition. He (or an assistant) has slathered on different colors of paint with a mop. And then, if you look at it in the context of the Hirshhorn exhibit in the photo above (or in person as I had the good fortune to do), a curator has marked Warhol’s argument with the curved wall of the museum and his or her own decisions about how to order the paintings. Can we agree on what it says? Probably not. What does the ambiguity mean?

Jackendoff’s parallel architecture is described as maintaining variations of structures that refer to a semiotic process involving different ways of interpreting phonetics and syntax, “The parallel constraint-based architecture is logically non-directional” (Jackendoff).  “The person who has acquired knowledge of a language has internalized a system of rules that relate sound and meaning in a particular way” (Chomsky). Generatively, Peirce’s developed a theory of “seeing the meaning process as a nexus of relations and mediation that enable thought to be productive of new meanings.” This parallel or non-linear process can also be borrowed to explain Postmodernism trend in art.

For instance, Sherrie Levine painted After Joan Miro (1984) in which she was mimicking Miro’s Triptych Bleu. Levine’s work maintains different interpretants of semiosis in generative memories. Viewers who are familiar with art would find this painting quite alike Miro’s masterpiece and would read this painting from a critical perspective. Kuspit described postmodernism as, “It constructed absurdity supposedly piques the reader’s interest or draws his attention, an exercise in curiosity that makes the whole enterprise worthwhile, or at least intellectually justifies it. In fact, it is a kind of intellectualization of the already intellectualized — the already known, historical, thematized, conceptualized and thus categorically the case.” However, for people who have no experience in art history and Miro is not in their lexicon, they would read the work as a tranquilizing abstract painting. “This icon can have a different object from that of the interpretant found by the viewer, and it could be understood through different grounds for different interpretants (Peirce).”

(Sherrie Levine, After Joan Miro, 1984)

(Joan Miró, Blue I, Blue II, and Blue III, 1961)

Kuspit explained that, “The artist becomes a cunning manipulator of the linguistically given, and the viewer an educated reader, rather than a person who has a certain complex, sometimes unexpected, not always immediately intelligible experience of the art.

Like language, art cannot be understood without the culture environment. In Barthes’s words, embedded in Sherrie Levine’s 1982 “Statement,” art is “a tissue of quotations drawn from innumerable centers of culture…. A multidimensional space in which a variety of writings, none of them original, blend and clash.”

Artists used parody, remix and collage made semiosis an unlimited meaning making process. Art, like language, is based off of cultural and personal grounds, because everyone’s generated grammar is different from another’s, an interpretation of art could be different from person to person. The example is to show that there’s no defined lexicon and both human language and other similar cognitive world such as art and film. In order to communicate with others, a continuous learning process is necessary.

Donald Kuspit, The Semiotic Anti-Subject, Artnet, 4.20.2001. [Essay on the postmodernisms (plural) in recent art.]

Signs, Symbols, Cognition, Artefacts: A Reader of Key Texts (Google Doc). 

Martin Irvine, Introduction to Meaning Making, Symbolic Cognition, and Semiotics.”

Although Peirce developed his theory of semiotics long before the “digital age,” the emergence of computational devices illustrates the adaptability of his model. Take, for instance, the icons on a computer’s desktop. How might they function semiotically according to Peirce’s schema of representamen, interpretant, and object, and how might a digital example prove more useful in clarifying the theory than one focused on “natural” signs?

According to Chandler, the representamen, or sign vehicle, is often misunderstood to exist materially in all cases (29). If we take, for example, a cloudy sky suggesting rain, these dark clouds would function as the representamen. However, the materiality of the clouds might lead to misunderstanding the representamen as always taking a material form. This is not the case, as we see with desktop icons. When I see a folder icon on my desktop, the representamen is far from material. Although it might rely on the material hardware of the computer in order to appear, I am not able to determine the physical existence of the folder, mainly because it does not exist physically but only as a metaphor for storage. As a result, this digital example precludes misunderstanding the representamen to always exist as a material object.

There appears to be a similar amount of confusion regarding the Peircean object. For Chandler, the object is “something beyond the sign to which it refers (a referent).” However, there are at least two things which exist beyond the sign and to which it refers: the mental concept and the thing itself. For instance, if we take the word “tiger,” both the tiger itself and the concept of tiger exist beyond the sign and, potentially, are referred to by the sign. However, as Prof. Irvine notes,” An object in his terminology is a position in a cognitive relation — a concept, idea, code, or mental rule,” not a “real-world referent” (“Semiosis and Cognitive Semiotics” (21). This, admittedly, confused me for quite some time because structural linguists, notably Benveniste, use the term “referent” to mean the thing itself. However, what Peirce labels the “object” more closely resembles Saussure’s “signified,” or mental concept. Therefore, the object of the word “tiger” is the mental concept of tiger, not the, or a, physically existing tiger. This confusion can be avoided through the use of digital examples. If I am trying to determine the object of a folder icon, I know that the icon does not refer to the real-world referent (i.e. a physical folder), but to the concept of a folder.

In addition, digital examples prove Peirce’s observation that signs point to other signs. The sign of the folder icon points to other signs (such as a conventional folder) which point to yet other signs in an infinite chain. However, I’m still left wondering how this might relate to poststructuralist notions of the chain of infinite signifiers by which the signified is never reached. In this model, meaning is always unstable because there is no transcendental signified by which to ground it. Is this also the case in Peirce’s semiotics?