visual art and computing in semiotics

Visual art and computing in Semiotics

It might not be a bad idea to start with the image done by Kara Walker in the latest exhibition Harper’s Pictorial History of the Civil War for this week. For me this image serves as a great example of the sign as an index. Considering the idea that an index is what calls our attention or what startles us (not necessarily need to be pointing to a physical or material object as is mentioned earlier), for me what draws my attention first is the breast of women, or better, the breast of the person. By catching the breast in this image, I then realize that the author probably wants to express an image of woman, or the whole category of women. My experience is a great example of seeing the representamen (the breast) as the box and label and realize there is something this representamen stands for inside the box, which is the refrent or the object (in this example, women). And according to Peirce’s model, the agency that enables me to correlate the breast with women together is the interpretant. Now the other interesting point is that while already knowing that Kara Walker is a feminist, I almost immediately correlate the image of the person into the category of women instead of specifically one particular woman. However, what if I know nothing about feminist and the only thing I know about this exhibition is civil war? My attention might therefore be driven by the facial features of both the two larger black figures and the smaller white figures. In the former case I am thinking Kata Walker’s intention of showing the power and liberation of women and in the latter case I am thinking about the elimination of slavery. My experience now might be able to explain how intersubjective and inter-individual social bonds of sign users, which are installed by symbolic cognition in lived context are preceding individual understanding of expressed meaning, in other words, meanings are dialogic and context-dependent.

Meanwhile, according to the fact that semiotics is the general discipline devoted to the study of signs and symbols as systems of expression, meaning, reasoning, and cultural memory in any kind of artefact (human-made being), we can also apply Peirce’s model into other fields. Still using this image above, how can we tell computer to represent it? Or broader, for every visual file, how do computer represent it? What we currently know is for an analog image, on computer we digitalize it into thousands of pixels where we give each pixel an RGB color for which we use a byte to represent. Before each color-standing byte we also have a key byte to tell the computer how many pixels are going to be filled with color and if we use one byte to give these pixels the same color or we assign each pixel with a different color by different bytes. From this level if we go up towards the top level we might go through more and more abstract and human readable algorithm and if we go down we are then amazed by how human beings are able to use 0s and 1s to represent ON and OFF which, is achieved by perceptible, physical and materialized combination of transistors. We use 0 and 1 as symbolic signs to represent the real electric states, where we might not be aware of a consensus within at least the community of engineers that such relation is achievable in computer (interpretant), or based on this interpretant, we can go one step further: we can break down analog things into digital one, separate it into several Boolean states and using logic gates on computer hardware to instantiate them. The interesting thing is that if we ask why through every detail of the above process, it might take us forever to answer the question (we are not even able to pause a while on the hardware where we find the exact real thing to interpret one concept).

But back to the idea of seeing a box and realizing there is something inside the box, symbols are not static or isolated things, but are structures of patterns in a dynamic cognitive process for meaning-making, a process about which we can never be consciously aware and cannot be observed except in the perceptible expressions and representations. The first point is that we are able to keep generate unlimited meaning with the help of word which is unique within human beings, the second point is unfortunately we might never be able to touch the object inside the box. That could probably explain why on Peirce’s successive model, the object appears always like the outsider: the line connecting infinite generated representamens and interpretant, probably using language as the agency, can only let us think in signs instead of things. I’m not sure if it’s appropriate to bring Lacan’s viewpoint here, but an interesting idea he brings is that unconsciousness is the construction of language and on the symbolic chain human beings are never able to touch the thing. Paradoxically, while human beings are able to live in a second world with the help of language, this second word, somehow doesn’t have an interface with the real world we live in, in other words, we are distinguished from other animals by language, but such privilege later undergoes an abdiction by index signs where we points the real thing find ourselves, like animals, finally pause a while on a real thing.



  1. Lacan, J. 1965. “The Mirror Stage as Formative of the Function of the I as revealed in Psychoanalytic Experience,” “The Insistence of the Letter in the Unconscious” in Ecrits.
  2. Kara Walker: Harper’s Pictorial History of the Civil War (Annotated). (n.d.). Retrieved February 14, 2018, from
  3. Irvine, M. 2018. Meaning Systems, Communications, Media and Technologies.
  4. Irvine, M. 2018. Introduction to Signs, Symbolic Cognition and Semiotics.
  5. Irvine, M. 2018. Semiotics: Meaning systems, Culture, Technology Key Texts.