Author Archives: Katie Oberkircher

De-Blackboxing Meaning Processes and Surrealist Art (Katie Oberkircher)

Abstract

Surrealist art is one of the most elusive genres to define. It is rendered realistically, but represents unrealistic, dreamlike states. As André Breton maintained in Manifestoes of Surrealism, surrealist art is based on the “play of thought” and “omnipotence of dream” (Breton 26). This representation complicates how we make meaning. I will use the Peircean semiotic model combined with Ray Jackendoff’s Parallel Architecture to explore feature extraction within paintings under the genre of surrealist art.

The process of feature extraction is dependent on culture and socialization. According to Dr. Irvine, “Peirce saw that one of the most important affordances of human symbolic cognition is precisely that which is combined in the dual necessary “platform” of semiosis: symbolic thought and expression necessarily require material-perceptible sign systems functioning in necessarily social, collective, and intersubjective communities of meaning-making (Irvine, “A Student’s Annotable Peirce” 3). In other words, semiosis is dependent not only on an interpreter’s ability to think symbolically, but it depends on the social context he or she uses to assign meaning, in this case, to the pieces of the paintings. It is important to acknowledge the tension between the individual nature of interpretation and the social context in which it is based in order to explore our semiotic understanding from a more well-informed perspective. This exploration will begin with the question: How does the combination of culturally implied meaning and feature extraction impact our understanding of surrealist art? How does a de-blackboxing process occur?

To achieve a deeper understanding of the de-blackboxing process that occurs when unpacking the surrealist tradition, I will focus on two paintings: René Magritte’s The Human Condition, (or La Condition Humaine,) (1933) and Salvador Dali’s Melting Watch (1954). These paintings are instances of the surrealist genre where viewers draw meaning by simultaneously understanding and subverting reality. From these paintings, I will explore the dialogic nature of these cultural artefacts in the context of the continuous historical, social and artistic network in which they reside.

Introduction

The research question stated above prompts an understanding of Charles Sanders Peirce’s semiotic model and Ray Jackendoff’s Parallel Architecture as they apply to other meaning systems besides language. These models and ideas will serve as a foundation to explore my research question: Peirce’s triadic model includes the representamen, interpretant and object.

  1. The representamen is the perceptible sign to which we associate meaning. In art, this would mean the elements of a painting (color, brush strokes, subject of image, etc.).
  2. The interpretant is the relationship between the viewer (or cognitive agent) who makes associations and the representamen. In art, this would be the recognition of the connection between elements of a painting.
  3. The object relates to the meaning associated with the representamen. In art, the emotional and/or social response generated by style and composition of a work.

Jackendoff’s Parallel Architecture can be leveraged beginning with this explanation: To comprehend language, Jackendoff gives us parallel architecture as a way to understand the relationship between semantics, syntax and phonology. These elements only produce sentences because they happen simultaneously (Jackendoff 126). Further, different combinations of these elements impact the construction and meaning of sentences, which we understand in the context of our environment/community. This idea will be extrapolated by focusing on the combinatorial power of various sign and symbol systems.

To understand how unpacking art is both similar and different than the process we use to understand language, I will include a historical and conceptual recovery, where I will use Jackendoff and Peirce’s ideas to make discoveries about surrealist art. Further, I will investigate what constitutes an appropriate interpretive possibility within the meaning system the paintings reside in (as well as the viewers who interpret them).

The historical perspective will include an examination into the meanings of these works in the dialogic network that precedes the artists as well as the interpreters in today’s community. The conceptual recovery will explore how signs and symbols are determined by possible meanings, and further, clusters of meanings within a genre. By presenting a simultaneously historical, contextual and conceptual perspective of surrealist art, the fluid, ongoing nature of sign formation can be more effectively understood. From here, we can see how signs gain symbolic significance when examining specific surrealist works as a whole. It is the combination, or marriage, of elements within a painting that allows us to enact meaning.

Main body of paper

In its simplest terms, the surrealist art movement was just that: a movement away from traditional bourgeois styles of art toward a newly subversive genre that “probed the structure of consciousness itself” (Hopkins 20). The movement was officially born in 1924 and grew to become a global phenomenon before its demise in the 1940s (Hopkins 15). In that time, artists such as a Salvador Dalí, Joan Miró, and André Masson and René Magritte used various approaches to transcribe “dreams” using “considerable conscious deliberation” (Hopkins 37). Their motivation to create realistically unrealistic paintings stemmed a belief that modern art could “forge a new relationship with its audience,” (Hopkins 21). This relationship changed the social experience viewers had when interpreting surrealist works through a “two-pronged attack on bourgeois social conventions and the aestheticism of an earlier Modernism” (Smith and Wilde 405).

In his work Dada and Surrealism: A Very Short Introduction, Hopkins articulates “surrealist artists such as Dalí or Magritte are frequently denigrated as ‘literary’” (Hopkins 82). Although a less currently relevant critique, Hopkins aligns their work with poetry. The word poetic suggests that these artists had the ability to appreciate complex emotional or aesthetic themes. Coupled with their sensibility, the word literary brings me to my unpacking of The Human Condition and Melting Watch in terms of Peirce and Jackendoff.

“Everything we see hides another thing, we always want to see what is hidden by what we see.” -René Magritte

With Peirce’s model as a foundation, we can see dense clusters of tokens and types within The Human Condition that impact our understanding of the relationship between normal and extraordinary.

The Human ConditionSource: http://www.nga.gov/content/ngaweb/Collection/art-object-page.70170.html

We begin by recognizing the painting as a cultural artefact of the surrealist movement, which allows us to recognize and extract the “Magritte” features and establish multiple chains or levels of Interpretants (Irvine, “A Student’s Annotable Peirce” 6). By conceptually labeling the painting as a piece of this larger genre, we look at it through the surrealist lens. In this way, the painting itself is a “complex Interpretant of many other paintings and types and styles of painting” (Irvine, “A Student’s Annotable Peirce” 9). This is what Peirce would refer to as an “immediate interpretant,” and is thus, the first layer of meaning (Irvine, “A Student’s Annotable Peirce” 6).

This interpretation begins we see this painting in its entirety, as one token, or instance of a type. When we break down the elements within the painting, we see a window with a view of a tree and forest area framed by curtains. These icons, in Peirce’s terms, are: the window itself, curtains, tree, tree line/forest, dirt path, and the blue sky and clouds (indicating that the scene looks from the perspective of inside to outside).

screen-shot-2016-12-15-at-10-17-03-am    screen-shot-2016-12-15-at-10-17-10-am

screen-shot-2016-12-15-at-10-17-27-am

screen-shot-2016-12-15-at-10-17-37-am          screen-shot-2016-12-15-at-10-17-53-am

These aspects of the painting are defined as icons because they are not necessarily symbolic. They are images that resemble or imitate an object. Our ability to recognize these icons connects to our pre-established meaning environment. As Dr. Irvine explains, much of what exists around thought and meaning is culturally implied. We reference our “encyclopedia” of “conceptual/symbolic cultural shared” knowledge (Irvine, “Semiotics, Symbolic Cognition and, Technology” 32). Through this action, we imagine a window, window dressings and a landscape as we have been taught to recognize them. Thus, these icons are representamen in Peirce’s terms.

However, once the eye moves to the three legs of the easel, our interpretation is complicated.

screen-shot-2016-12-15-at-10-18-05-am

The view outside the window is interrupted by a few uncharacteristic icons: The clip at the top of the easel interrupts the image of the sky.

screen-shot-2016-12-15-at-10-18-24-am

The sides of the easel break the seamlessness of the landscape as well as the line of the curtain.

screen-shot-2016-12-15-at-10-18-33-am                 screen-shot-2016-12-15-at-10-18-40-am

Therefore, what we first identify as icons is problematized by the fact that they stretch onto multiple plains (the painting itself and the painting within the painting). Viewers must understand how the icons are simultaneously as they appear, and not as they appear. For example, what looks like one, single dirt path that extends across the image is actually two separate renderings. This subtle shift from a seemingly normal landscape to something entirely different reflects two themes associated with Magritte: repetition and “masquerading as ordinary” (Magritte et al. 20). This landscape appears to resemble a common, familiar outdoor scene, as does the painting on the easel. However, that the painting does not necessarily reflect what the viewer cannot see. Therefore, the themes of repetition and replication are complicated, since viewers cannot actually determine what exists behind the painting on the easel. We cannot determine the painting on the easel as an accurate representation and it cannot fill in the gap of what we, as viewers, cannot see when looking out the window. In his works, Magritte used “normalness” to invoke memorability (Magritte et al. 20). Once these elements outlined above are identified as dissonant, and in this way not ordinary, we can begin to dissect the painting as something other than a landscape through a window.

The Human Condition becomes symbolic when we see that there is a painting within a painting. The painting itself represents a cultural artefact in the surrealist genre, and it functions as a singular piece of art completed by Magritte. Within this image is another painting. We know it is a painting because of the way it is depicted on the easel and canvas. Viewer expectations are subverted due to the realistic representation of the painting and the assumption that the painting reflects the view that is blocked by the canvas. In this way, reality is questioned because viewer assumption is based on a false premise. The icons situated in Magritte’s work are “real” if the painting on the easel iconically represents that reality. In keeping with Magritte’s stylistic features, “His paintings from this period are invitations to look closely and to pay careful, critical attention to what is seen, both in each image and, when turned away from them, in the surrounding world (Magritte et al. 21). Thus, the viewer sees one depiction as real and the other as a representation. The painting itself appears as real, while the painting within it appears as a replication.

This view, however, is not possible when looking at the surrealist work as a whole. Both the painting and the painting inside are a part of the same work, suggesting that all of the elements work in parallel to contribute to a unified sense of meaning of the whole image. In his work Foundations of Language, Ray Jackendoff articulates the “multiple parallel sources of combinatoriality, each of which creates its own characteristic type of structure” (Jackendoff 107). His explanation centers around the properties of language (phonology, syntax, and semantics), but if we extrapolate his understanding of the combinatorial power of one symbolic-cognitive system to the interface of surrealist art, we can see how separate elements of the painting come together to form one whole image, or instance of surrealist art. When discussing the linguistic structure, Jackendoff explains it as a “collection of independent but linked levels of structure” (Jackendoff 131). This explanation connects to Peirce, who articulated that sign/ symbol structures are stacks of features developing in parallel as a result of logical sequences (Irvine, Presentation for discussion: “Cognition, Symbols, Meaning” Slide 53).

In this way, the icons represented in The Human Condition, link together to form multiple combinations of meaning. These icons establish the connection between certain syntactic constituents with conceptual structures (Jackendoff 131). For example, the painting on the easel is only identified as a replication of a landscape because of the way it is portrayed. It is sitting on an easel, painted on a canvas, inviting viewers to think that the painter used the portion of the scene that we cannot explicitly see as his or her subject matter. All of the separate pieces of the image come together simultaneously when viewers make the connection that the portion of the window scene that they cannot see is not necessarily what is represented in the painting on the easel. Thus, we can understand the work as a meta-painting. All at once, The Human Condition becomes one whole image as well as a series of separate renderings.

“Those who do not want to imitate anything, produce nothing.” -Salvador Dalí

Much like Magritte’s work challenges what is ordinary, Salvador Dalí’s Melting Watch challenges how we conceive time and reality. He does this by presenting the concept of time in a way that is unfamiliar to us. According to Hopkins in Dada and Surrealism: A Very Short Introduction, Dali’s paintings discredited reality (Hopkins 100). Melting Watch discredits reality by showing how time is warped in a dream setting. This type of work falls into the surrealist genre, and it should be noted that this conceptual label is an immediate interpretant. By assigning this label, we can begin to understand how the components of the painting impact the syntactical and conceptual interfaces of surrealist art.

screen-shot-2016-12-15-at-10-33-23-amSource: http://www.dalipaintings.com/melting-watch.jsp

In keeping with the surrealist genre, Dalí uses the distorted watch to symbolize how time passes while we are dreaming (MoMA). In this way, the watch acts to subvert our understanding of time. To come to this conclusion, though, we have to first identify that the object in the picture as a clock, then we have to analyze how it is being represented differently than a normal clock, so that we can understand that Dalí is actively portraying the construct of time as arbitrary and useless.

Therefore, while we do recognize the object as a watch, we have to do so by acknowledging it is less watch-like. This process, when described by Dr. Irvine, involves “Meanings, intentions, or values [which] are not properties of — or “baked into” — any substrate or medium used to produce perceptible, interpretable forms” (Irvine, “Introduction to Technical Theory of Information” 5). Meaning does not come from the medium of art, but it is enacted by viewers who use their collectively understood material sign structures in the context in which they reside to interpret and understand what they see.

We can label the watch as an “icon” because it “resembles or imitates” an object, but there are characteristics that do not match up with our mental image, creating a tension between what we see and our previously established reference to a clock (Irvine, “Semiotics, Symbolic Cognition, and Technology” 31). We see the object is circular, has two hands to denote the hours and minutes, and has numbers spaced out evenly. However, its distorted, exploding state suggests it is more of a “hypoicon,” specifically a metaphor, which denotes the representative character of a sign by representing a parallelism in something else (“Semiotic Elements and Classes of Signs” 7). In this case, the shape informs us that the watch symbolizes something other than time as we understand it. It does not represent it as structured and powerful.

Other icons include the moth, fly, water, numbers on the clock and numbers separated from the clock and the mountains in the lower right hand corner of the image.

screen-shot-2016-12-15-at-10-33-37-am   screen-shot-2016-12-15-at-10-33-45-am  screen-shot-2016-12-15-at-10-33-51-am screen-shot-2016-12-15-at-10-34-25-am screen-shot-2016-12-15-at-10-34-06-am screen-shot-2016-12-15-at-10-34-12-am

In connection with Peirce’s theory of semiotics, signs and symbols are replicable expressions that are only recognized to have meaning because they are understood (Irvine, “Introduction to the Technical Theory of Information” 6). Viewers understand the separate pieces of this painting as replicable objects that they have seen, or experienced before, which enables them to identify the objects. This is the first layer of meaning.

Once these objects are examined in the context of Melting Watch, their status becomes both iconic and symbolic. To further interpret the theme of time, we can look at the icons as they relate to the clock. To demonstrate this process, I will look specifically at the moth and the number five. The presence of a moth in close proximity to the clock could indicate that it represents something other than just an insect.

screen-shot-2016-12-15-at-10-33-37-am

This leads to the question: is the moth connected to the watch? Is it connected to how we conceptualize time? This realization bridges the gap between iconic and symbolic significance. Moths are a nocturnal species, which indicate nighttime and connect to a manipulation of time during a dream. They are also attracted to light. The shadows in the image suggest that there is some source of light. There are many connotations associated with moths, which open up a wide range of possible meanings depending on how viewers combine aspects of the image. Moths can signify vulnerability, death, faith, transformation and sensibility (Leith 56). These possibilities alone offer multiple interpretations of Dalí’s work and relate back to the specifically “Dalí” features of this surrealist work, which centered on “dream painting” and subversion of reality (Hopkins 40).

Similarly, the number five, which is separated from the clock as an object, questions what is real with regard to time.

screen-shot-2016-12-15-at-10-34-25-am

Frozen in time, it no longer marks the hours or minutes, but exists as an object in opposition to the structure of time. To interpret the number in this way, viewers have to acknowledge the way we tell time and the role that the numbers on a watch play in framing our conception of time. To connect this discovery to Peirce’s explanation of semiosis, the number five is first identified as an icon because it resembles an object, then as an index in how it is used in relation to keeping time, then as a symbol in how it is positioned away from the clock, suggesting its disassociation with the traditional understanding of time.

The process of sign formation is fluid, though. That is, to understand why the number five is separated from the watch is to understand its relation to clocks, time and reality. Jackendoff explores this idea when he says “The parallel constraint-based architecture is logically non-directional: one can start with any piece of structure in any component and pass along logical pathways provided by the constraints to construct a coherent larger structure around it” (Jackendoff 198). The sign systems work concurrently in a parallel architecture in which all of the layers are engaged simultaneously in the sign processes of any one of images individually. Further, as Dr. Irvine writes, “creating combined symbolic expressions that require engaging two or more sign systems in parallel…” involves “visual representations with many “vocabularies” of image-making…” (Irvine, “Introduction to Linguistics and Symbolic Systems” 16). In other words, when viewers look upon Melting Watch, they are drawing upon collective memory to engage in a contextual and conceptual way, creating a parallel between symbol systems.

Conclusion

 Surrealism as an artistic movement challenged how viewers interact with and understand subject matter using various sign and symbol systems through feature extraction. Surrealist art provided a new opportunity for viewers to use their imaginations to actively subvert what they believed they knew about art, culture and most generally, about the themes of time and space. By reinforcing this narrative through feature extraction, we build upon the culturally constructed meaning system.

These meaning systems are part of complex and deep ideologies and value systems that we activate in order to produce interpretant expressions for the meaning, in this case, of two surrealist paintings (Irvine, “A Student’s Annotable Peirce” 8). Surrealist paintings can act as an interface to a larger system of meanings and values. From here, feature extraction is used to identify the perceptible features that appear in the images, in the way they are presented (e.g., in a museum) and the way they are described. These features are situated in the context of historical and conceptual interpretation associated with the paintings. Both The Human Condition and Melting Watch are a part of a cultural meaning system framed by institutional contexts.

It is important to note that meaning comes when we, as cognitive agents, enact it. This process happens on a few levels and can be understood by bringing together Peirce’s semiotic model and Jackendoff’s Parallel Architecture. The layers established within and around each surrealist painting are pieces of a greater network of sign systems. The sign systems work concurrently in a parallel architecture in which all of the layers are engaged simultaneously in the sign processes of any one of images individually. This is because the layers of abstraction and representation are in an evolutionary ecosystem of sign systems. As previously noted, the surrealist movement of art declined in the 1940s. Viewer understanding of The Human Condition and Melting Watch is impacted by the current community of interpretation in which they reside. These communities offer spaces for discovery of the meaning processes behind surrealist art.

Further, it should be noted that while this paper concentrated on feature extraction in a historical and conceptual context, it did not focus on the construction of meaning within the physical space at a museum. There are opportunities for further research when exploring museums as systems of categorization, memorialization and classification.

Even still, these surrealist works serve as examples of how dreamlike features prevent viewers from rationally understanding what a picture represents. To interpret the dense clusters of signs and symbols within these paintings, it is important to acknowledge the combinatorial power of the elements as well as the fluid, ongoing nature of interpretation and sign formation. Especially with regard to surrealist, and therefore subversive, imagery, understanding how each sign works both separately and in conjunction with the rest of the image is crucial to de-black boxing the function of illusionistic representational paintings. Combinatorial power is relevant in how it reveals the significance of realistically depicted, yet simultaneously dreamlike renderings presented by surrealist artists.

Works Consulted

Ades, D., & Vancouver Art Gallery. The colour of my dreams: The Surrealist revolution in art. Vancouver: Vancouver Art Gallery. 2011. Print.

Breton, André. Manifestos of Surrealism. Trans. Richard Seaver and Helen R. Lane. N.p.: U Michigan P., 1969. Print.

Clark, Andy and Chalmers, David. “The Extended Mind.” Analysis 58, no. 1 (January 1, 1998): 7–19.

Clark, Andy, Supersizing the Mind: Embodiment, Action, and Cognitive Extension (New York, NY: Oxford University Press, USA, 2008).

Fanés, F. (2007). Salvador Dalí: The construction of the image, 1925-1930. New Haven: Yale University Press.

Hopkins, David. Dada and Surrealism: A Very Short Introduction. Oxford; New York: Oxford University Press, 2004.

Irvine, Martin, “A Student’s Annotable Peirce: Primary Texts On Signs and Symbolic Thought With Transcriptions of Unpublished Papers from Peirce’s Manuscripts”

Irvine, Martin, Presentation for discussion (conclusion): “Cognition, Symbols, Meaning

Irvine, Martin, “Introduction to Linguistics and Symbolic Systems: Key Concepts” (intro essay).

Irvine, Martin, “Introduction to the Technical Theory of Information” (and using Information Theory + Semiotics)

Irvine, Martin, Selections from: Semiotics, Symbolic Cognition, and Technology: A Reader of Key Texts

Irvine, Martin, The Grammar of Meaning Making: Sign Systems, Symbolic Cognition, and Semiotics. Communication, Culture & Technology Program, Georgetown University.

Jackendoff, Ray, Foundations of Language: Brain, Meaning, Grammar, Evolution. New York, NY: Oxford University Press, USA, 2003.

Leith, James A., and George Whalley. Symbols in Life and Art. Kingston, Ont.: Published for the Royal Society of Canada by McGill-Queen’s UP, 1987. Print.

Luhmann, Niklas. Art as a Social System. Trans. Eva M. Knodt. Stanford, CA: Stanford UP, 2000. Print.

Magritte, R., In Umland, A., D’Alessandro, S., Museum of Modern Art (New York, N.Y.), Menil Collection (Houston, Tex.), & Art Institute of Chicago. (2013). Magritte: The mystery of the ordinary, 1926-1938. New York: Museum of Modern Art.

“MoMA Learning.” MoMA. The Museum of Modern Art, n.d. Web. 14 Dec. 2016.

Pinker, Steven. Big Think. Steven Pinker: Linguistics as a Window to Understanding the Brain. Accessed September 19, 2016. https://www.youtube.com/watch?v=Q-B_ONJIEcE.

“Semiotic Elements and Classes of Signs.” Wikipedia. Wikimedia Foundation, n.d. Web. 13 Dec. 2016.

Shanes, Eric, Salvador Dalí, Inc Ebrary, and Proquest (Firm). The Life and Masterworks of Salvador Dalí. Vol. Rev. and Updat 2nd;1. Aufl.;Rev. and Updat 2nd;2nd;. New York: Parkstone International, 2010;2012;. Web. 5 Dec. 2016.

Smith, Paul, and Carolyn Wilde, eds. A Companion to Art Theory. Oxford: Blackwell, 2002. Print.

“The Human Condition, 1933 by Rene Magritte.” The Human Condition, 1933 by Rene Magritte. ReneMagritte.org, 2009. Web. 12 Dec. 2016.

Zaslavskii, O. B. (2005). Language as an underlying idea in Salvador Dali’s works. Word & Image21(1), 90-102.

Computation as a Practice of Discovery (Katie)

Never did I ever think that I would explore computational thinking. I’ve typically approached academic thinking and reasoning concretely, and perhaps too narrowly, especially in light of my decision to apply to the CCT program. Until now, I’ve believed technology to be constructed in opposition to human intention. However, since the beginning of the semester, I’ve started to see that the meeting point of technology and communication can be studied by exploring human meaning systems and technology as more than a black box.

Thinking in a way that bridges computation, cognitive science, information theory, semiotics, and linguistics has helped me to connect the dots between humans, computers and intelligence. I’ve started to view computer systems less as machines devoid of human reasoning, but instead as computational devices that are capable of extending and distributing human cognition. As cognitive agents, we use computer systems as a space to explore information processes through different concepts, designs and abstractions.

I have also begun to view this process on a historical continuum. As Dr. Irvine explains, “a contemporary computational system (large or small) is a design for implementing pre-existing human symbolic-cognitive processes to enable ongoing interpretations through the interactive metamedia design for all our digital encodable symbolic artefacts” (Irvine, 3). In other words, these interpretations are situated in time and the represent our current environment. They are moving pictures of meaning making. Because computers are a physical symbol system, they produce “an evolving collection of symbol structures” as they move through time (Simon, 22).

If we view computers as a way to understand human behavior, we can explore the relationship between objectivity and subjectivity as it relates to signs and symbols. That is, the objective parts of information are related to signs, while the observer of those signs is subjective because he or she imposes meaning on them (Denning, 808). Human symbolic thought is founded on the idea that we draw meaning from collectively “perceptible material and physical structures” (Irvine, 4).

Finally, I have a clearer understanding of computation as a practice of discovery. Through the extension of human cognition via a computer system, we can identify patterns that stand for something vs. those that do not. We use media, signal and symbol representations to identify the significance of these patterns. In other words, “the association between a representation and what it stands for” can be examined through computing (Denning, 808). In the context of a computer system, we act through an interface, which has specific design features and software layers that map pixals onto our screen in a particular way.

Although there is still much to learn and digest, I have started to chip away at the idea that “Intelligence is the work of symbol systems,” by bringing together different academic and scientific disciplines (Simon, 23). Computational thinking has helped me better understand the connection between computing, information and semiotics in the context of our socio-technical system.

References

Martin Irvine, “Introduction: Toward a Synthesis of Our Studies on Semiotics, Artefacts, and Computing.

Herbert A. Simon, The Sciences of the Artificial. Cambridge, MA: MIT Press, 1996. Excerpt (11 pp.).

Peter Denning, “What Is Computation?” Originally published in Ubiquity (ACM), August 26, 2010, and republished as “Opening Statement: What Is Computation?” The Computer Journal 55, no. 7 (July 1, 2012): 805-10. Note the important revised definition of “computation” by leaders in computer science:

Peter Wegner, “Why Interaction Is More Powerful Than Algorithms.” Communications of the ACM 40, no. 5 (May 1, 1997): 80–91.

Mediation and Transparency (Amanda and Katie)

These days, it’s easy to assume that we can do just about anything on our computers. Whether we’re working on a laptop, desktop, tablet, or even a smartphone, we are capable of listening to music, drawing and creating graphics, shooting and editing photographs, writing content, and and much more. According to Alan Kay, computers are – and were – the first “metamedium,” consisting of media that have either already been invented or have yet to exist (Manovich, 23).

Instead of drawing a graphic at a desk using pen and paper, we can use a stylus on a tablet and create the same image. There is the illusion that we’re interacting with a pen and piece of paper, but instead, the stylus is sending signals that the computer picks up on, and the pixels create the image that imitates the job that a pen would perform. As Jay David Bolter and Richard Grusin mention in their book, Remediation: Understanding New Media, even ten years ago, these metamedia did not exist; people saw computers as devices that were used exclusively as numerical engines (23), while individuals such as Alan Kay envisioned they be used as a much more basic, day-to-day communication platform that even children could utilize (Kay, 321).

Now, however, we think of computers in a whole new way – “…we now think of them also as devices for generating images, reworking photographs, holding videoconferences, and providing animation and special effects for film and television” (Bolter & Grusin, 23). As computer technology progresses, it appears that new media keep building upon each other, leading us back to the concept of metamedia. There is media within the first metamedium – the computer – creating a new dimension of metamedia that constantly develop, progress, and evolve; this causes the language that we use with the computer machine – the interface – to change, and we adapt with it.

As computer technology has evolved, we have made the transition from hardware to software. As Manovich explains, software re-adjusts and re-shapes everything that it is applied to, just like other technologies such as the printing press, the alphabet, and even the first computers. Software plays a vital role in shaping how we use and interact with our computers, and in turn, this relationship shapes the way that we contribute to our specific human culture (Manovich, 14-15).

An interesting concept that stems from much of this week’s readings is the idea of the socio-technical system, or the ways in which we are conditioned by the technologies that we interact with each day. While it appears that our computers can do everything, there’s still much room for progress, and Kay pointed that out when discussing his Dynabook concepts. Our devices come to us in what we think is the complete package – all of the hardware is put together, all of the software has already been downloaded, and all that we have to do is use the apps the way we’ve been conditioned to use them. However, no one is taught how to de-blackbox their device. As Kay states in his TIME Magazine interview, people – or children – cannot create apps for each other (Greelish, 2013); every portable computer these days contains elements of the Dynabook idea; however, they all lack the collaborative and inventive concepts that Kay and his team had hoped for in the 1970s (Greelish, 2013).

We wonder: why don’t computers come blank, as a blackbox? Kay had imagined something more complex, and perhaps he thought that people would be able to adapt to such an idea where we could individually create our own software and apps. However, this involves a certain element of literacy and knowledge in the field of computing, which many people lack. Computers are not sold as black boxes, and therefore, there is no need for people to learn how to de-blackbox them. Children are not taught how to build computers or software, and thus, we are stuck in a socio-technical system where we have only learned the basic symbolism behind computer interaction and use.

Computing can be understood as the intersection between human intention and the symbolic process. In this intersection, though, we often treat computer systems as black boxes. The knowledge we gain (and share) about our devices is limited by how we learn through an interface, so we are in some ways conditioned by the technology we use.

Moving forward, interfaces like voice and touch recognition, along with ones that foster education, knowledge sharing and procedural literacy, could aid in realizing some of Kay’s ideas surrounding software that gives “life to the user’s ideas” (Kay, 234).

In connection with the idea of sharing/collaboration, Kay discusses the “helpful agent,” which hasn’t been realized yet (Greelish, 2013). For example, we cannot create an app and share it directly with friends — we have to submit it to the App Store, wait for formal approval and then release it via Apple. In other words, we have not achieved “symmetric authoring and consuming” of computer stored data (Greelish, 2013).

In short, we do know that building blocks of a computer systems comprised a metamedium that houses and manipulates different data using certain techniques to create and store that data (Manovich, 110). 

As computer systems continue to evolve, it will be interesting to see how the relationship between hardware and software develops as well as the level of transparency that programmers and users seek when using devices that typically mediate how we understand, store, retrieve and manipulate information.

References

Bolter, Jay David and Grusin, Richard, Remediation: Understanding New Media. Cambridge, MA: The MIT Press, 2000.

Kay, Alan C. Xerox PARC, Alan Kay, the Dynabook/Metamedium Concept, and Possibilities for “Personal Computers”

Kay, Alan C.  Kay’s original paper on the Dynabook concept: “A Personal Computer for Children of all Ages.” Palo Alto, Xerox PARC, 1972).

Kay, Alan C. “Microelectronics and the Personal Computer.” Scientific American 237, no. 3 (September 1977): 230-44.

Lampson, Butler. Butler Lampson’s original 1972 memo on the Xerox Alto computer, the first “personal” computer implementing a GUI Windows and mouse system and networked via Ethernet.

Manovich, Lev, Software Takes Command, pp. 55-239; and Conclusion.

The Cultural Element (Katie)

This week’s readings helped frame computation as a stop on a historical continuum of cognitive artefacts in relation to affordances, meaning making and interpretation. All of these elements revolve around culture – this is a point that has interested me since the beginning of this course. An artefact is a meeting of properties of something and the environment in which it exists. Mahoney, in particular, articulates this when he says “any meaning the symbols may have is acquired and expressed at the interface between a computation and the world in which it is embedded” (Mahoney, 129). The world that meaning is embedded in is created by humans (cognitive agents). In other words, symbols have meaning to us – not computers – because symbols and the way we string them together expresses our representation of the world.

We derive meaning via an interface, which connects two systems by transcending the boundaries of those systems. As Irvine discusses, the interpretations we make come from the way we are socialized. Thus, affordances are “good” when they fulfill our expectations (Irvine, 3). Interfaces are created when we use perceptible, physical features to make meaning, which is reflective of our culture, values and intentions.

In this way, affordances draw meaning when they correctly communicate human intention through detectible features. As cognitive agents, we enact meaning through applied collective associations that we learn. Computing can be understood as the intersection between human intention and the symbolic process.

Some further questions:

As technology continues to evolve, should we be aware of the levels of mediation that affect meaning making? What if a new “meta layer” forms?

How specifically does the history of computation inform us moving forward? Do we ever take steps “backward”? What is the relationship between past and present?

References

Engelbart, Dave. 1962. “Augmenting Human Intellect: A Conceptual Framework.” New Media Reader. Wardrip-Fruin, Noah, Nick Montfort, ed.. 93–108. Cambridge, MA: The MIT Press, 2003.

Irvine, Martin. 2016. “Introduction to Affordances and Interfaces: The Semiotic Foundations of Meanings and Actions with Cognitive Artefacts”.

Mahoney, Michael S. (2005) “The Histories of Computing(s).” Interdisciplinary Science Reviews 30, no. 2 p.119–35.

Computation: More Than Just Programming (Katie Oberkircher)

While stumbling my way through Python, I found myself asking the question: How can we express human interests through computers?

Python uses a type of language grounded in computation. It’s founded on the idea of meta-function: some symbols can perform actions on other symbols. As Dr. Irvine explains in his video, symbols are a logical bridge – they provide a connection from human interests (also represented in symbols) to electronics and digital processing. We can then interpret and act on software encoded symbolic representations.

Since the symbols we use in software spaces don’t usually represent meaning, is it fair to say that in some ways, they are a means to an end?

We can better understand how creativity and efficiency relate to computation by developing procedural literacy. As Evan explains, “designed languages” are “created by humans for a specific purpose such as for expressing procedures to be executed by computers” (Evan, 19). This type of programming language is precise and unambiguous with a specific syntax.

I wasn’t aware of the level of precision of Python until I entered an invalid input and received an error. I added an extra space in the line: lion.(upper) and as a result, Python produced a message alerting me that I erred in typing a correct string of symbols. One space prevented the input from being interpreted correctly. (This is much different than natural language, which almost hinges on our ability to evolve and accept slight changes and adaptations of certain words and phrases.)

This level of detail seems simultaneously simple yet complex. It’s simple in how details are hidden, so we can focus on higher level operations and it’s complicated in how much control we have, as programmers, over certain mechanical resources (Evans, 38). In this way, we can think about abstraction as a means to differentiate between levels of programming language. In Great Principles of Computing, Denning and Martell indicate that there are over 500 programming languages (Denning and Martell, 84). These versions are based on abstractions, yet they are all precise and free of ambiguity. They are also based on scale, scope and complexity.

As I worked within Python, Jeannette Wing’s thoughts on humans as computers brought up a few other questions. If, based on speed and economics, humans can still solve some tasks better than machines, how can we reconcile this? Or, should we reconcile this? If, as Wing explains, we can do things like process and interpret natural language better than a machine, should we keep these ideas in mind when we measure an efficient, correct, usable abstraction?

References

David Evans, Introduction to Computing: Explorations in Language, Logic, and Machines. Oct. 2011 edition. CreateSpace Independent Publishing Platform; Creative Commons Open Access: http://computingbook.org/.

Martin Irvine, Introductory Video Lecture on Computational Thinking (Prof. Irvine, from “Key Concepts”)

Peter J. Denning and Craig H. Martell. 2015. Great Principles of Computing. The MIT Press.

Jeannette Wing, Computational Thinking YouTube video, 1:04:58. Posted by ThelHMC. October 30, 2009.

Message Delivered (Katie Oberkircher)

This week, the readings got a little meta. They brought up ideas like motivation, context, time and community, and they made me think about how we form relationships that frame the way we interpret signs and symbols. Dr. Irvine in his “Introduction to the Technical Theory of Information,” describes “meta-information” to be a collection of our past experiences (individual and collective), social norms, shared meaning, etc. (Irvine, 5).

In keeping with Peirce’s theory of semiotics, signs and symbols are replicable expressions that are only recognized to have meaning because they are understood (Irvine, Primary Texts, 6). It’s a seemingly simple point, but when we think about signs, whether that’s a word a cell phone screen or a key change in a musical composition, we interpret it in a particular context, drawing on our shared knowledge.

So when I send a text message, I assume the recipient will know exactly what I’m talking about. These assumptions are situated in background knowledge, social norms, etc., and they act outside of the signal itself. In this way, communication is never isolated. It exists in the context of past and future messages. If I send a text that says, “what’s up?” to my friend from high school, that single text becomes part of a continuous interaction. And when I send it, I’m drawing meaning from our relationship up until that current moment in time.

This thought process brought up a few thought/questions: Sending a text involves encoding, transmitting and then decoding information, but the simple act of delivery doesn’t really mean the message was “delivered.” To that end, technical, social and cognitive elements of meaning making seem to be directly related to each other.

Since we’re inserting agency into a system that’s comprised of words on a screen, are there boundaries to the meaning we can and should draw from a single text? With context in mind, I often find myself downplaying the significance of a text message because the medium suggests that the content is brief/superficial and it’s devoid of emotion. In fact, sometimes texts invite us to over demonstrate how we are feeling with emojis and punctuation to represent how we feel, or how we want the receiver to think we’re feeling.

Along those lines, what happens if someone sends a joke that is intended to be humorous but the receiver doesn’t understand it? Is this what Floridi refers to as entropy (Floridi, 47)? Does that reflect the nature of their relationship or does it simply mean that there are limits to how we can draw meaning from a text? Or is it a combination?

References

Floridi, Luciano, Information, Chapters 1-4 (Oxford University Press Inc., New York, 2010).

Irvine, Martin, “A Student’s Annotable Peirce: Primary Texts On Signs and Symbolic Thought With Transcriptions of Unpublished Papers from Peirce’s Manuscripts”

Irvine, Martin, “Introduction to the Technical Theory of Information” (and using Information Theory + Semiotics)

Unpacking the Concept of Time (Katie Oberkircher)

When you picture a clock, what do you see? Most likely a flat, circular object marking the time, mounted on the wall—or maybe you see a wristwatch, pocket watch or even a sundial. Either way, when we hear the word “clock,” we have an image that we reference. This reference connects to our pre-established meaning environment. As Dr. Irvine explains, much of what exists around thought and meaning is culturally implied. We reference our “encyclopedia” of “conceptual/symbolic cultural shared” knowledge (Irvine, 32). Through this action, we imagine a clock as we’ve been taught to recognize it.

Screen Shot 2016-09-27 at 9.46.04 AM

Source: https://www.moma.org/learn/moma_learning/1168-2

Salvador Dalí’s “The Persistence of Memory” challenges how we conceive time and reality. He does this by presenting the concept of time in a way that is unfamiliar to us. This type of work falls into the surrealist genre—and it should be noted that this conceptual label is an interpretant. By assigning this label, we can begin to understand how the components of the painting impact the syntactical and conceptual interfaces.

In keeping with the surrealist genre, Dalí uses the melting and distorted clocks to symbolize how time passes while we’re dreaming (MoMA). In this way, the clocks act to subvert our understanding of time. To come to this conclusion, though, we have to first identify that the objects in the picture are clocks, then we have to analyze how they are being represented differently than a normal clock, so that we can understand that Dalí is actively portraying the construct of time as arbitrary and useless.

Therefore, while we do recognize the objects as clocks, we have to do so by acknowledging they are less clock-like. We can label them as “icons” because they “resemble or imitate” an object, but there are characteristics that do not match up with our mental image, creating a tension between what we see and our previously established reference to a clock (Irvine, 31). We see the object is circular, has two hands to denote the hours and minutes, and has numbers spaced out evenly. However, its melted state suggests it’s more of a “hypoicon,” specifically a metaphor, which “represents the representative character of a sign by representing a parallelism in something else” (Wikipedia, 7). In this case, the shape informs us that the clocks are symbolizing something other than time as we understand it—they do not represent it as structured and powerful.

As the viewer, we grapple with interpreting the distorted objects. What does their physical nature tells us about time?

Through the lens of the surrealist genre, art questions our understanding of meaning because it presents images, concepts and arguments differently than how we normally perceive them. “The Persistence of Memory” contains objects that don’t quite fulfill the sign/symbol function that we expect them to, so our expectations have to shift to recognize how the work creates meaning. We can do this by considering all of the elements of the painting, including how we label it, how we perceive its components, and what we think the work means as a whole. In other words, we are parallel processing.

His technique also brings up questions about interpretation: is there a right and a wrong way to interpret this painting? How can we unpack his intentions in relation to the way he constructs symbols?

References

Martin Irvine, Selections from: Semiotics, Symbolic Cognition, and Technology: A Reader of Key Texts

(“MoMA | Salvador Dalí. The Persistence of Memory. 1931” 2016) https://www.moma.org/learn/moma_learning/1168-2

Ray Jackendoff, Foundations of Language, selections on the “Parallel Architecture” model of language as a combinatorial system. Chap. 5.5, pp. 123-128; Chap. 7, pp. 196-200.

Semiotic Elements and Classes of Signs (Wikipedia)

 

He be workin’ (Katie Oberkircher)

As Steve Pinker points out at the beginning of his video, language is at the center of how we think, how we evolve as humans, how we form social relationships, and how we understand biology. All of these factors together intersect with the themes of community and memory.

To explain language to someone unfamiliar with linguistics, I would start by explaining the relationship between words, rules and interfaces. We arrange words into combinations that follow certain rules to share and receive ideas. Then, I would refer them to Jackendoff.

To comprehend language, Jackendoff gives us parallel architecture as a way to understand the relationship between semantics, syntax and phonology. These elements only produce sentences because they happen simultaneously (Jackendoff, 126). Further, different combinations of these elements impact the construction and meaning of sentences, which we understand in the context of our environment/community.

Even though all societies have a language, we exist in specific “communication environments” within those societies, which influences how we understand each other (Irvine, 3). Dr. Irvine articulates that this environment is comprised of “assumptions, collective knowledge, and a repertoire of speech genres shared by participants but not explicitly stated in the formal semantics of expression” (Irvine, 3). Although sentences are constructed the same way, we rely on certain beliefs and collective information not necessarily reflected in the form of a sentence to understand each other.

Pinker explains this idea as a third interface between language and mind. We use the context of our social and cultural environment to understand language.

But how entrenched is language in a particular culture? This question brings up the idea of dialects. Pinker uses Ebonics/Black English as an example (He be workin’ vs. he workin’). The main verb “be” used here differentiates between employment status vs. someone’s current actions. Using the unchanged version of the verb “to be” confuses word tense, but indicates that actions are habitual, so we can infer that using “is” or “are” would not produce the same meaning in this context. So, people who use this dialect learn this rule in order to understand each other, although it is not a rule that we attribute to universal grammar.

This distinction brought up a few questions: Does dialect impact how we think? Is dialect considered a different language? In other words, what boundaries do dialects create, and are we aware of them? Do dialects overlap and/or evolve? Is it possible to know where one language leaves off and another begins?

Questions for further exploration:

  1. Is the way we communicate via social media considered a dialect? How does the inclusion of abbreviated phrases like LOL, u, nvm, onw, and other shorthand impact how we think and communicate face to face?
  2. If some thoughts don’t take place in sentences (i.e., music), how does language impact how we conceptualize those products? What is the significance of the intersection of language and nonverbal forms of expression?

Big Think. Steven Pinker: Linguistics as a Window to Understanding the Brain. Accessed September 19, 2016. https://www.youtube.com/watch?v=Q-B_ONJIEcE.

Martin Irvine, “Introduction to Linguistics and Symbolic Systems: Key Concepts” (intro essay).

Ray Jackendoff, Foundations of Language: Brain, Meaning, Grammar, Evolution. New York, NY: Oxford University Press, USA, 2003.

Non-Linear Technological Development

In “The Morning of the Modern Mind,” Wong concludes by highlighting the sometimes non-linear nature of technological advancement. She writes that “Tasmanians possessed a much more complex tool kit, one that included bone tools, fishing nets, and bows and arrows” several thousand years before the more recent Middle Paleolithic, who possessed “little more than basic stone flake tools” (Wong, 94). While she attributes this to the rising sea level cutting off the island to the mainland, this idea of non-linear progression prompted me to think about the ways that other technologies may have progressed non-linearly within the context of American culture.

After our demonstration of the telegraph in last week’s class, I thought about how the telegraph—essentially the original method of texting—infiltrated the U.S. long before cell phones did in 1973. Even with ability to talk live, we were still unable to text (until 1992). Although the telegraph and the first cell phone are separated by years of technological breakthroughs and social changes, it’s significant to point out that we have had the ability to text in one form or another since the 19th century.

Screen Shot 2016-09-13 at 10.19.04 AM
Source: http://graphics8.nytimes.com/images/2013/03/17/magazine/17wmt/17wmt-articleLarge-v2

Although it may be bit of a leap (hopefully I’m not too off base), we can apply the past/present paradox that Renfrew discusses in “Mind and Matter: Cognitive Archaeology and External Symbolic Storage,” in some ways to this example. He asserts that modern representations of the past, like current hunter-gatherer societies, complicate “temporal” sequence (Renfrew, 4). In other words, technological breakthroughs are not always chronological in nature. And, we don’t always stop using a tool once a newer one has been invented.

In this way, the evolution of texting is more so “dependent upon matters of cultural context,” rather than a timeline (Renfrew, 4). Why and how we text to communicate today differs from what motivated us to use the telegraph. We never lost the ability to use the telegraph or the desire to communicate in symbols with each other over large distances, but those needs evolved to reflect advancements in modern American culture.

Even though the telegraph was the first representation of texting as a means of communication, our current texting capabilities “should not be regarded as living representatives” of the past (Renfrew, 4). Texting as we understand it today is not a direct representation of the telegraph, but we should acknowledge its foundational importance in the context of the history of communication.

To better understand the connection between past, present and future texting communication technologies, I want to focus on this quote: “Every culture has a “network architecture” that directs the flow of knowledge among individuals, institutions, and external memory devices” (Donald, 219). Technology is one of the ways that we can direct the flow of knowledge (i.e., the World Wide Web). The devices we have access to allow us to “engage many minds and their cognitive activities” through the act of “interlinking those minds…into large functioning networks” (Donald, 219). We are linked through the use of technological devices, like a cell phone, which bring us together both spatially and as a community to form a network.

References

Brustein, Joshua. (2015). “The Story Behind the First Cell Phone Call Ever Made.” Bloomberg.com, April 24. http://www.bloomberg.com/news/articles/2015-04-24/the-story-behind-the-first-cell-phone-call-ever-made.

Donald, Merlin (2007). “Evolutionary Origins of the Social Brain,” from Social Brain Matters: Stances on the Neurobiology of Social Cognition, ed. Oscar Vilarroya, et al. Amsterdam: Rodophi.

Irvine, Martin (2016). A Samuel Morse Dossier: Morse to the Macintosh, Demonstration of the Morse Telegraph: Electric Circuits and “A System of Signs.” Communication, Culture and Technology Program, Georgetown University.

Kelly, Heather. (2016). “OMG, the Text Message Turns 20. But Has SMS Peaked? – CNN.com.” CNN. Accessed September 13. http://www.cnn.com/2012/12/03/tech/mobile/sms-text-message-20/index.html.

Renfrew, Colin (1998). “Mind and Matter: Cognitive Archaeology and External Symbolic Storage,” in idem and Christopher Scarre, Cognition and Material Culture: The Archaeology of Symbolic Storage, McDonald Institute Monographs (Cambridge, England: McDonald Institute for Archaeological Research) 1-6.

Wong, Kate (2005). “The Morning of the Modern Mind.” From Scientific American. 292, no. 6: 86-95.

Katie Oberkircher, Week 2: Limited vs. Unlimited

As I read through this week’s material, I noticed a tension between two ideas involving signs and symbols. This tension arose among the themes of time, community, memory and culture. The first idea is laid out toward the beginning of “The Grammar of Meaning Making: Sign Systems, Symbolic Cognition, and Semiotics,” and focuses on the inherent social nature of signs and symbols.

Idea 1: The way that we automatically process and transmit thoughts in our communities is unlimited due to the fact that “we make mental relations between perceptions and thought and generate further relations among thoughts connected in vast networks of collectively understood signs spanning many states of time” (Irvine, 4). Our understanding of perception and thought is always generating further relations, creating a never-ending cycle. By transcending time, signs act as a connector between past, present and future generations, propelling society forward (what is defined in the reading as “progress”) (Key Writings, 13).

Idea 2: However, I felt there was a tension between the notion of unlimited thought and the fact that, “ideas cannot be communicated at all except through their physical effects” (Irvine, 3). This brought up a few questions: are the “physical effects” of an idea all that we need to understand it? Are there barriers keeping us from recognizing other non-physical effects that we cannot communicate? How important are those effects?

In his work, De Saussure explores some of these ideas. He explains that if we assume that language is a naming process only, we infer that, “ready-made ideas exist before words” (Key Writings, 9). In response to his theory, I want to focus on the quote, “We think only in signs” (Key Writings, 20). I believe this means that we think in a way to translate our ideas to others.

Further, Dr. Irvine touches on this idea when he writes, “human culture, social relations, and technologies are thus inseparable from our interrelated cluster of symbolic systems…” (Irvine, 2). The word “inseparable” suggests that without one, we could not have the other. So, I understood that to mean, human culture, social relations and technologies are only possible because of the way we communicate with one another through signs and symbols.

That idea was complicated by the quote, “The bond between the signifier and the signified is arbitrary” (Key Writings, 10). If the bond is arbitrary, like the idea of the baseball and its lack of connection to the word “baseball,” then how is it possible that we can fully understand what the signified is? (Irvine, 11) Do we always identify objects/concepts through the context of our social community?

I believe these questions are answered by the way that the themes of community and time are woven into the readings. In “Semiotics, Symbolic Cognition, and Technology Key Writings,” it says, “symbols, or general signs, [have] become associated with their meanings by usage” (Key Writings, 18). In other words, communities use, and reuse, symbols and signs until they become familiar. In connection with that idea, a concept on page 5 of “The Grammar of Meaning Making,” stuck out to me: technologies are “stored value” memory systems (Irvine, 5). These systems are the foundation of the way we communicate and generate new thoughts—strengthened by the notion of collective memory.

As we move forward this semester, I hope to better understand the role of culture and community as they relate to semiotics. And further, how the role of technology has altered the “progress” we continue to make.

References

Irvine, Martin (2016). The Grammar of Meaning Making: Sign Systems, Symbolic Cognition, and Semiotics. Communication, Culture & Technology Program, Georgetown University.

Irvine, Martin (2016). Semiotics, Symbolic Cognition, and Technology Key Writings. Compiled and edited with commentary by Martin Irvine. Communication, Culture & Technology Program, Georgetown University.