Author Archives: Wenxi Zhang

The Tokenization of Advertising Instances (Final Project)

Wenxi Zhang


As one of the most important and long-lasting components among branding strategy, advertising, incorporating symbolic cognition at multiple levels and is mediated in various forms and substrates, operates in a complicated socio-technical system.

The purpose of this paper is to show that the contemporary advertising follows the Peircean conceptual framework of type-token where the tokenization (i.e., creating perceptible representation), though being medium specific and variant, happens always under the umbrella of an invariant type (i.e., the theme of the advertisement which the advertisers intend to convey and correlate to the brand). By applying the conceptual framework of C.S. Pierce, Roland Barthes, Mikhail Bakhtin, MacRury Iain and Lev Manovich, I explain how the tokenization instantiated in terms of the type is being remediated based on different physical mediums and the cultural encyclopedia. I use Coca-Cola company’s advertising strategy under the campaign “Taste the Feeling” since 2016 as a case study, by analyzing how it presents distinctly on various physical mediums, how it incorporates the spirit of Rio Olympic into the brand and how it specifies the structure of the advertisement in terms of different countries or regions (i.e., in this paper, China and North America).

 1. Introduction:

By using Coca-Cola company’s advertising strategy since it launched “Taste the Feeling” as its campaign and theme in 2016 as a case study, I explore and analyze how advertisements as tokens based on the type, i.e., Taste the Feeling is remediated via multiple physical mediums (e.g., television, outdoor, new media online) and by other movements or events (e.g., Olympic) and how the type according to which they are instantiated is invariant and re-enforced during the processes. Further, since audiences as semiotic agents have different cultural memories and knowledge which would affect their interpretation of semiotic representations, I would discuss how the influence of cultural encyclopedia as a hierarchal tree structure is embedded in the advertising processes.

Therefore, the main thesis of this essay is that while in advertising, the tokenization of advertisement can be re-tokened, re-instanced and re-mediated across different mediums (not just physical mediums but also every agency that mediate the tokenization), the type, or general genre of advertising (i.e., the theme per se), is invariant and re-enforced by the unlimited instantiations of tokenization.

In the first part of the paper, I select three advertisements on three different physical mediums, i.e., outdoor street billboard, television and online software application, I use C.S. Pierce’s triadic model of cognitive correlations and his conceptual framework of the three modes of signs to analyze how audiences select and perceive these features and conduct multi-level interpretations. Meanwhile, I also apply Mikhail Bakhtin’s idea of dialogism where the interpretation of message receiver is always involved in one’s dialogism with others, in this case, i.e., the physical context while perceiving the advertisement. I also combine Lev Manovich’s idea about new media’s being interactive and how such interactivity is involved in the generation of meaning as a ongoing dynamic event.

In the second part of the paper, I use the Rio Olympic campaign in 2016 as a context, by studying how different Olympic specified advertisements launched differently in China and North America, I combine C.S. Pierce’s conceptual framework and Roland Barthes’ legacy which still apply well in contemporary media and analyze how interpretation across multi-level is formed under the umbrella of cultural encyclopedia and how the spirit of Rio Olympic serves as an intertextual discourse which involves into the dialogism between itself and “Taste the Feeling” as the theme of Coca-Cola and how the advertisements combine these concepts together. I show that operating under the wave of Rio-Olympic, the theme “Taste of Feeling” (i.e., the type) is still invariant no matter how the tokenization is customized.

2. The tokenization across different media.

Coca-Cola starts a new round designing and distributing advertisements upon launching the new campaign “Taste the Feeling” in 2016. It maximizes the utilization of different mediums including traditional media such as television, cinema, outdoor and new media based on computing and internet.

“Coca-Cola is one brand with different variants, all of which share the same values and visual iconography. People want their Coca-Cola in different ways, but whichever one they want, they want a Coca-Cola brand with great taste and refreshment.” Says de Quinto, the Chief Marketing Officer of Coca-Cola (Coca-Cola Announces New ‘One Brand’ Marketing Strategy and Global Campaign, 2016). “Taste the Feeling”, therefore, first relates the audiences to the feeling of the moment when they drink the Coca-Cola, i.e., delicious and refreshing, which thus requires the advertisers to accurately simulate such user-experience of the product itself. Besides, feeling itself could be interpreted differently in terms of different people and different scenes, in other words, different media presentations are scene specific and the tokenization, i.e., advertisement instantiated based on the concept “Taste the Feeling”, should also be designed in terms of different platforms. In the following subsections, I select a TV commercial, an outdoor street billboard and a customization-supported advertising app for analysis. As Irvine mentioned, meaning is always Remix+, the emergence of meaning always goes through the combinational and dialogic process where we select syntactically possible units in contexts of prior symbolic relations and encyclopedia values and re-contextualize the selected unit by embedding it in the compositional structure of the new expression, in other words, we are not only interpreting the advertisement itself, but also interacting with the context surrounding us where every element of the context could affect our interpretation of the token (Irvine, 2014, pp.21,29). However, no matter how the structure of the tokens changes depending on different situations, the core concept, i.e., “Taste the Feeling” is invariant.

I design a diagram to illustrate how the representation of the concept, i.e., in the figure, the object (invariant) is re-mediated depending on different physical mediums and how humans as semiotic agents’ interpretation of the perceptible representation of the object is influenced by the physical context.

Figure 1. Tokenization being medium specific

2.1 Street Billboard in front of commercial street

– Taste the Feeling of pursuing fashion and being relaxed through shopping.

Figure 2. The Coca-Cola Festival Bottle. (n.d.).

Above is an advertisement presented on a street billboard on a commercial street. At the top right corner of it are the logo of Coca-Cola and the “Taste the Feeling” slogan. The textualization of this advertisement is realized by using a picture of a beautiful girl in blonde hair holding a bottle of Coca-Cola, she is in a stylish fit and her nails are decently polished, she slightly turns her head left and laughs happily. Being presented on a commercial street, this advertisement is a mixture of iconic sign and symbolic sign. Since one of the designing ideas that advertisers follow to appeal to their consumers is by depicting people in advertisement like those consumers, or resembling the ideal that consumers aspired towards, the girl in this advertisement is an average ideal type of a lot of female potential consumers who the advertisers assume to be more likely the group of people shopping or hanging out at the commercial street (MacRury, 2008, p.173). The sign, on the other hand, is symbolic where by perceiving the dress style of the girl, consumer can easily connotate to beauty and healthiness (by her bronze skin and well-defined body shape). The surrounding environment contributes to the interpretation as well where the shopping malls, fashionable clothes and make-up suit all represent consumer’s pursuit of beauty and the whole experience of shopping itself represents a kind of feeling where the customers can leave stress behind them for an afternoon, enjoying shopping and pursuing beauty. Thus, by simulating users’ feeling of taking a break while shopping and enjoying their beauty, the advertisers thus instantiates a specific scene-based token which still represent the core concept, i.e., Taste the feeling (the feeling of being beauty, fashion, modern, healthy, relaxed, etc.).

2.2 Television commercial

– less limitation on time and more space for representing multiple kinds of feelings.

As another kind of traditional media, television, nevertheless, presents advertisement differently compared to street billboards (to be sure, the street billboards I mention here are the traditional street billboard instead of the fancy versions on, for example, Times Square) which presents visually static textualization. TV commercials, instead, are mostly motion picture sequenced featured by sound effect or music (most of the time theme song), while TV commercial have more space to represent the core concept compared to static traditional outdoor commercials, what is more challenging is how to make the frames well-organized and to match the image, sound, text simultaneously and at the same time to make sure the concept is well conveyed. In this case, the structure of the token, i.e., the advertisement presented on Television as media, is remediated.

This is a screen shot of a one-minute TV commercial that Coca-Cola published on March 2016 (Taste the Feeling, 2016). At the same time, Coca-Cola adds a line for explanation to emphasize what the commercial is trying to convey: there’s a coke for every feeling. The advertisement uses a sequence of scenes each of which is fitted with a text following a general mode: A with B, and in the next scene the B in the previous scene would become A’ where a B’ is generated matching with the visual motion image (e.g. Strangers with fire à fire with Coca-Cola).

The TV commercial, to some extent, is an integration of multiple street billboards where feelings of different people under different scenes are all simulated and put together. Such A with B model is a great example of a symbolic chain since according to C.S. Pierce, signs yield interpretants expressible in further signs in unlimited and open-ended chains or networks, this model utilize the time flexibility in TV commercial and is therefore able to convey as much tokens of feelings as time allows (Irvine, n.d., p.19). Besides, while advertisement texts are constructed with visual cues that imply an endless chain of meaning from which the viewer can choose some and ignore others, audiences’ interpretation of each scene is therefore guided by the textual narratives (Danesi, 2013, p.473).

The sound effects are also elements in the structure of the token (TV commercial) that are remediated by television as a medium. At the beginning of the commercial, the scene being displayed is the whole process of pouring Coca-Cola into a bottle containing ice featured with the sound effect of liquid falling onto the glass and pieces of ice crushing on each other. The whole scene could be explained as a reproduction of the prototype, i.e., customers conducting the whole process (i.e., pouring the Coke in reality) and such structure which enables the combination of sound and motion image sequence, is not feasible on other medias such as street billboard or traditional print media. However, what is still unchanged is the concept “Taste the Feeling”: each scene of the sequence in the commercial (e.g., hanging out with friends, having a crush on a stranger, playing games with family, etc.), no matter how authentic it is simulating specific feelings of customers in different cases, still goes back to the core concept where customers can taste, experience the feeling that Coca-Cola brings them.

2.3 interactive advertising app

– Taste the feeling of customizing your meta-feeling

“Taste the Feeling” launches at a time when computing and internet-based technology is growingly becoming matured. New media, as a convergence of two separate historical trajectories: computing and media technologies, thus is a necessary interactive platform to confront contemporary potential customers (lev Manovich, 2001, p.20). Unlike traditional media where the order of presentation is fixed, the user can now interact with a media object (lev Manovich, 2001, p.49). Thus, the tokenization of the feeling that need to be instantiated could be remediated and redefined through interactive mediums.

Figure 3. Yi Se Lie Ke Kou Ke Le Hu Dong Guang Gao Pai-Zi Ding Yi Hu Wai Guang Gao (2013).

Above is a real time customizable advertising app launched in Israel. User can imagine whoever or whatever that fits in the blank in the sentence “Share a coke with __”. When customers walk closer to the screen, the customized advertisement will thus be displayed.

This advertisement on new media thus provides multiple tokens of feeling that customers could taste and relate to the taste of Coca-Cola. The literal message conveyed from the sentence thus provides a hint for customers to experience the feeling of sharing coke with friends, lovers, family or strangers. The textualization incorporates the purpose of advertisement, i.e., attracting the customers, guiding them to generate positive attitude towards the product, turning such attitude to purchasing behavior, stimulating repeatable behavior and eventually letting them to persuade other surrounding people to purchase as well. By correlate Coca-Cola to the feeling of sharing, therefore, the advertisers embed such correlation into customers’ memory.

New media, by being interactive, remediates the tokenization of “Taste the Feeling”. Instead of guiding user’s interpretation by fixed presentation, it puts user into a dialogism with the advertisement where meaning, as an event, is mutually stimulated. Thus, customers can also experience the feeling of self-defining and producing the feeling, in other words, the meta-feeling.

3. Taste the feeling under the umbrella of cultural encyclopedia

– Coke incorporating all kinds of feeling in Olympic

Advertisement is one of the most important ways to transfer a product into a brand. Earlier in the 20th century, Roland Barthes has brought the idea of myth onto table where he argued that myth is a type of speech defined by its intention much more than by its literal sense (Barthes, 2006, p.265). While in a language, the sign is arbitrary and unmotivated, the mythical signification, on the other hand, is never arbitrary, it is always in part motivated, and unavoidably contains some analogy (Barthes, 2006, p.266). He pointed out the idea of literal message, i.e., the product of denotation and cultural message, i.e., the connotation in his work rhetoric of the image where by such intention of the information transmitter, the reader, or the receiver, could choose some and ignore others among all the perceptible information (Barthes, n.d., p.516).

Barthes’s idea is widely used in multiple field including politics, films, advertisement, etc. at that time and in contemporary media as well. To convey the idea of a brand, the negotiation between brand maker and brand user is crucial, a main purpose of such brand communication, according to Thellefsen, is creating symmetry (Thellefsen et al., 2013, p.486).

The following diagram shows the structure to be followed under the purpose of creating symmetry.

Figure 4. Creating Symmetry between brand maker and brand user. (Thellefsen et al., 2013, p.486).

The intersection node of the triadic model indicates a shared cultural background, or memory between the target community and the brand maker’s intention and expectation (Thellefsen et al., 2013, p.486). As the context and background knowledge is crucial here, Barthes’ earlier idea which mostly utilized Saussure’s model (i.e., the signified and signifier) is better to be incorporated into C.S. Pierce’s model where he brought the third element, i.e., the interpretant which is the sense made of the sign and the acknowledgement that correlates the representamen and object. Pierce’s model provides a dynamic continuum through the meaning generation which is operated under the cultural encyclopedia, i.e., a system of culturally organized meanings and values, codes, genres and symbolic association (Irvine, 2014, pp.20,25).

In the following paragraphs, I would use Coca-Cola’s advertising strategy during Rio Olympic in 2016 as a case study where I would focus on two points: first, by comparing the difference of the structure of the advertisement such as the usage of image, the extra textual narratives as tokens in America and China, I analyze how the advertisers expect and involve the advertisement into the interpretation of target community across multi-levels, i.e., the immediate, dynamic and final interpretants generation (Irvine, n.d., p.1); Second, I demonstrate that the discourse formed around the spirit of Olympic becomes a intertextual one, itself, together with the theme of Coca-Cola since 2016, i.e., Taste the Feeling, generates a dialogism which deeply influence the instantiation of the tokens in terms of the theme and how the spirit of Olympic remediates the tokenization representing the theme “Taste the Feeling” and how the theme is still invariant though being represented by different tokens during this special time period. In terms of the analysis of customers’ interpretation processes, Dr. Irvine’s diagram for the outline of semiotic models for kinds and levels of interpreted Meanings is a great model to apply (Irvine, n.d., p.1).

Figure 5. Peirce’s model of material-cognitive correlations for semiotic substrates. (Irvine, n.d., p.1).

3.1 “此刻是金”, Chinese advertisement during Rio Olympic, 2016.

– Coca-Cola China embracing Chinese audiences’ feeling of Rio Olympic

“此刻是金”, namely, gold is this moment, is the slogan defined during Rio Olympic, 2016 for the marketing strategy among China. The literal message that gold convey, in this case, is gold medal and success which are provided by the context under the Olympic period. “this moment”, on the other hand, relates customers to every moment in their daily lives. By correlating “gold” to “this moment”, the advertisers thus start with the moment of success in everyday lives that people can easily relate themselves to and brings the spirit of Olympic to customers no matter how ordinary they think they are (besides, the advertisers also intend to convey that no matter you are celebrities or ordinary people, you always have something in common, i.e., the ordinary but meaningful and special moment).

Considering how the advertisement could stimulate the symmetry through the negotiation with customers, it is important to understand what it is that matters to Chinese audiences. While in Chinese culture, family and collectivism are two of the most important ideologies. Such cultural knowledge where staying close to your family, friends, colleagues, etc., helping each other, valuing corporation and being supportive and caring are considered ethical is embedded in everyone’s cultural memory since they are educated so from the first day when they went to school. Therefore, relating those successful but ordinary moment to family and friends is an easy way to stimulate the emotional symmetry among the Chinese audiences.

Figure 6. Ke Kou Ke le Sheng Wen Ci Ke Shi Jin. (2016).

The picture is a series surrounding the theme “Gold is this moment” where each one is featured by three scenarios and a text illustration. In the first section, the instances that we can immediately perceive are the logo of Coca-Cola, and the Olympic logo which displays five parallel lines each of which has one of the color of the Olympic rings, the major image where a girl is smiling while hugging a man, behind whom is a man who is clapping his hands, the textual narrative which says “gold is the tears of your proud family” and two other images the first of which is a man holding tightly some stuffs and looking at them and the second of which is two women hugging each other head to head. From these physical substrates we are thus able to form our first level symbolic recognition (according to Dr. Irvine, the recognitions generated at this level are the immediate interpretants): this is an advertisement of Coca-Cola and it is related to Olympic, the one that is hugging with the girl is her family and the one behind her clapping his hands is someone that is congratulating her success, the two other images could be relate to the same pattern with the help of the textual narrative, i.e., the success of someone and the proud family behind the person (Irvine, n.d., p.4). The output of our first code correlation selections for recognition then enables us to move on to the next level of interpretation, which, if we relate back to Barthes’s idea, is the process of connotation which generates the cultural message where we select additional code correlations to larger genre and meaning categories, the cultural and social values we live in and experience everyday that are deeply embedded in our cultural memory thus enables us to understand what the advertisers intended to convey and what’s the ideology behind and how it relates to the brand per se (Barthes, n.d., p.155; Irvine, n.d., p.4). While China is highly influenced by the Confucian culture where family is a crucial part which treasures harmony and morality, the three scenarios thus relate many Chinese people to their own memories when their parents get up early every morning cooking them breakfast, take them to school, interest-oriented classes, auditions, competitions, etc. or stay up late with them when they are preparing for big exams. The daily narratives such as no matter who you are, how bitter your life tastes, how struggle you are pursuing your own dream, the family is always behind you, supporting you and never leave you and your success or failure is never just belonging to yourself are also very common in China and are considered as “小确幸” (i.e., ordinary but real happiness) for Chinese people. Thus, Chinese people can interpret the advertisement in such way that the spirit of Olympic is not just success itself, but also the family behind you and such interpretation can easily invoke their emotion symmetry and thus relate the value of Coca-Cola as a brand to such cultural ideology.

In a similar way, the second section, with the textual narratives “gold is the encouragement behind success” and the major image where the pianist Lang Lang is practicing piano with his tutor behind. The advertiser, in this section, intends to convey the cultural ideologies that a good student need to honor and respect one’s teacher and a good teacher needs to treat one’s students as if they are one’s children. Coca-Cola, in this case, extend the meaning of gold into virtue and goodness and relates the brand onto it.

The third textual narratives, i.e., gold is the gesture of cheering up among brothers, (to be sure, brothers don’t necessarily mean siblings in China, it could also refer to good friends, colleagues, etc.), it provides hint to guide the readers’ interpretation of the three images where in the major image, the guy behind the camera gives a thumbs-up gesture to his brother. In the second picture, there are co-workers gathering together celebrating their project and in the third picture, there is a guy (to be sure, we can’t tell directly from the image but from the narratives we know that this is a male instead of a female) in his helmet riding a motor. This section is a contemporary token of the collectivism as a cultural ideology where people value the virtue of working or simply hanging out in a team.

These sections use both celebrities and ordinary people as the characters where both the successful moments of ordinary people and the ordinary moment of successful people are shown in the series. This thus refer to another cultural ideology or virtue in China where people believe and treasure the ordinary people for being hardworking and successful in their own fields. By defining the intersection between the connotation of Olympic to success and every customer’s own moment of working hard to achieve their own success, Coca-Cola therefore establishes a firm bond among Chinese audience during the Rio Olympic period.

3.2 “THATSGOLD”, the advertisement launched in North America during Rio Olympic 2016.

– Taste the unbeatable feeling of Rio in North America

Figure 7. Coca-Cola North America athletes Ashton Eaton and Alex Morgan feature in the #ThatsGold campaign. (2016).

Figure 8. How Coca-Cola Is Activating Its #ThatsGold Campaign for Rio 2016. (2016).

Above are two advertisements launched across America during Rio Olympic 2016. The two advertisements follow similar pattern structuring the advertisement where in there is a logo of Coca-Cola at the left-top corner, five parallel lines featuring the five colors of Olympic rings, the slogan THATSGOLD and a textual narratives at the right-bottom corner and the image of athletes on different scenarios (there is still a slight difference where the first advertisement adds the overall theme “Taste the Feeling” below the logo but since all those advertisement during the Olympic period are designed under the umbrella, this slight difference can thus ignored in the following explanation).

The first advertisement offers us the immediate physical instances including two athletic athletes (i.e., Alex Morgan, a key member of the U.S. Women’s National Team that won the 2015 FIFA Women’s World Cup and Ashton Eaton, a gold medalist in men’s decathlon at the London 2012 Olympic Games) with big smiles holding Coca-Cola in glass bottles, bright sunny day, grass and the Olympic rings behind them (Coca-Cola Goes for Gold in Rio 2016 Olympic Games with Global #ThatsGold Campaign, 2016). The textual narrative, i.e., unbeatable taste is a pun which refers to both the taste of being unbeatable in the competition as a champion and the unique, best taste of Coca-Cola as a beverage as well.

The image of the second advertisement selects a swimming pool as a background where Nathan Adrian, the “Fastest Man in the Pool” who amassed three Olympic gold medals and seven World Championship golds since 2008, holding a glass bottle of Coca-Cola, is celebrating with another swimmer after finishing a training or competition, sending up pearly spray (Coca-Cola Goes for Gold in Rio 2016 Olympic Games with Global #ThatsGold Campaign, 2016). The textual narratives “A refreshing finish” refers both to the feeling of finishing a round of swimming and to the feeling of refreshment by drinking Coca-Cola.

Unlike the Chinese advertising series where the advertisers select both celebrities (i.e., Lang Lang, Yang Sun, Ting Zhu) and ordinary people as the characters, the campaign launched in North America incorporates an elite group of five Team USA athletes and hopefuls and an Olympic legend who have won a total of more than 20 Olympic medals, including nine gold: Alex Morgan, Ashton Eaton, Tatyana McFadden, Nathan Adrian, Leo Manzano and Nastia Liukin (Coca-Cola Goes for Gold in Rio 2016 Olympic Games with Global #ThatsGold Campaign, 2016). Since these athletes are well-known in America, American audiences can thus immediately select these characters as features and refer to energy, refreshment, championship at the first level as literal message and then generate connotation which refers to patriotism and nationalism as cultural message.

The following is a table comparing the advertisement series between North American version and Chinese version in terms of Pierce’s model of material-cognitive correlations for semiotic substrates. (To be sure, I explained the previous cases with the concept denotation and connotation which are based on Saussure’s static sign model just for corresponding to some resources I refer to, but I’m still applying Peirce’s triadic model to them).

First level correlation Second level correlation
R (only visual elements are analyzed due to space limitation, but textual narratives are also crucial) O I R’ O’ I’ (The next level would correlate the I’ to Coca-Cola)




Family affection, special moments of ordinary people  




Familism, collectivism Chinese culture
North America Athletes,


Swimming pool

Energy, brightness, refreshment,




American culture


Table 1: Comparison between Chinese version and North American version.

3.3 Taste the Feeling of Olympic: make the invariant theme special under the Olympic campaign

How do advertisers enable people to relate Coca-Cola immediately to “Taste the Feeling”? According to Lencastre, the brand makers turn a product into a brand by adding augmented identity (e.g., slogans, labels, mascots, iconic signs, etc.) beyond the core identity and actual identity of the product per se (Lencastre, 2013, pp.493-494). Advertisement, thus follows the Peircian principles of firstness, secondness and thirdness where the thirdness is the repeated interpretation in space and time of the relationship between the immediate stimuli and the objects (a process of recognition, short-term memory to long-term memory), in other words, to immediately correspond the product to the brand and the brand to its value profile by such repetition (Thellefsen et al., 2013, p.565). By branding and advertising, the brand makers and advertisers thus want to create an emotional filter for the product which is intended to create an emotional state in the brand user mirrored in the product, so that product become a cognitive-symbolic habit of interpretation (Thellefsen et al., 2013, p.563). Therefore, for Coca-Cola as product since 2016, the value profile that Coca-Cola company wants to build on it would be every kind of feeling customers could imagine presenting on the slogan “Taste the Feeling”. The propaganda team thus emphasize that every moment for everyone could be made special by drinking Coca-Cola. In this case, certain periods, spaces, events, movements, etc., all become parts of the “every moment for everyone”. “Taste the feeling”, according to Peirce, is the invariant type that could be instantiated infinitely based on different scenarios. The Rio Olympic campaign, is thus a setting of one of the tokens of the type where the spirit of Olympic itself serves as a medium that could remediate audiences’ interpretation of the feeling of tasting Coca-Cola.

Figure 9. The tokenization of “Taste the Feeling” under Rio Olympic 2016 campaign.

Above is my explanation of how advertisers instantiate “Taste the Feeling” as a type under Rio Olympic 2016 as a context for generating tokens and relate the value to Coca-Cola both as a product and as a brand. Taking Coca-Cola and Olympic as two objects to analyze, the advertisers would stand on the potential customers’ viewpoint to generate multi-level interpretations based on these two objects and the concepts extended from them, i.e., the series of signs from “Taste the Feeling” of Coca-Cola and the spirit of Olympic of Rio 2016. While the customers retrieve their memories for interpretation which relies on the social-cultural context they live, advertisers need to define socio-cultural specific values as symbols that may stimulate the emotion symmetry among customers and simulate their feelings as prototypes, thus based on different countries or regions (here the example is China and North America), advertisers generate different elements that would relate audiences to their context specific interpretation of the Olympic spirit and feelings (to be sure, this does not happen purely sequentially, different levels of signs of Coca-Cola on the left and Olympic on the right actually form a network where each node can be a source of otherness that involves intertextually into the dialogism and affect the reader’s perception and interpretation of other nodes simultaneously and dynamically). Finally, among those signs, advertisers thus are able to find the intersection by which they can generate new augmented identity around Coca-Cola under the umbrella of the Olympic campaign.

4. Conclusion:

Whenever you drink a bottle of Coca-Cola, you are never merely drinking, or tasting the coke itself. Every single part of the whole propaganda chain behind that is highly symbolic.

I demonstrate in the Coca-Cola example that the invariant type in the advertisement can be instantiated infinitely depending on different mediums (to be sure, the mediums here include not only physical mediums such as television, print, but also anything that can mediate and remediate the tokenization including different scenarios, movements, historical-socio-cultural context, etc.).

The first part of the paper demonstrate that different physical mediums remediate the tokenization in terms of an invariant type, In the example of Coca-Cola, the invariant type is “Taste the Feeling” which can be instantiated infinitely. I explained that the advertisers simulate the feeling that customers would experience medium specifically and realize the theme “Taste the Feeling” by various means and platforms. However, no matter how the structures of the tokenization are remediated depending on various physical mediums, the intention to correlate customers the feeling drinking Coca-Cola to every special moment they experience is not changed, but even repeated and re-emphasized.

I use Rio Olympic in 2016 in the second part of the paper as a context which remediates the tokenization of the advertisement to incorporate the spirit of Olympic. By applying C.S. Pierce’s triadic model which could be further expanded to dynamic and multi-level interpretation, I consider cultural encyclopedia as an umbrella that is not ignorable in customers’ interpretation of the advertisement. I compare the advertisement launched in North America and China by analyzing how cultural value and memory are imbued into different versions of advertisement under Olympic campaign. I finally concluded that cultural context and specific campaign are also mediums that remediate, retokenize and re-instantiates the tokenization of the theme, however, “Taste the Feeling” is always the type that is never changed no matter how the advertisements vary.

Advertising, of course, is just one element on the whole brand wheel, i.e., from the name, logo, slogan-jingle, to the textualization which put the real advertisement onto the table, and finally the design of the package of the product (Danesi, 2013, p.465). And the design of every “department” on this brand wheel, can be considered as the tokenization which could be instantiated without limitation following the general rule of the type. One of the limitation in this paper is that we don’t have enough space to talk about the tokenization in the whole brand wheel. Besides, I only use space (i.e., China and North America) as the variance to talk about how multi-level interpreted meanings are generated under the culture encyclopedia, but since the cultural context not only various across regions but also changes dynamically through time, to explore how such tokenization along the time line would make the research more comprehensive.

Further, the translation of slogan in different countries can also be explored deeply in the future research. From the phonology, morphonology, to the management of lexicon and syntax, to semantics and pragmatic restricted by cultural context (e.g. different idiom works differently depending on the country or region), etc., are worth exploring in the future research.

Work cited

  1. Bolter, J.D & Grusin, R. Remediation: Understanding New Media. Cambridge, MA: The MIT Press, 2000.
  2. Barthes, R. (2006). Myth today. Cultural theory and popular cultural: a reader, 3, 293-302.
  3. Barthes, R. (n.d.). Rhetoric of the Image. Retrieved 2018, May 2, from
  4. C. (2016, March 31). Taste the Feeling – Sri Lanka (English). Retrieved from
  5. Chandler, D. (2017). Semiotics: The Basics (2nd ed.). New York, NY: Routledge.
  6. Coca-Cola Goes for Gold in Rio 2016 Olympic Games with Global #ThatsGold Campaign. (2016, July 13). Retrieved from
  7. Coca-Cola Announces New ‘One Brand’ Marketing Strategy and Global Campaign. (2016, January 19). Retrieved from
  8. Danesi, M. (2013, September). Semiotizing a product into a brand. Social Semiotics,23(4), 464-476. doi:10.1080/10350330.2013.799003
  9. The Coca-Cola Festival Bottle. (n.d.). Retrieved from
  10. Yi Se Lie Ke Kou Ke Le Hu Dong Guang Gao Pai-Zi Ding Yi Hu Wai Guang Gao (2013). Retrieved from
  11. Ke Kou Ke Le Sheng Wen “Ci Ke Shi Jin”, Ao Yun Jing Shen Yi Chu Ji Fa. (2017, July 9). Retrieved 2018, May 2, from
  12. How Coca-Cola Is Activating Its #ThatsGold Campaign for Rio 2016. (2016, August 22). Retrieved from
  13. Irvine, M. (2018). Introduction to Signs, Symbolic Cognition and Semiotics. Retrieved 2018, May 2, from
  14. Irvine, M. (2018). A Student’s Guide to Mikhail Bakhtin: Key Terms and Main Theories. Retrieved 2018, May 2, from
  15. Irvine, M. (2014). Remix and the Dialogic Engine of Culture: A Model for Generative Combinatoriality. The Routledge Companion to Remix Studies (E. Navas et al, Eds). New York, NY: Routledge, 15-42.
  16. Lencastre, P. D., & Côrte-Real, A. (2013, September). Brand response analysis: A Peircean semiotic approach. Social Semiotics,23(4), 489-506. doi:10.1080/10350330.2013.799005
  17. MacRury, I. (2008). Advertising. New York, NY: Routledge.
  18. Manovich, L. (2001). The language of new media. Cambridge, MA: MIT Press.
  19. MacRury, I. (2008). Advertising. New York, NY: Routledge.
  20. Thellefsen, T., & Sørensen, B. (2013). Negotiating the meaning of brands. Social Semiotics,23(4), 477-488. doi:10.1080/10350330.2013.799004
  21. Thellefsen, T., Sørensen, B., & Danesi, M. (2013). A note on cognitive branding and the value profile. Social Semiotics,23(4), 561-569. doi:10.1080/10350330.2013.799010

draft proposal ongoing — Wency

draft & ongoing proposal — Wency

Research questions:

  1. The development of the content of advertisement through timeline. How does it imply the perception and interpretation of the content from different communities through different time and space? Study the different level of semiotic and meaning generation from perception, feature extraction, pattern recognition, pattern mappings: token into type, syntactic combination, semantic frames to cultural encyclopedia and dialog.

Analyze objects and possible keywords:

  • Peirce’s triangle: object, representamen, interpretation;
  • Three modes of signs: symbolic, iconic, indexical;
  • Meaning is not something stable, it’s an event, it’s generating through dialogic and is affected by macro cultural encyclopedia through time.
  • Barthes’s mythology, connotation.
  • Through learning slogan and images/ patterns: type-token, syntax and grammar, semantics.

– From emphasizing the function to personalized and emotional requirement, how does such mythology work?

– How does coca cola combine their concept during certain events (e.g.: Olympic) (How do we correlate the connotation between two objects).

– persuasion: how does sugar + water become different and stand out among other beverages.

– How do these advertisements simulate your perception drinking the product?

– How does the cultural encyclopedia affect people’s perception and interpretation?

EX: the barrier of translation of idiom, ambiguity of language in different country; different spokesperson through different timeline.


  1. Advertisement on different media: e.g.: why Louis Vuitton doesn’t have advertisement on YouTube?


Analyze objects and possible keywords:

  • Reproduction, remediation. Digital media/meta-medium
  • People’s connotation about digital media and mass communicationà aura, prestige, wealth, scarcity.
  • Interface/channel through information exchange, both users and producers become transmitter of information. (QR code)
  • Socio-technical system, the precondition of digital advertising and the socio-economic-political-cultural drive behind it.


– How does digital media being as an interface affect the user and the brand’s interaction?  (enhance? Speed up? More data collection and multiple types?)à compared to traditional marketing.

– semiotics behind brand

– semiotics behind different media platform for presenting the advertisement (accessible? Approachable? Mass perception?) EX: advertisement embedded in dramas, reality shows, films, Youtube Channels, Instagram, search engine and advertisement, etc.

– the perception on the same content on different platforms.

– affordance


Weeks to be explored:

Week 4, 5, 6, 9, 10, 12, 13

  • Language: our modeling system for meaning & symbolic thought
  • The grammar of meaning-making: sign systems & symbolic cognition
  • Applying semiotic methods to media: how meaning systems work
  • Media theory: from medium to mediation
  • From medium and mediation to de-blackboxing socio-technical systems
  • From computing machines to digital media interfaces & metamedia
  • Mediation, representational media & metamedia to the Google art project.

Ongoing bibliography:

  • Steven Pinker, “How Language Works.” Excerpt from: Pinker, The Language Instinct: How the Mind Creates Language. New York, NY: William Morrow & Company, 1994: 83-123.
  • In Bennett, P., & In McDougall, J. (2013). Barthes’ Mythologies today: Readings of contemporary culture.
  • Danesi, Marcel. “Semiotizing a Product into a Brand.” Social Semiotics23, no. 4 (September 2013): 464–76.
  • Lencastre, Paulo de, and Ana Côrte-Real. “Brand Response Analysis: A Peircean Semiotic Approach.” Social Semiotics23, no. 4 (September 2013): 489–506.
  • MacRury, Iain. Advertising. London; New York: Routledge, 2008.
  • Thellefsen, Torkild, and Bent Sørensen. “Negotiating the Meaning of Brands.” Social Semiotics23, no. 4 (September 2013): 477–88.
  • Thellefsen, Torkild, Bent Sørensen, and Marcel Danesi. “A Note on Cognitive Branding and the Value Profile.” Social Semiotics23, no. 4 (September 2013): 561–69.
  • Berger, A. A. (1997). Narratives in popular culture, media, and everyday life. Sage.


Inside and outside the digitalized museum: Thoughts on the Google Art Project- Wency

Inside and outside the digitalized museum: Thoughts on the Google Art Project

The history of the reproduction of the work of art can be traced back to thousand years ago. But it was not until the age of technological reproducibility when mechanical equipment, or better, apparatus that higher quantity of the reproduction was enabled and critiques and debates around reproduction was started. However, people at that age might have never imagined that almost 100 years after the beginning of the era of mechanical reproduction, a new invisible apparatus is bringing a new revolution where the work of art, together with the interface of the work of art, are both being remediated by digitalization (luckily, Benjamin’s idea in the age of mechanical reproduction seems to be still useful under the digital reproduction era). One of the best examples in this case might be the Google Art project.

  1. Inside the museum: from informational to emotional

A common problem with digital reproduction of art works is that people often consider these reproductions as mere reference to the real thing (Proctor, 2011, p.219). The first generation of museums on the web, according to Serota, was concerned of quantity of information and getting as many objects on line as possible (Proctor, 2011, p.215). The purpose of digital reproduction, in the first generation, is to instantiate as much token as possible in terms of the original work, or better, the prototype.

The google art project, to some extent, brings the digital reproduction era onto the second stage where the gigapixel scans by which artworks are rendered into digital data streams are enabling intimate encounters with images at visual depth not possible even in galleries (Proctor, 2011, p.215). The high-resolution images and high-definition video are therefore, guarantees the quality of the art work which is even better than the real thing, in other words, the online interface, under the second generation of museums on the web, serves as a supporter instead of distractor for the artwork (Proctor, 2011, p.219). The director, under this circumstance, is thus ambitious about transform the pure informational digital token into an emotional one (Proctor, 2011, p. 215).

However, while the exhibition itself is a mutual experience where standing at the perspective of audience, their perceptions are not merely determined by the quality of artwork. In terms of the concept of intertextuality and dialogic principle, audiences’ interpretations of the artwork are contextually situated and tied to their own experience visual culture (Irvine, 2018, p.5; Beil, 2013, p.23). Therefore, the Google art project, on the other hand, is benefit from the understanding audience’s own knowledge of the conventions of artwork reproduction which enables them to select the features of an image during the representation and interpretation (Beil, 2013, p.23).

  1. the digitalized museum

The experience of visiting a museum involves more than looking at a painting. For most audiences, wandering in a museum without knowing what kind of artwork they are looking at might not be a good user experience. Therefore, for instance, if I want to see some great work from ancient Africa, the first thing I would do is heading to the African art area and start the whole adventure. But there might also be a problem, for a real museum, for example, if Europe art and postmodernism are two categories in two different areas, what would the director place an art work that belongs to both categories? But in a digitalized museum it seems to be not a problem. While the backstage software allows you to generate infinite tokens with the same degree of quality, such problem is therefore easily manipulatable.

As is seen in the picture, the artwork can be ascribed into multiple categories. It’s also a great example of how the director is organizing the museum using the idea of abstraction, type and token.

Besides, users’ exhibition experience, or the function of museum is also reproduced. The 360-degree Street-view style tour, by simulating the first-person point of view, is therefore actually simulating the whole exhibiting experience (looking at the artwork and also wandering around the museum).

What’s more, from what we learnt last week about digital devices, applications as metamedium, the Google Art project is also a great example of being a metamedium by manipulating, organizing and digitalizing museums as interfaces between audiences and artworks. The meta-painting example shows how a painting can be a painting about paintings and inspires the digitalization to be a more precise way of representing such meta-painting (Irvine, 2018, p.7), and what’s more, by using close-up, user can see every detail of the painting clearly which is unimaginable in a “thingly” real situation.

  1. the socialized & politicalized interface

– cult value to exhibition value: the socio-political factor behind the design of technology for production

According to Benjamin, in the age of mechanical reproduction, more and more audiences are able to be involved into the perception of art. On the one hand, people are arguing that the aura of art is dispelled because of the uniqueness and permanent of the work are destroyed during the endless generation of same tokens (Benjamin, 2013, p.254). On the other hand, within the improvement of the technology of reproduction, the appreciation of work of art thus becomes a mass project where the difference between social class in terms of art perception is at least decreased. While in the earlier age, the value of art is evaluated in terms of cult value where people are told to watch in distance as if they are in a ritual, now the value of work of art itself is being under a transformation where the cult value is turning into the exhibition value (Benjamin, 2013, p.256). Therefore, when we are talking about how technology is defined by reproducibility or even the work of art is designed for reproducibility, there is also socio-political factors lying behind.

Nevertheless, Benjamin also points out how work of art (e.g. paintings, photographs) and film actors are subjected to the technical apparatus, many critiques at that time were also pointed to the performance of the actor where people are doubtful that the actors, instead of live performances, are relying on the editing techniques, besides, while trying to build a fake personality outside of the studio, the actors themselves, are alienated (Benjamin, 2013, p.261). At the present, it seems that the Google art project has not encounter such problem yet, but the software application makes the digital manipulation much faster and easier, one can expect as many post-editions as possible in the future. What, then, would be the future of the art works and artists involved in the museum? What would be the future of the museum as a meta-interface of those art work? That might be some new questions.



  1. Agostino, C. (2015). Distant Presence and Bodily Interfaces: Digital-Beings and Google Art Project. Museological Review – University of Leicester.
  2. Beil, K. (2013). Seeing Syntax: Google Art Project and the Twenty-First-Century Period Eye. 40 (4). Afterimage.
  3. Benjamin, W. (2003). The Work of Art in The Age of Its Technological Reproducibility. Cambridge, Massachusetts: the Belknap press of Harvard university press.
  4. Explore – Google Arts & Culture. (n.d.). Retrieved April 13, 2018, from
  5. Irvine, M. (2018). Malraux and the Musee Imaginaire: (Meta)Mediation, Representation, and Mediating Institutions.
  6. Meta-Painting. (2013, January 05). Retrieved April 13, 2018, from
  7. Proctor, N. (2011). The Google Art Project. Curator: The Museum Journal.


iPhone: a non-stop developing metamedium – Wency

iPhone: a non-stop developing metamedium – Wency

It seems to be something happened just yesterday where people wake up in the morning, carrying an MP3 on their way to work, going all the way to a company 10 miles away from their office for a conference, buying a newspaper or magazine from a newsstand after business, call their friend on telephone and complaining about what happened in the workplace, turning on the TV and crying for missing their favorite show.

People at that time might never have a chance to imagine that less than 20 years later, all the things I mentioned above can be done on a single, portable, fashionable device: the iPhone.

Take myself as an example, I listen to music on Spotify, apple music on my iPhone where I can almost access all kinds of music I can imagine. I have video conference or class on some extremely bad weather where I can enjoy the real-time face to face conversation just like what I have in person. I check newsify, twitter, Facebook and even Instagram everyday to see if there’s something new happened in this world (of course, not only news but a ton of gossips from a lot of celebrities). I snapchat my friend everyday with photo, video and add some words and emojis to draw their attention. I never wait in front of the TV in case I might miss the TV series or reality show because I’m sure that the same version would still be release on, for instance, YouTube, within a few days.

Never would those people imagine that those applications existing today are releasing new version almost non-stop: New functions, new GUI (like one application I’ve been using has just changed the way of displaying users’ profile recently), new interfaces added to connect to other third-party applications and new competitors offering similar but extended or refined functions emerging every day.

The emergence of these new media (digital, or software-based media), thus brings out a new word, i.e., metamedium, while according to Manovich, computer is the first metamedium, all these digital devices including smart phones, PC, iPad, etc., can be defined as metamedium which is a combination of existing new, and yet to be invented media, a media for presenting, transforming other media (Manovich, 2016, p. 335; Irvine, 2014). The fourth mode of interface, i.e., graphic user interface (GUI and the first three are electrical, symbolic and textual), turns today’s digital computer into a remediation machine which continues the bridge connecting the previous media and computational media and further, users can control this graph to display different amount of information (Manovich, 2016, p.58-59, p.73).

The beauty of the idea to use software to implement and realize the function of media is the flexibility and high efficiency: we can add new properties or even invent new types of media by simply changing existing or writing new software, therefore, new media is new because new properties (i.e. new software techniques) can always be easily added to it (Manovich, 2016, p.92). Besides, the definition of medium, in terms of software, is now turned into the sum of algorithm and a data structure (Manovich, 2016, p.207), in other words, all previously existing and newly invented mediums share some common properties, they rely on a set of common software techniques for data management, authoring, and communication (Manovich, 2016, p.123), therefore, the material itself of different media becomes less important, it is rather the universal media machine that makes all seemingly different platform happen easily, just as in the conversation between Apple and Microsoft when they both share a same rich uncle, it is thus the same source which is later reimplemented that makes the magic happen (Manovich, 2016, p.70; Irvine, 2014). Besides, since the manipulation of data, code is much faster than that on the physical materials (although the actual operation of computer is relied on the physical hardware), the update and release of new version of media on the computing devices is thus much faster than it used to be, the whole sequence from user, user interface, algorithm and database is highly organized and connected which thus enables multiple factors and contributor to participate into the design and updating processes, such permanent extensibility thus enables software-based media to always be new (Manovich, 2016, p.156).

Besides, as Engelbart mentioned in the earlier years when he believed that properly trained and with the right computer tools, we could raise out collective IQ, namely, computer is not simply a tool, but also a medium through the interaction with which we are therefore able to gain an increased capability to approach a complex problem system (Englebart, n.d.). As a metamedium, computer not only offers us an integrated platform involving all the previous medium, but also, it detects the affordance latent in the environment which was used to be considered impossible. Meanwhile, with the constant update of technology and interface, users are also required to adjust and work with the technology which in turn helps them develop the augmentation means (Englebart, n.d.).

Another interesting point about computer as a metamedium is based on its technical aspect where the interaction between user and computer is becoming more and more important in that the application layer lies on the top of the hierarchy of computer. According to Murray’s explanation of HCI (human-computer interaction), users are not merely consumers or audience, but also are invited into the participatory design process (Murray, 2012, p.64, p.71). This, on the other hand, introduces the situated action theory where it is not enough to merely see users as interchangeable processors of information, who all behave the same way, but as uniquely positioned, complex actors whose frames of reference is shaped by the surrounding social and material world (Murray, 2012, p.62). Take Twitter as an example, while we are checking the news on Twitter, our opinions are not only easily affected by our friends, families, schools, workplaces, but also those real time interactive comments by other users which to some extents serves as another source of information and shapes our interpretation of the news. Thus, computing devices, together with the whole complex social, technical network, makes such metamedium not only an umbrella which contains and manipulates all other kinds of media, but also a hybrid one where information is exchangeable and inter-related (Manovich, 2016, p.170).


  1. Engelbart, C. (n.d.). Augmenting Human Intellect: A Conceptual Framework – 1962 (AUGMENT,3906,). Retrieved April 07, 2018, from
  2. Irvine, M. (2014, October 18). Retrieved April 07, 2018, from
  3. Manovich, L. (2016). Software takes command. New York: Bloomsbury.
  4. Murray, J. (2012). Inventing the Medium: Principles of Interaction Design as a Cultural Practice. Cambridge, MA: MIT Press, 2012.
  5. Iphone App Development India. (n.d.). Retrieved April 07, 2018, from
  6. (n.d.). Retrieved April 07, 2018, from images&source=lnms&tbm=isch&sa=X&ved=0ahUKEwjV1Mf6t6jaAhWKx4MKHf9yCa4Q_AUICigB&biw=1422&bih=629#imgrc=vESOgQx7QDx_gM:
  7. Explainer: What is Snapchat? -. (2018, February 20). Retrieved April 07, 2018, from
  8. Deorukhkar, P. (2017, September 22). At a Glance: Newsify’s New App Icon • Beautiful Pixels. Retrieved April 07, 2018, from
  9. (n.d.). Retrieved April 07, 2018, from images&source=lnms&tbm=isch&sa=X&ved=0ahUKEwi6-5LNuajaAhWJ0YMKHd_7D2MQ_AUICigB&biw=1422&bih=629#imgrc=5c-lHMgimz0SzM:


Computational thinking and countless possibilities of implementation– Wency

Computational thinking and countless possibilities of implementation– Wency

It might not be a bad idea to start the discussion with an example we’ve learnt this week. We want to obtain a word which could be determined by everyone that have seen our need. We want to make it neat with all the characters in this word to be lower case. We also want to do something with the word, we want to replace the first character onto the end of the word and add some other self-defined characters onto the end of the new word.

Now it seems that we have a clear need with clarified task already. To accomplish the task, we need to break it down into several procedures with the help of our nature language:

  1. Define a string variable and assign a value to it
  2. Define a variable and assign the user input on it.
  3. If the length of user input is more than zero and the all the characters in the user input are alphabets, conduct the task below
    • convert all the characters within the word into lower case
    • define a variable and assign the first character in the word onto it
    • add the first character and the value of the string variable defined in the first step onto the variable
    • cut the first character of the new variable from it
  4. If the length of the user input equals to zero, then output empty.

The above steps, i.e., initiate a need, define the task, analyze the task and break down into several logic steps or logic functions. It’s human nature to break accomplish a complex task step by step each of which is relying on a specific function relatively independent from the other parts, and interestingly, we are sometimes unaware of that. The way human beings solve problems, thus, is a way of computational thinking, which, as Wing mentioned in his work, is a way that humans, not computers, think. Within just a few steps we are thus able to conceptualize the large chunk of task into multiple level of abstraction (Wing, 2006, p.35).

At this level, however, our computational thinking is better to be ascribed onto a mathematical level, to connect our daily lives tasks with computer, we need further transformation. As is mentioned by Evans, there are several inevitable problems with nature language including complexity, ambiguity, irregularity, uneconomic and limited means of abstraction (Evans, 2011, p36). Therefore, just as human society has its own grammar of nature language, there is a set of syntax that we need to follow to interact with computers (Irvine, 2018, p.7). Speaking of this example, we are using the syntax of Python to transfer our nature language into the first level of computer understandable language. We implement the Boolean logic thinking, abstraction of different variables into the code stored in, the interpreter later converts the language into binary values, i.e., 0s and 1s which could be manipulate by machines (Evans, 2011, p38).

Since 0s and 1s, basically speaking, stand for two states which represent the concept of digitalization where analog values are divided into several regions based on standards or boundaries, to merely implement two states physically, we can use a ton of different physical devices such as the fluid computer, the jacquard loom, etc., however, since the speed of electricity equals to the speed of light (i.e., 3*10^8 m/s), and since the electricity only need to travel nanometers between transistors, the two states of switch, i.e., ON and OFF, could happen millions of times within one second which makes the information processing much faster, therefore, using electricity to implement the binary operation is a more popular way in modern computation. By combining transistors into logic gates including AND, OR, NOT, XOR and combine the logic gates into more complicated modules including logic arithmetic units such as adder, comparer, etc., and those modules such as memory, control unit, arithmetic logic unit, etc., thus are combined together and interact with each other under control to work efficiently (Irvine, 2018, p.5). we are therefore utilizing instruction to perform several tasks based on the binary sequence (Hillis, 1999, p.20-27). At this point, back to Wings conclusion of the characteristic of computational thinking, it is easier to understand the concept that computational thinking complements and combines mathematical and engineering thinking (Wing, 2006, p.35). Computation, thus, although understood by many several scientists as mathematical thinking, is physically limited and measures the power of a computing machine includes how much information and how fast it can process (Evans, 2011, p.3).

Of course, the example I provided above is just one case in our daily lives. In fact, as Evans mentioned, we can use sequences of bits to represent many kinds of data including Numbers, Discrete Values, Text, Rich data (e.g.: picture is divided into discrete squares known as pixels) and there is a limitation in terms of frequency that human beings can distinguish, whatever that could break down into smaller chunks and procedures in daily lives and illustrated as logical steps could therefore be transferred into the machine understandable language, interpreted into binary sequence and thus implemented by electricity and transistors (Evans, 2011, p.11).

To look back on each step that we use to accomplish a complicated task, therefore, it’s not difficult for us to recognize how much symbolic meanings and representation are assigned by us onto each level of the task and how we implement such symbolic representation to enable our interaction with computers. We are offloading and distributing human agency and cognition into software not only because we are able to manipulate computers and enabling its automatic information processing, but we are also at the same time thinking computationally with abstraction, recursion, modulization ourselves through every single task we confront in our lives. We are in a digital-analog continuum for implementing many kinds of design concepts software designed for automating symbol process, computation is the outcome of cumulative human symbolic thoughts for representing abstract patterns and process, it performs metafunction which, not merely represent meanings, can also be used to perform actions on other symbols (Irvine, 2018).


  1. Evans, D. (2011). Introduction to Computing: Explorations in Language, Logic, and Machines.
  2. Hillis, W.D. (1999). The Pattern on The Stone: The Simple Ideas That Make Computers Work.
  3. Irvine, M. (2018). Introductory video.
  4. Irvine, M. (2018). An introduction to Computational concepts.
  5. Wing, J. (2006). Computational Thinking. Communications of the ACM 49(3). pp. 33-35

The history of music player: a sociotechnical analysis — Wency

The history of music player: a sociotechnical analysis — Wency

It seems to become a real commonplace for most music amateurs today to take a pair of earpieces and an iPhone with them to listen to music anywhere and anytime. Either you are hiking, going to the gym, on a travel, working on a project, studying for an exam, it has become a habit that you put on your earpieces and start to enjoy the music as well as getting better focused on your job.

However, things are different if we move back even just 10 years when iPhone was not as popular as today and people needed external music players (i.e. MP4) for them to listen to music. Further, if you ask your parents about their experience as music amateurs when they were young, they probably have a huge nostalgia back to the days when several children gathered together in some cool kid’s house and listen to the popular song on the tape or few years later, feeling extremely cool about walking on the street, carrying a huge Walkman, shaking their whole bodies with the music beats.

  1. From ancient live performance to phonograph and records: a need to preserve the temporal audio piece

People’s need of enjoying various forms of art started in the very earlier time when people made instruments and did live performances. They also invented notes to communicate with each player as well as preserve the songs. However, the key, rhythm, beats, etc. of the song could be preserved, the performance itself couldn’t. For those pure instrumental performance, it might not be a huge problem since people were able to produce many tokens of the original instrument as well as tokens of the original performance so long as they follow the rules recorded on the notes, but what about the singers? Words, notes per se would never be able to fully record the original voice of the singer. At this point, therefore, we see the need of humans to invent a technology, a durable tool that could extend, maintain the skills and thus break the limitation of time and space, they need to maintain the entire succession of accumulated elements for future innovation (Latour, 1994, p.61; see also Latour, 1991, p.109).

The phonograph, to some extent, sees an affordance which is latent in the environment (Zhang & Patel, 2006, p.336). While phonograph incorporates a desire to preserve the temporal music or broader, sounds, accompanying that desire would be a need to play that audio piece in the future. Therefore, records and LP records, 10 years later, are invented to play the audio piece and to keep refining the standard.

  1. Records –> tapes –> Walkman: an increasingly need of portability and convenience.

If the invention of the phonograph and records are intended to preserve and play music which is still focus on the music per se, the long-term transition from records to MP3, nevertheless, might be more or less deviated. Can we invent a technology that would allow users to carry them so that they could listen to music everywhere? Can we make them smaller, lighter so that users won’t be annoyed while carrying them? The idea of new innovation, the transition of an idea to a project, a project to an object, is not only incorporating the people who inhabit it but also they wish to effect: product development team sees the potential of the user’s need and imbues such potential into the update of new innovation. On the other hand, users’ earlier roles, habits, functions also provide a precondition for such innovation (they’ve get used to the earlier mode of listening to music and they themselves would see some lacks on the current product, e.g.: while going out for a travel, they might want to carry something with them to kill time) (Latour, 1994,  p.49). The music player, therefore, is becoming smaller and smaller and Sony reached one of its annex by selling the Walkman since 1979.

  1. From Walkman to MP3: analog to digital

The well-known Sony Walkman didn’t last forever, unfortunately. While the main technology inside it was the vibration of magnetic sheet, the external noise and the dust accumulated on the sheet would be inevitable to lower the tone quality. As the whole technology industry’s working better and better on computing and the utilization of digitization, this also facilitate the development of digital music. Through the analog to digital transition, people were therefore able to numerically view the noise and eliminate them more accurately, the quality of listening to music, were therefore guaranteed.

At the same time, the users need is incited and diversified by the environment as well as their previous experience, if we think about how users were more and more used to the better quality provided by the digital music, more and more of them might not be willing to return back to the Walkman age when there were too much noise interrupted, it would not be difficult to relate this to the modern distributed cognition theory where the boundary of cognition is not individual anymore, the external sources, tools in the environment should also be considered in the range (Hollan, Hutchins, & Kirsh, 2000, p.175, p.193).

Meanwhile, the blossom of digital music allows the musicians to delegate part of the real performance in the recording room to some electrical synthesizing effect (the computer is later be programmed to be able to simulate a ton of sounds from different kinds of instrument which gives musicians multiple choices as well as save a lot of money to find the players for each instrument needed in the song (Latour, 1994, p.39). But at the same time, modern musicians are required and trained to be more and more adjusted to work with those new electrical technologies and new occupations are created in a macro social level.

  1. From MP3, MP4 to Streaming media: not music itself but the market of music player devices is under a revolutionary– mixed affordance: the whole industry

The story seems to finish if we are only focusing on listening to music conveniently with high quality and without the limitation with time and space. Nevertheless, the reality today seems to be unpredictable 10 years ago. In the digital age, everything can be integrated and thus be defined as a computer, while MP3 is focusing on digital music which also provides a large potential of storage, why don’t we add some new features onto it? Can we watch videos, download pictures, play games on that as well? The iPod touch is later known for offering anything but communication that a smartphone does.

But as the price is becoming higher and higher as the new music player is becoming more and more function diversified, why not just directly integrate the music listening function onto the smartphone? It’s highly integrated, convenient and much cheaper than purchasing an external device. Such need thus accelerates a whole revolution in the streaming media where Apple music, Spotify, Pandora, etc. came out almost overnight and everyone starts to download those application on their phones to listen to music. While these applications substitute the previous music players, they are competing with each other as well as collaborating with the smartphone industry such as Apple store. An economic mediation through time and space is thus emerging as well (Latour, 1994, p.45).

  1. What would be the next?

That could be a tough question. In fact, technologies are not fetishes, they are unpredictable, not means but mediators (Latour, 1994, p.53). Both the society and technology are mediating what’s currently mediating themselves, the development of technology is thus non-linear. We are living at a period when technology, society, politics, economics and culture are all serving as agencies where the whole system is shifting from single actor to many agents, from homogeneous agency to hybrid constellations and from hierarchy to framed interactivity (Rammert, 2008, p.13-16). Meanwhile, the larger system is broken into subsystems, or modules which work relatively enjoy independencies but are also dependent on and interacting with each other invisibly (Irvine, 2018, p.2). Like what we learned in 506, the whole complex system is like a blackbox which contains infinite blackboxes inside. To understand a system, we need to define it, explore the interaction of its components as well as elaborate its relation with the external environment.

Date Event
1857 Leon Scott de Martinville’s Phonautograph 

The phonautograph could record, but could not reproduce sounds. The original design for the phonautograph eventually led to the gramophone.

1877 Thomas Edison’s Phonograph 

The phonograph made recorded music possible. The device recorded sound, including human voices.

1887 Emile Berliner’s Gramophone 

Emile Berliner created the Gramophone, the first device to play a disk of recorded music, in 1887. The gramophone made recorded music accessible.

1896 Gramophone on the Market 

By 1896, the gramophone was on the market as a Victrola, playing disks of recorded music. This is the first commercially available record player.

1905 Beginning of 78 RPM Standard 

The 78 RPM standard was introduced. This enabled shoppers to be sure that their records would play on their Victrolas, and play correctly. This remained the standard until the introduction of the LP in 1940.

1954 First Transistor Radio 

In 1954, the first transistor radio allowed listeners to take music with them, as the radio was now small and portable.

1962 First Portable Stereo 

The first portable stereo integrated speakers into a record player, allowing people to take their record player with them, moving it wherever they went.

1963 Audio Cassette 

The audio cassette offered music in a smaller and more portable format than ever before. Audio cassettes also enabled the first mix tapes.

1965 Release of the 8-Track Tape 

The 8-Track tape brought recorded music into cars, long before audio cassette players were integrated into car stereos.

1979 The Walkman 

In 1979, the first personal music player was released by Sony. The Walkman combined an audio cassette player and headphones.

1983 The First Compact Disc 

The Compact Disc offered higher quality recording, and increased durability compared to an audio cassette. By 1984, portable CD players were available.

1998 First MP3 Player 

The first MP3 player, playing audio files, was released in 1998. The player eliminated the need for another media to hold music.

2001 Apple’s First iPod 

Apple released its first iPod, taking the MP3 player mainstream in 2001. The iPod made digital music significantly more popular.

2007 iPod Touch 

Apple released the iPod Touch. The iPod Touch served as a music player, but also offered access to the Apple App Store, games and other features.

Table 1. (Evolution of Music Players Timeline, n.d.)


  1. Evolution of Music Players Timeline. (n.d.). Retrieved March 27, 2018, from
  2. Hollan, J., Hutchins, E., & Kirsh, D. (2000). Distributed cognition: Toward a new foundation for human-computer interaction research. ACM Transactions on Computer-Human Interaction,7(2), 174-196. doi:10.1145/353485.353487
  3. Irvine, M. (2018). Media, Mediation and Sociotechnical Artefacts: Methods for De-Blackboxing.
  4. Latour, B. (1994). On Technical Mediation. Common Knowledge 3, no. 2. 29-64.
  5. Latour, B. (1991). Technology Is Society made Durable. A Sociology of Monsters: Essays on Power, Technology and Domination, edited by John Law, 103-31. London, UK; New York, NY: Routledge, 1991.
  6. Rammert, W. (2008). Where the Action is: Distributed Agency Between Humans, Machines, and Programs. Social Science Open Access Repository (SSOAR).
  7. Zhang, J., Patel, V. (2006). Distributed Cognition, Representation and Affordance. Distributed Cognition Pragmatics and Cognition Pragmatics & Cognition,14(2), 333-341.




From GUI to Embodied interface: Who is next? — Wency

From GUI to Embodied interface: Who is next? 

From 506 we have learnt the concept of interface which serves as a boundary of two separate components or systems that enable them to interact with each other. Partly being considered as a channel for transmission, media have been considered for a long time as an interface between human and different contents. According to Dourish’s interface paradigm, there are four modes of interface (i.e. electrical, symbolic, textual and graphical). Today, as the whole digital industry is keep trying to make everything more human readable and user friendly, the graphical interface, think about every icon on the screen on your computer for instance, also known as GUI, is being more and more important providing the visual metaphor functions.

According to Manovich, media today is becoming new media where everything presented on the GUI, for instance, are computable, the visual metaphor of icons resembling the real object, the pattern recognition, would all be broken down into the smallest controllable unit of digital display called pixel and then transferred into numerical representation for machine to understand and manipulate (Manovich, ,2011, p.20-27). Manovich also mentions the variability as a property of new media where users can customize automatically the media composition as well as to create elements themselves (Manovich, 2011, p.37). GUI, on the other hand, is a conceptual model aligned with changes and development of hardware, the traditional unidirectional impact, therefore, is transferred into bi-directional or even multi-directional network.

On the one hand, technology is immersed into the development of media to change the way we interact with contents, on the other hand, just as professor Irvine said in the video: we need to think about media and mediation beyond channels of transmission (Irvine, 2018). As I’ve mentioned in the above paragraph about conceptual model, the HCI (human computer interaction) means that we, as individuals living in this social, economic, cultural and political world, are also interacting and shaping the technology, or digital interface. The technology in the whole system of mediation are never autonomous, they come together through a large amount of policies, institutions, law, etc., power, authority, social factors are implemented into new technology which is keep reconfiguring (Irvine, 2018). In Bolter and Grusin’s work about remediation, they mentioned the example of visual reality the goal of which is let users forget about the computer interface, the interface thus becomes transparent, or erased (Bolter & Grusin, 2000, p.22). Similar case goes to the E-book which is mentioned by professor Irvine in the video, the goal of which is somehow let the user feel as if they are reading a real printed book. The user experience in the VR industry, provides feedback and thus shapes the industry due to the industry’s economic and business interest. In the latter example, we can also see new artifacts as an interface to invisible historical dependency where people have experienced a previous technology (the printed book in this case) and become dependent, or better, lock-in (the concept is derived from Latour) to the previous technology that they sometimes refuse to transfer into a new version of technology (Irvine, 2018, p.9).

Today, the elements being interfaced is increasingly growing where our location, context, gesture, body movement, voice, etc. can all be captured and computed. The embodied interface, as a new stage of interface after graphical interface, emerges also with social, cultural, political and economic factors behind it. For example, the Kinect captures the users’ motions and almost simultaneously display them onto the screen. The interesting part of Kinect, for example, is that if you are playing a virtual tennis game, you will even feel the power of heating the ball, the sound simulation tracks and matches so perfectly with the movement that users’ sometimes even feel like they are playing a real game. Here we are again back to the double logic of remediation. On the one hand, the development of new media provides us new ways of interacting with every single content in the real world, but on the other hand, such mediation, or according to Debray, the transmission which stands for changes along time, is always affected by historical development: the rules of the game, the display of the settings, the speed, power measurement and sound effect, are all designed to resemble the original tennis game at maximum. Kinect, or better, every modern media or digital interface we can imagine, is a combination of spatial communication and temporal transmission (Debray, p.3). The development of media, the mediation of media itself, are thus happening under the sociotechnical system where the development is always non-linear but at the same time having a more predictable direction.



  1. Bolter, J, D. & Grusin, R. (2000). Remediation: Understanding New Media. Cambridge, MA: The MIT Press.
  2. Debray, R. (2000). Transmitting Culture. New York: Columbia University Press.
  3. Irvine, M. (2018). Introduction to Media and Technical Mediation.
  4. Kinect sensor with Kinect adventures! (n.d.) Retrieved March 21st, 2018, from
  5. Manovich, L. (2001). The language of New Media. Cambridge, MA: The MIT Press.


Interaction and common grounds—A further understanding of the information theory-Wency

Interaction and common grounds—A further understanding of the information theory-Wency

 The development of different forms of information transmissions is inseperatable with human’s need of mutual interaction. In the earlier times, people were only able to communicate with each other in person when they need human resource to pass the information (i.e. letters) if they need distant connections. It was when later people started to realize the usability of physical resources (i.e. electrical and magnetic waves that are not completed limited by space and distance) that the blossom of electrical telegraph and the utilization of electromagnetic system of signals began. At this point, human beings and those machines were nearly isolated where the human communication system seems to live in a world separated from the physical communication and transmission system. On the one hand, each individual in the human communication system generates meanings constantly where they were able to communicate with each other under the social and cultural context which more or less embedded into their memory and affect the whole process where they make selections in the perception of signs, realizing what is inside the box (i.e., the object), come up with further connotations and also as an output that contributes back to the social, cultural context. On the other hand, people have understandings of how to utilize the bottom layer (i.e. the electronic signals) of communcation to pass the signals no matter the distance. By correlating pulses of electrical current to an abstract symbol set, Morse was able to implement the interface of such bottom layer with the earliest human compiler when they interpret those symbols and transfer them onto a higher level of human readable form of information. At this point, according to Shannon, what matters was not the distance but the control, in other words, how to minimize the loss of information, counteract with the noise which was inevitable during the transmission of signals alone the channels and how to recover the information on the receiver side, in the Mathematical theory of communication, he pointed out that the entropy of information (i.e., the randomness and uncertainty) can be measured and controlled in a mathematical way where information could be guaranteed to be transferred in a maximum way as well as not being changed or lost significantly. To be sure, human beings are still interact with the physical communciation system, but because the update of the whole information system is quiet slowly input onto the system, such interaction is too slight to be taken account.


  1. Growing interactions between human and machine

However, Shannon’s classical information theory that information can be transmitted received accurately by process that do not depend on the information’s meaning seems to be more and more doubtful in today’s standard. After Alan Turing’s first presumption of what makes it a computing machine (although it was pure imaginary without physical support), today the blossom of computer industry obviously is making the transform of information (i.e., the creation of new information) incredibly faster. This seems to be easy to explain, while computers are operating based on the electricity transferring between the transistors, such electricity is passing in a speed identical to the light (3*10^8 m/s), which means the update of a large binary sequence could happen in a second. Moreover, while different compliers are substituting the position of earlier human reader in the Morse’s era, the interaction between human and machine is becoming much faster in a human readable way. When we are reading a text message, digital image, listening to a music piece, watching an online Youtube video or writing some comments at the under interface with digital devices, we are using symbolic interpretative process all the time to map the representations onto recognizable patterns and make inference about what motivated the encoded information that gets transmitted through the system, all of these events happen under a macro social, cultural context where we absorb knowledge, basic rules that affect our interpretation as well as changing the context by ourselves. Our user actions dynamically goes onto the front end programming language of computer and gets transferred layer by layer until they reach the binary system where data packets are thus able to be sent under the manipulation of the TCPIP model. Therefore (according to the figure above), information, although still unchanged in the bottom level while being transferred, is constantly being updated at the application level (through the user interface) within the help of human agencies as the constant meaning generators.

  1. Understand the similaries between the tokens of Turing Machine

The Turing machine, though depicted in an imaginary way according to Turing in the earlier year, is a universal one which set the standard of all the machines or systems to be a computable one. According to the earlier depiction of the universal Turing machine, three major items are necessarily to be mentioned here: tape, symbols and states. A tape is a long strip divided into squares where symbols are written to or read from through a head of the control unit. States determine the machine’s actions according to themselves which in short can be described as in state i, execute algorithm Ni. We can thus easily assign those items into different tokens of the universal Turing machine and therefore realize increabily large amount of connections between human mind and computer. In a macro way, tape, as temporal memory storing the earlier symbols written into it, can bring us to an analogy to human’s memory (including long term and short term) and also the memory in a mechanical way (RAM, ROM, hard disk and today’s cloud storage). The symbols therefore would be those content stored in the memory where we can also read them, or better, retreive them for later actions (e.g. think about human beings’ analyzing a sentence according to the linguistic system, they percept the words and retrieve the memory to map those lexicons in a syntactic way). Last but not least, the states stands for all the process when we analyze something or make a decision (if this is the case, then I will…other wise I will…), and such states also map well in a digital world where we first interpret those states of mind into pseudo code and then code them into a large amout of IF-ELSEs.

Further, if we look in a micro way, what amazes us is that the physical structure of signals and systems even matches well in a biological way – Every neuron, could somehow be considered micro system containing transistor (axon terminals), channel (axon), receiver (dendrites), and the electricity passing between the transistor in a computer system here could be substitute by electrochemical signals and they are more or less all controlled by the central nervous system (probably is similar to the CPU). What is more interesting is that even the electrochemical signals are not completely analog signals (there is somehow several instantaneous changes between states). Currently what confuses a lot of psychologists is how these electrochemical signals are later generate in a larger semantic way (now in computer system we know how electricity are represented in binary system and goes all the way up to human readable programming languages). But what we now know is human’s ability to jump out of the existing symbols and information to a external virtual imagination in order to complete their logic thinking (If we understand fully about how this is realized in an anatomic way we are therefore probably be able to bring computers into a more autonomic level, but fortunatly we are still on the progress of making computer more and more human-like, including enabling computers to compute in a more emboddied way, i.e., the capture).

Nevertheless, no matter how abstract we’ve been right now, we need to bear in mind that all the forms of information lie upon a thermodynamic system where information is inseperatable with energy and physical structures (computer release heat, human beings are consuming calories while thinking, etc.), and while according to the second law where there is always a tendency where entropy is going to increase, how to recycle information in an efficient way can never be avoid to discussion.


  1. Martin Irvine, “Introduction to the Technical Theory of Information
  2. Luciano Floridi,Information: A Very Short Introduction
  3. James Gleick, The Information: A History, a Theory, a Flood. (New York, NY: Pantheon, 2011).
  4. Peter Denning and Tim Bell, “The Information Paradox.” From American Scientist, 100, Nov-Dec. 2012.

Music Association with other forms of work – Wency

Music Association with other forms of work

Last weekend I did an interesting psychological test which I personally think is highly semiotic so it might not be a bad idea to start from it. In this test, there are overall seven questions each of which plays a different piece of music, based on the music, you are asked to choose which painting, scene, drink, etc. that you think fits best with the piece and at last the result will show you the gender construction of you. The details are as follows.

  1. Which painting do you think fit best with the music?
  2. What time do you think fit best with the music?foggy morning; b. sunny morning; c. a nightfall with beautiful setting sun; d. a night full of stars
  3. What kind of drink do you think fit best with the music?
  4. What do you think is expressed by the piece of music? a. The excitement of seeing an old friend. b. The expectation for a great future. c. Missing the lover who is far away. d. The helplessness of the unfair destiny
  5. Which scene comes into your mind while hearing this piece of music? a. A farm boy walking on the field. b. A driver driving on the highway. c. A wanderer lying on the window. d. The brave soldier on a battle ground
  6. Which place do you think the music comes from?
  7. Use a word to describe your feeling towards the piece of music.a. Despair; b. glory; c. rebirth; d. quiet.

This test serves as a combination of how opinions, comments and verbal or visual associates to music (Tagg, 1999, p.33). After listening to a piece of music, we first get an immediate interpretant of the piece based on the then based on the codes and categories we have learned within the sociocultural system we are in, we are thus able to add other associations for building a stack of multiple interpretant, which, according to Peirce, is a way of Dynamic Interpretants (Irvine, 2018, p.18). Our previous experiences somehow build into our memory that enables the successive connotation to happen (Tagg, 1999, p.5). Take the second question as an example, what I heard from the piece was a combination of two electrical guitars and a drum where the tonic guitar and the rhythm feats well together in harmony and the drumbeat feats exact at each point, in addition, though not sure which key the song is playing, I could tell that the piece is played within a major mode. But the knowledge of the frames within this piece is not the only source for me as a receiver to interpret the piece. I also come up with several words such as relax, straightforward, cozy, free, happy through listening to the melody.

But at this point, my interpretation is still more or less ambiguous and I can’t depict a specific scene according to these words. As for me, the reason why I couldn’t give a specific description is that those words could stand for several different scenes, probably several friends sitting near an outdoor swimming pool, or me sinking in my bed in a sunny, warm afternoon after having all my homework done. Luckily, the options in these questions somehow gives me a hint or a constrain for my depiction. Since a symbol may have an icon or an index incorporated into it, by looking at the symbolic description of each of the scene, I was able to go to my memory, find out similar experiences I’ve been through before seeing the symbolic forms of these scenes, in this way, these four scenes somehow becomes the instances of the prototype, i.e., my original previous experience which is unique in a 4 dimensional space (i.e., a 3 dimensional space and the other dimension stands for time) (Irvine, 2018, p.19). Then, if we consider each scene as a new starter, i.e., the first signifier in our second connotation process, we can therefore develop other lists of symbolic representamens for each scene, by comparing the list I developed through hearing the music and the list I developed through these four scenes, I would be able to select the one that satisfies the largest number of matches: I remember a typical 80s movie scene where on a highway of California, there was a stunning nightfall and three cow boys are singing and yelling on ragtop. In this way, the interpretant of music also moves through the timeline where historical styles and moments are recalled and also a California style is highly cultural defined (Irvine, 2018, p.11).


Now what’s interesting is that I barely have any clear concept of what, for example, blue, Jazz, folk, pop music would be like, nevertheless, as a person who has a very limited vocabulary of describing different established music genres, I can somehow distinguish different token-instantiation of types, i.e., different song elements based on their frames including rhythm patterns, instrument and timbres, melody, harmony repetitions, etc. that they belongs to types, or better, they are objects or instances of different classes (Irvine, 2018, p.8). Such concept of different genres, though not clearly described, constrains the range of the following signified I would come up with during the connotation.

Besides, appropriate elements of the respondents’ sociocultural habitat, such as age, gender, nationality, education, profession, etc. also count into the whole procedure of communication, i.e., how the information is transferred from transmitter to the receiver and the degree to which the information is interpreted as what the transmitter expected (Tagg, 1999, p.33). In this test, while I and most of my friends are sharing a similar cultural background, we respond highly different according to each music piece. This, according to Tagg, is an intersubjectivity from the receiving end (Tagg, 1999, p.22). Now if we move back to the association chart I draw, every individual, even in a same cultural background, is holding a unique memory which affects the whole process where she or he see a form of representamen (e.g. a music piece or a painting), select in a short reaction time the elements in the large scale and generate meanings ongoingly based on these signifiers and further ones. To be sure, I would slightly criticize this text for containing more or less gender stereotype, but the mode one uses to think and interpret symbols, though one can argue that happens unconsciously, as for me is highly context embedded. If we move back to the basic communication model according to Tagg, each component is therefore mutually interactive with others, therefore, while work of art can be considered as a macro unit in a dialogic, the response is multi-directed and somehow unpredictable (Irvine, 2018, p.14).



  1. Irvine, M. (2018). Introduction to Visual Semiotics.
  2. Irvine, M. (2018). Popular Music as a Meaning System.
  3. Tagg, P. (1999). Introductory notes to the Semiotics of Music.

visual art and computing in semiotics

Visual art and computing in Semiotics

It might not be a bad idea to start with the image done by Kara Walker in the latest exhibition Harper’s Pictorial History of the Civil War for this week. For me this image serves as a great example of the sign as an index. Considering the idea that an index is what calls our attention or what startles us (not necessarily need to be pointing to a physical or material object as is mentioned earlier), for me what draws my attention first is the breast of women, or better, the breast of the person. By catching the breast in this image, I then realize that the author probably wants to express an image of woman, or the whole category of women. My experience is a great example of seeing the representamen (the breast) as the box and label and realize there is something this representamen stands for inside the box, which is the refrent or the object (in this example, women). And according to Peirce’s model, the agency that enables me to correlate the breast with women together is the interpretant. Now the other interesting point is that while already knowing that Kara Walker is a feminist, I almost immediately correlate the image of the person into the category of women instead of specifically one particular woman. However, what if I know nothing about feminist and the only thing I know about this exhibition is civil war? My attention might therefore be driven by the facial features of both the two larger black figures and the smaller white figures. In the former case I am thinking Kata Walker’s intention of showing the power and liberation of women and in the latter case I am thinking about the elimination of slavery. My experience now might be able to explain how intersubjective and inter-individual social bonds of sign users, which are installed by symbolic cognition in lived context are preceding individual understanding of expressed meaning, in other words, meanings are dialogic and context-dependent.

Meanwhile, according to the fact that semiotics is the general discipline devoted to the study of signs and symbols as systems of expression, meaning, reasoning, and cultural memory in any kind of artefact (human-made being), we can also apply Peirce’s model into other fields. Still using this image above, how can we tell computer to represent it? Or broader, for every visual file, how do computer represent it? What we currently know is for an analog image, on computer we digitalize it into thousands of pixels where we give each pixel an RGB color for which we use a byte to represent. Before each color-standing byte we also have a key byte to tell the computer how many pixels are going to be filled with color and if we use one byte to give these pixels the same color or we assign each pixel with a different color by different bytes. From this level if we go up towards the top level we might go through more and more abstract and human readable algorithm and if we go down we are then amazed by how human beings are able to use 0s and 1s to represent ON and OFF which, is achieved by perceptible, physical and materialized combination of transistors. We use 0 and 1 as symbolic signs to represent the real electric states, where we might not be aware of a consensus within at least the community of engineers that such relation is achievable in computer (interpretant), or based on this interpretant, we can go one step further: we can break down analog things into digital one, separate it into several Boolean states and using logic gates on computer hardware to instantiate them. The interesting thing is that if we ask why through every detail of the above process, it might take us forever to answer the question (we are not even able to pause a while on the hardware where we find the exact real thing to interpret one concept).

But back to the idea of seeing a box and realizing there is something inside the box, symbols are not static or isolated things, but are structures of patterns in a dynamic cognitive process for meaning-making, a process about which we can never be consciously aware and cannot be observed except in the perceptible expressions and representations. The first point is that we are able to keep generate unlimited meaning with the help of word which is unique within human beings, the second point is unfortunately we might never be able to touch the object inside the box. That could probably explain why on Peirce’s successive model, the object appears always like the outsider: the line connecting infinite generated representamens and interpretant, probably using language as the agency, can only let us think in signs instead of things. I’m not sure if it’s appropriate to bring Lacan’s viewpoint here, but an interesting idea he brings is that unconsciousness is the construction of language and on the symbolic chain human beings are never able to touch the thing. Paradoxically, while human beings are able to live in a second world with the help of language, this second word, somehow doesn’t have an interface with the real world we live in, in other words, we are distinguished from other animals by language, but such privilege later undergoes an abdiction by index signs where we points the real thing find ourselves, like animals, finally pause a while on a real thing.



  1. Lacan, J. 1965. “The Mirror Stage as Formative of the Function of the I as revealed in Psychoanalytic Experience,” “The Insistence of the Letter in the Unconscious” in Ecrits.
  2. Kara Walker: Harper’s Pictorial History of the Civil War (Annotated). (n.d.). Retrieved February 14, 2018, from
  3. Irvine, M. 2018. Meaning Systems, Communications, Media and Technologies.
  4. Irvine, M. 2018. Introduction to Signs, Symbolic Cognition and Semiotics.
  5. Irvine, M. 2018. Semiotics: Meaning systems, Culture, Technology Key Texts.