Author Archives: Xiaoyi Yuan

Challenge of Applying Language Models to Visual Genres

This week’s reading is about linguistic theories and the potential of applying language models to other nonlinguistic semiotic systems. As you all know, I did research on building film cognition models based on Parallel Architecture. I re-read the materials again for this week and in this post I want to think through the challenge of applying linguistic model to visual genres. One big challenge laying in front of me is that after realizing the parallel cognitive model of visual images or films: How can we go further on top of this realization to do more valuable research?

In film studies, there’s a branch that focuses on film narration structure since 1960s. The mainstream film narrative analysis is to look film as a structured language. Based on Chomsky’s generative grammar, they build multiple theories based on the methodology called film semiotics. Within this approach, scholars try to synthesize possible narrative structures of film generally. Figure 1 is a famous film narrative model called grande syntagmatique, built by Christian Metz. He believes that based on these film specific syntax, we generate stories and meanings, which echoes Chomsky’s generative grammar theory that the sentence and meaning/semantics is produced on the based of grammar/syntax.


Figure 1

But after this week’s reading, we know that meanings are in networks (Figure 2.) Syntax, however, is not the presumption of meaning. Same for film meaning construction. The process of encoding and decoding of meaning in film is embedded in larger cultural context and the syntax (narrative structure), semantics, sound and other elements work together at the same time.


Figure 2

However, I think the big challenge for me so far is that after realizing the model of film meaning construction, it is hard to go further. There are indeed so much potential to start with things I talked above. But how to frame further possible research? I do get lost in this sense.

Let’s Think About Hominid

Last week’s seminar we talked about the misunderstanding we have and presumptions we take granted for the discipline of Communication. That was very thought provoking to me but at the same time, I am afraid that some “mainstream” research cannot convince me as much as before (Mia’s example in her last week’s blog). However, the curiosity to find the interdisciplinary method also drives me to dig deeper by stepping back to get a broader landscape. The landscape is historical (why and how human became symbolic species) and related to other disciplines (linguistic, communication technologies, media, etc.)

Since I’ve read all the materials for this week in last semester’s, I tried to understand the supplemental reading on mimetic skills, Homo erectus, and Homo sapiens. I am not sure if I get the technical part of those explanations on brain structures but the idea of human evolution triggered intense thoughts about the final paper from last semester. I used language models to understand film cognition processes. The only reason for me using language models was that: it makes sense to me. However this week actually gave me a chance to rethink the real and solid reason for me to apply linguistic to filmic cognition.

According to Donald, there are three systems of memory representation: mimetic skill, languages, and external symbols. Each of these three is based on an inventive capacity, which means they are always growing— new lexicons, “new” languages (computer languages), new ways of presenting informations (digital versus old none digital) etc. Mimetic skill is the presumption of human language. We reflect on what we do and think. When we start reflecting, we became a symbolic species. We act and then think and eventually refine our actions. The refined actions are external representation of internal thinking and reflection. This process of converting internal information (meaning or thoughts) to somewhere external or vice versa is fundamental. From this perspective, languages, hunting tools, even just refining actions of grabbing things are essentially the same— something that helps exchanging meanings between the internal and external.

When I rethink of this argument and cognitive basis of language as “technologies”, I start to realize the solid research foundation for my final paper. Borrowing linguistic model to other media forms is not just “useful” but also based on the fact that human beings are fundamentally symbolic species. Cognitive approach is a good way to break the walls between different disciplines as cognition is beyond specificity of cultures and societies.

What’s Interesting in Interdisciplinary Media Theory and How That Helps Us Think about Digitalization?

This week’s readings take a landscape view of the fields that help us to open our way of thinking out of the box. It challenges the assumptions that typical communication text books based on. There are indeed overlapping with the concepts that we were talking about in another class last semester with Professor Irvine. So in this post, I want to talk about something I think is both fresh and thought provoking to me.

Before read this week’s materials, I have a sense of the broad landscape of this class. Also I had a background of journalism and communication. What stroke me most is that, bearing the interdisciplinary method and Debray’s mediology concepts in mind, when I looked back the more mainstream communication discipline. McQuail gives us a very systematic way to understand the field of communication and I think can perfectly answer the question that most of people will ask: What does the discipline Communication mean? The term communication is so mundane that makes it not easy to understand in a academic scenario. Also we usually are confused with the use of the terms communication, information, media and medium. All those confusion and questions lead me to think: do we have to draw a strict boundary of communication as a discipline? If we don’t, what kind of interesting question that we can ask that is beyond the boundaries? Like what Debray says, “by looking from the perspective of mediology, we can ask more ‘interesting’ and ‘missing’ questions.”

So what is the assumptions that will change if we break the wall of disciplines? I think one of the most prominent changes are looking at information technologies and medias as interfaces. As what Luciano Floridi said in his essay Information, “What all these and many other metrics have in common is that they are all historical.” Every technologies that we have now are interfaces of the past and future. It is based on the historic development in both technologies and the numerous interaction with the society. Moreover, technologies and media are interfaces with other systems at the same time. Language, as a model of our “medium” illustrates this point to a great extent. Also, Stuart Hall’s theories of encoding and decoding help us to understand media as “interfaces”. We have our meaning system inside of our brain which embedded in a larger social/cultural context and we communicate meanings through language with other people. Every time we speak, we “offload” the meanings onto language and convey them to the people we talk to. And they receive the language and decode in their own brain. Thinking about other information/media systems such as computer (coding), cultural products (films, paintings, and music.) We “offload” our internal meanings onto those media vehicles and constantly encode and decode them at the same time.

The last subject that I want to discuss particularly is the question that professor Irvine raises in his article about digitalization. Professor Irvine asks that, what does it mean by digitalization? What is digitalized and what is not? What is the implications? I think those are all very thought provoking questions that we normally take granted. However, many years ago, Walter Benjamin talked about mechanical production of photography and its impact on traditional art (eg. paintings.) Interestingly, he then asked almost the same question that we are still struggling with nowadays (or maybe more confused about.) His most prominent concept “aura” stirs heated discussion about the “impact” of new media technologies on old ones. Benjamin was afraid that photographies with its features that can be reproduced unlimitedly, will impact the way we think of art. However, he is less interested in claiming that photography is bad/good than proposing that our NOTION of art is changing. I think every media technology is trying to define and redefine what is real. Same as digitalization. We are trying to use digital representations to build a new mode of reality. Our notion of realness and reality is constantly changing. Too illustrate my point, I’d like to use the Google gallery as an example. Strictly speaking, we cannot experience the so-called “aura” of the paintings and other objects from Google gallery but it does “augment” the reality in some sense. We can juxtapose two piece of arts on computer screen and read information and do comparison. We can zoom in and see the details of them, to a degree that I assume that we cannot see when we are literally standing in front of them. I think from the interdisciplinary point of view, we are less interested how those kinds of technologies IMPACT our culture because they are part of our culture, but more interested in asking questions like: how do we help those technologies to present and transmit culture better? I guess it is still an abstract question. I think more specifically, the question can be, is it possible to include every art all through the world into this gallery and what is our political and economical barriers that block us to do so?

I know I am talking too much about something we WILL talk about in the future. But this post for me, is a way to make my thoughts clearer and hopefully, will help people who read it, a way to think through those readings.