Category Archives: Week 11

How Personal is the Personal Computer?

The emergence and development of the graphical interface was definitely a foundational concept that supported computing to serve as a mediating platform. Numerical (formal) representation played a key role in translating any “media” into data that can be manipulated, changed, deleted, meddled with and copied through means of computation. Much of Alan Kay’s ideas stemmed from his deep insight of learning in human children and adults. For children he referred to the potential application of the Dynababook (personal computer) aptly as a ”dynamic book” as supposed to an inactive one. Alan Kay mentions Papert’s work that children should be taught to program the computer instead of the other way round. “Should the computer program the kid, or should the kid program the computer?” (S. Papert). When someone buys a personal computer today, the act of “ownership” gives them certain freedoms to utilize or manage their “property”. Rarely do we enjoy those freedoms with the personal computing gadgets we own today, except maybe be able to change the desktop wallpaper or change the time. There are two things I would like to see developed in industry. Firstly, the personal computers allowing users to build their own programs. This idea at first didn’t sound feasible to me until I read Alan Kay’s written work at length about the computer as a tool that “manipulates information” for humans. Secondly, I look forward to a keyboard and mouse free interface future, although much later than if the first feature is afforded by the PCs on the market. That would, I can only imagine, include directly talking to a computer as the primary interaction (or communication), mainly with the development of natural language processing. However I have difficulty in pointing out the trumping benefits of such a user interface than what we are used to today. But then again, according to Alan Kay, “interface is a conspiracy”. I guess the question that we need to ask is what conspiracy do we want people to embroil in?


Computer as Metamedium

Nowadays I can’t imagine a single day without using computers (including my laptop and smartphone). Everyday, I encounter different kinds of media through those two machines. I deal with text when I do my readings for the courses.  Various images keep popping up with I browse around the Internet. Video and audio hit my cognitive system through computer Interfaces when I watch basketball games or Rick and Morty. Moreover, text, image, audio and video are the media employed when I use instant messaging software on my smartphone to contact my friends in China.

These symbolic activities that we take granted of in our daily life are possible because of a set of concepts established by Alan Kay and other pioneers in the field of computer science of his generation. Kay thought of computer as a Metamedium, which is a platform where all possible media used for human expression and communication can be implemented, re-mediated, and manipulated, in contrast to the early idea for computer that one machine designed to serve a certain set of purposes. And here we are–I can manipulate all kinds of media through my laptop and smartphone. And I don’t need to buy a new computer If I want to deal with a new kind of media, instead I just need to implement some software in most cases.

Kay also developed the Graphical User Interface to facilitate users’ learning, discovery and creativity. His design was influenced by cognitive psychology and aimed to enable three human mentalities–enactive, iconic and symbolic, which referred to mouse, icons and windows, and programming languages, respectively. These three types coincide with Peirce’s typology for signs, though using “enactive” instead of “indexical”. Computers we are using nowadays basically inherit and further develop those features.

However, there are visions that are not achieved. For example, both Kay and Engelbart envisioned a scenario  where users to create, share and alter their own tools for media manipulation, but now people tend to use computers to consume media created by others. Programming language literacy is still only possessed by a small fraction of people. The reason of this setback may lie beyond technologies, but in economics or society. Kay envisioned his Dynabook as a medium for learning, experimentation and artistic expression not just by adults, but also children of all ages. But from what I have observed for children in their learning process today, computer is more distraction than useful instrument. The reason, I think, is that at least in China, average student is taught to use some certain software, but not systematic computational thinking as Jeanette Wing would suggest. A second reason is that students consumes media created by others more than create their own.

As I am learning and trying to do researches as a student, the interface I want to propose is about note-taking, for reading is one of my major tasks. For the time being, I am used to reading on paper book or computer while taking notes on my notebook by pen. I wonder if there will be more efficient ways. For instance, a note-taking software that can store my excerpts combined personal thinking as a whole and in the meantime link every point in my notes to the place where I generate it in the original material so that I can refer to it repeatedly later. This idea resembles the “hypertext” proposed by Ted Nelson, of which usually we only use a type of it–“chunk style” hyperlink from page to page.

Interfaceless Interface and Remediation

Well… maybe I really should wait till this week to talk about 3D manipulating in 2D virtual displays, or Pixar’s Graphical Interfaces revolution… But anyway, I find something new so provocative that can’t wait to discuss.

Interfaceless interface

“…no recognizable electronic tools-no buttons, windows, scroll bars, or even icons as such. Instead the user will move through the space interacting with the objects ‘naturally’, as she does in the physical world.”(Jay D. Bolter, Richard Grusin, 2000, Remediation: Understanding New Media”)

This Ironman’s Jarvis-like, transparent interfaceless interface will definitely be my proposal that I would like to see in our PCs and mobile devices. By realizing that, Engelbart’s “view control” shall be implemented in virtual reality. Also, the immediacy is vital. To realize immediacy require high-speed data transformation. (I read somewhere that once the 5G network is implemented, data transformation speed can literally leap so far that cache or image delay will be no problem anymore. So this future is actually foreseeable, fortunately.)

Right now, we can simulate this future by using VR glasses, which in my point of view is a highly hybrid media. In Manovich’s word, “Medium=algorithm+a data structure”. Now the trend is to hide the algorithm as deep as they can so they can only display useful data to users, in beautifully graphical ways. Here’s a VR shopping commercial released last year. This seems to be an example on how well the so-called “interfaceless interface” has been realized so far.

BTW, I would also propose the way how SQUID used in Strange Days to be implemented someday. Sending messages directly to people’s cortex seems like the next step after VR technology.


“…the remediation of one medium in another.” (Lev Manovich, 2013, Software takes command)

This is my favorite part this week. By “remediation”, Bolter and Grusin refered to new media simulating or re-mediating the old ones, so as the computer. How does that work? They wrote, “the goal of the computer graphics and is to do as well as, and eventually better than, the painter or even the photographer.” Remediation can be understand as the logic of new media in this case.

Remediation also is different from simulation. The Memex, the Dynabook, or the Kindle, are all new mediums compared to hard copy books. During the development of the PCs or personal reading devices on this trail, inventors are exhausting their ability to make the e-book experience feel like real book experience. The color, the light, the page turning display, the note taking function……all the features in today’s e-book devices are invented step by step to simulate the real book. A sardonic commercial released by Ikea is a perfect example for my idea.

In this commercial, Ikea intentionally indicates the use of a real book is simulating the use of an e-book, although the fact is the other way around.


Remediation or simulating, either way, human beings are somewhat nostalgia in this field. That’s might be exactly why we are looking forward to an “interfaceless interface”, cause in that way, the interacting will be natural, like the physical world which we have been interacting for god knows how long.

media refashion meanings

We have talked a lot about semiotic systems. According to Peirce’s triadic model, the “interpretant” process weighs a lot, almost the most, in the whole meaning-representing system. Contexts, collective memories, social traditions, cultures, personal experience and so many other factors decide the meaning eventually emerging in receivers’ minds. This week’s readings, however, reminds me of another part that plays an increasingly critical roles with the update of technology in our times in this whole semiotic process, which is, the SIGN (the vehicle, the signifying element) per se.

We have created many kinds of media to carry meanings in the previous five thousand years’ civilizations of human beings. This stable quantitative accumulation seems to result in a qualitative change, sharply in recent decades stimulated by the computer and information science and technology. In Software Takes Command, Lev Manovich coins the term “metamedia”, referring to the software-sized media on the basis of all information, digital, computational and other technologies. What he emphasizes in the book repeatedly is that, it’s new media more than “remediating”, or “imitating” the old ones, but the fundamentally new semiotic and technological system which includes most previous media techniques and aesthetics as its elements (P81). In short, the metamedia collects all features of old media and generates into new characteristics, enabling to hold all the already-existing, as well as not-yet-invented media in the future.

If we compare the text-processing software with traditional text media, for example MicroOffice Word with a notebook, we may get how unprecedentedly the metamedia functions. Both of them can be used to write articles. By Word, the author can easily delete a passage when he reviews previous works, insert a new one, change the order, search a paragraph with key words, or use different font and size of the characters as much as he wants, all of which however, cannot be done on a notebook. We’ve all seen how a master’s manuscripts look like — they are filled with messy characters, lines, marks, corrections and deletions. It can’t be searched, even very hard to be recognized, but what a notebook keeps is the process of the thoughts forming — all errors and changes are kept, as well as the trajectory of great minds.

The result of Word is a neat, accurate, mature text, like an ending point of the writing activity,  while what the notebook presents is a whole developing process. Therefore, even though we assume it’s the same author writing the same article just by different media, clearly what we eventually get is the same outcome, but contents and meanings in that notebook and the software Word are hardly to be regarded as the same.

Such words said by Marshall McLuhan are mentioned in one reading:

The medium is the message.

This can be better understood with Bolter & Grusin’s works Remediation. In this book, authors raise terms “immediacy” and “hypermediacy”. To my understanding, these two indicate two contrary features of medium that the new media now is trying to realize — the transparency of the medium, in contrast to the extreme expression.

For the former, kindle is a good example. This digital book reading device, launched by Amazon years later than people start reading online, is a new media with the desire to embrace the oldest text-vehicle — books. The appearance looks like a book and the screen is as fade as paper, of course with some modern digital functions such as text-searching and downloading. As a modern reading device, kindle in fact is trying to minimize its existence as a digital medium, trying its best to bring users the feeling of reading an authentic, printed book.

For the latter, we can compare those novels with their adopted films. In the film the Grate Gatsby(2013), the story is authentically represented in general. However, traditional film elements like music, camera language, gorgeous settings, as well as new techniques such as large numbers of digital graphic effects, all make this film contain more or less contents than the original novel. The art designer’s understanding of the book results in the setting of the film. The director’s understanding results in the camera and editing. In this case, the medium extremely expresses itself, and the medium itself brings about new meanings (and loses some as well). 

The medium is the message?

Well, according to Bolter and Grusin’s ideas, some of the medium is, like the modern film adopted from novels. While some maybe not, like the kindle.

Why the medium is the message?

Because some medium itself contains a process of understandings, like the modern film adopted from novels. Also, none of the medium we use is naturally existed in accord with our cognitive capacities. All of them are created with certain contents and boarders. Each language has its own semantic fields, leading to each nation’ s different cognitions towards external world. Other symbolic system, including maths, dancing, paintings etc., also has its own limited  coverage of meanings and emotions, which brings certain messages coming with signs of the system beyond contents.