Category Archives: Week 12

Software lies underneath everything that comes later

As we have learned by studying different concepts throughout this course, there is no “magic”, when it comes to the world of technology and computers.  But somehow, it is so hard for people to understand how something works and why does it work in that specific way? While there are many theories and research on fields like cognitive sciences and psychology,  that can come with different explanations to the human brain and how we perceive information, I highly believe that by living in a world where we are consumers, by living in a consumer culture, we have lost the sense of participating in the process of building things, we now can just buy what we need, and just make sure that the things we buy work, and never worry about how those things work.

Today, you hear about Iphone X and the “new amazing features” that the new phone can offer to it’s consumers, or maybe you looked at the new Apple Macbook Pro, or Microsoft’s Surface laptop with new improvements and more ways to make it interactive.  So many new things, and in order to participate in the discussions happening in social media (because who doesn’t want to share their personal opinions with the world) and let the world know how “in” they are with the new technologies, you have to buy the newest products, because everyone else seems to use them, and you don’t want to stay behind, right?

I have done that mistake too, and part of this is because you never actually see what’s happening behind the visible layer, what’s behind that blackbox. To cite Bruno Latour, blackboxing is “the way scientific and technical work is made invisible by its own success. When a machine runs efficiently, when a matter of fact is settled, one need focus only on its inputs and outputs and not on its internal complexity. Thus, paradoxically, the more science and technology succeed, the more opaque and obscure they become.

Everyone knows about the new features, but I doubt that people actually know the history of how these new features were invented? And where did they come from?

Lev Manovich, in his book “Software takes command” makes the point that industry is more supportive of the new innovative tech & applications than academia is. Modern business thrives on creating new markets, new products, and new product categories.

But to analyze his point, new discoveries almost always don’t include new content but rather new tools to create, edit, distribute and share this content. To add new properties to physical media, it requires to modify it’s physical substance. But since computational media exists as a software, we can add new properties, new plug-ins, new extensions, by combining the services and the data.

Software lies underneath everything that comes later.

So, the next time you hear about the new cool features of a new product, think of the branding and  marketing side of it.

Ted Nelson and his idea of software, as mentioned in his article Way Out of the Box

“In the old days, you could run any program on any data, and if you didn’t like the results, throw them away.  But the Macintosh ended that.  You didn’t own your data any more.  THEY owned your data.  THEY chose the options, since you couldn’t program.  And you could only do what THEY allowed you — those anointed official developers”. This is a quote by Ted Nelson, in his article Way out of the Box.

In his article, Nelson brings to our attention all the possible ways that we can do things. Just because some companies (Apple and later Microsoft) took the paper simulation approach to the behavior of the software, doesn’t mean that that is the only way to do it. They got caught up to the rectangle metaphor of a desktop, and used a closed approach. Hypertext was still long rectangular sheets called “pages” which used one-way links.

Nelson recognized computers as a networking tool.

Ted Nelson’s network links were two ways instead of one-way.  In a network with two-way links, each node knows what other nodes are linked to it. … Two-way linking would preserve context. It’s a small simple change in how online information should be stored that couldn’t have vaster implications for culture and the economy.

This is an example that demonstrates not to get caught up by the whole computer industry, as software gives plenty of possibilities to look at new ways to implement, rather than just believing and thinking that there is only one way.

Alan Key’s idea of a computer as a “metamedium”, a medium representing other media, was groundbreaking. It is the nature of computational media that is open-ended and new techniques will be invented to generate new tools and new types of media.

Vanneva Bush’s article “As we may think” in 1945, discussed the idea of the Memex, a machine that would act as the extension of the mind, by allowing its user to store, compress and add additional information. It would use methods of microfilm, photography and analog computing to keep track of the data.

You can clearly see the metamedium idea at the Memex.  The second stage in the evolution of a computer metamedium is about media hybridization, which as Manovich explains, is when different medias exchange properties, create new structures and interact on the deepest level.

It was Douglas Engelbart who recognized computers not just a tool, but a part of the way we live our life. The mother of all demos, demonstrated new technologies that have since become common to computers today.  A demo featured first computer mouse, as well as introducing interactive text, video conferencing, teleconferencing, email, hypertext, and real time editing.


All these examples make you think about different ways that software could behave and interact, and how these pioneers continued to push their tools to new limits to create creative outcomes, even without having access to the technology that we have today.

It really is inspiring to look at their work and understand that sometimes it is us who creates limitations to our technology, sometimes pushed by the computer industry and other factors, but it is crucial to understand that there are no limitations to the development of software and graphical interfaces in order to create new ways of human computer interaction (HCI)


Bush, Vannevar “As We May Think,” Atlantic, July, 1945.

Engelbart, “Augmenting Human Intellect: A Conceptual Framework.” First published, 1962. As reprinted in The New Media Reader, edited by Noah Wardrip-Fruin and Nick Montfort, 93–108. Cambridge, MA: The MIT Press, 2003.

Latour, Bruno“On Technical Mediation,” as re-edited with title, “A Collective of Humans and Nonhumans — Following Daedalus’s Labyrinth,” in Pandora’s Hope: Essays on the Reality of Science Studies. Cambridge, MA: Harvard University Press, 1999, pp. 174-217. (Original version: “On Technical Mediation.” Common Knowledge 3, no. 2 (1994): 29-64.

Manovich, Lev. Software Takes Command. New York: Bloomsbury, 2016. Print.

Nelson, Theodor Holm.  WAY OUT OF THE BOX. EPrints, 3 Oct. 2009. Web. <>.

Thoughts on Google Art Project

Personally, I’ve never heard of Google Art Project, and I prefer physically to be inside museums and see paintings physically in front of me, which always thrill me because I am able to witness the masterpiece by myself. However, after I searched and opened Google Art Project, I found it beyond my expectation since I can receive recommendations and feature stories about the type of arts I like, and I can have more informed choices of seeing arts. For example, I am a huge fan of Picasso, so when I searched his name on Google Art Project, it presented me his biography and critiques of his work, which I find very useful for amateur art viewers like me to be more educated in terms of art history.


On a semiotic level, what I can derive from the layout and contents of Google Art Project is that it is very different from the traditional art space we see. In the traditional art space, albeit the innovation made this year trying to make traditional art more relevant among young people and geeks, it is a secluded place where you walk in and feel the space and time is halted. It is designed to make you feel somewhat isolated and only talk about and think about art. Whereas Google Art Project is designed in a way consistent with almost every pop culture related, trendy website. It does not seek to exclude, rather it strives to be included.


Postmodern problem led by metamedia

From “0s” and “1s” to literal commands and till several years before, the graphic user interface revolutionized the use of computer. Basing on a symbolic system (human collective mind), computer and other electronic devices becomes better interaction medium for human to communicate with program. When looking at the two classic guidelines on graphic interface design, we can see that the user(human) always takes the initiative, which is coordinated with the computing thinking idea.

With the development of digital technology, the electronic devices we use every day like computers and smart phones are becoming more and more powerful metamedia, being able to present and transform other media more efficiently and with better quality.

Certainly, we are not stop on the graphic user interaction, more ways and higher level of senses hybridity of interaction based on human body and social collective mind are being developed and all these meta media are working more and more powerful? So is it possible for us to build up a sophisticated enough meta media system that interacts perfectly with human?

From my perspective, the idea of HCI and the technology development in the field of metamedia and HCI accelerates  the generation of “simulacrum” (Jean Baudrillard, 1988). Just as what has been done to human society by Disneyland, “it is no longer a question of a false representation of reality (ideology), but of concealing the fact that the real is no longer real ” To me, the HCI technology and the changing idea on computer makes this postmodern problem that is cited by Baudrillard in Simulacra and Simulations more ‘touchable’ and ‘reachable’. Remember what’s happening in The Matrix, when the computer is being able to generate a whole word and get people involved in it without being realized of getting involved in, how would people react to this world? What shall we(as real human) do when we don’t know whether we are communicate with a program or a real human mind?

Still remember how you feel the moment you leave Disneyland? Lost?Sense of alienation? In this way, it might be reasonable for me to speculate that even being conscious of getting trapped in Matrix, some of us might choose to stay in this data world. What has been true in the human society is that even without the fabulous VR technology, millions of people had gotten addicted to the traditional video games and just cannot get themselves out and back to the real world.


Manovich, Lev. Software Takes Command. vol. 5;5.;, Bloomsbury, London;New York;, 2013.

Irvine, Martin. “Key Concepts in Technology: Week 8: Computers from Information Processors to Metamedia.”

YouTube, YouTube, 18 Oct. 2014,

Wilson, Stephen. “The Aesthetics and Practice of Designing Interactive Computer Events.” The Aesthetics and Practice of Designing Interactive Computer Events,

Baudrillard, Jean, and Mark Poster. Jean Baudrillard: Selected Writings: Chapter8 Simulacra and Simulations. Stanford University Press, Stanford, Calif, 1988.

Human-Computer Interaction

Once a friend of mine told me that when she changed her laptop from Windows to Apple, she suddenly realized PC is just a computer, but MacBook is a tool for humans. How did she jump to this conclusion? The main reason was that of the strong bond of human-computer interaction, especially various gestures applied to a touchpad. According to her, it was much easier to manipulate a touchpad on MacBook than on other PCs, and she didn’t even need to use a mouse anymore. This is a small daily example to show how the design of human-computer Interaction influences users’ senses of control.

Recently, I was dealing with HCI for a period of time because my group in 506 picked a projection keyboard as the technology we focused on. Basically, a projection keyboard enables users to type on any flat interfaces at any time they want. It can be applied to more using scenarios, such as when hands are dirty or high portability is required. It is evolutionary and pioneering in transcending the boundaries of a physical keyboard and its typing function and realizing typing freedom to a larger extent.

The reason we chose this technology is that we regard it as a representative of a tendency of HCI, which is to liberate users from more and physical devices and empower them senses of control. However, it is no need to overstate how deeply a projection keyboard alters ways people interact with computers. A projection keyboard is not successful or universal as we expected, which might be due to the abundance of other means of interaction, like voice recognition system, motion capture, or facial recognition. These ways amplify strengthen and amplify the connections between humans and computers, making “we are always interacting.” to become true. In addition, the rate of typing accuracy impedes a projection keyboard to gain more market share. Therefore, a projection keyboard is an intermediate product in the development of HCI. It is not influential enough, but it does indicate its symbolic meanings.

An interesting fact on a projection keyboard is that when users type on the virtual keyboard, it will emit sounds mimicking real typing sounds to make users have a better experience.  This design can also be seen as a basic example of  “Augmented Reality“. Though it is not a real keyboard, it can make you feel like it is by augmenting different dimensions of senses.

Like the case of projection keyboards, Graphical User Interface shows how the design of interfaces increasingly concentrates on user-friendliness. In most occasions, they stem from the desires or requests of users or are good at discovering users’ potential needs. For instance, the evolvement of ways of interaction is from electrical, symbolic, textual to graphical. During this process, the thresholds of interaction fade away and it becomes more direct and suitable for people’s using habits after socialization. This alteration also fits with the trend to diminish the barriers of human-computer interaction and to make users play more positive roles in it.

Alan Kay’s design principles on meta-media, such as computers, still deeply affect big companies’ decision-making process, and make user experience design a significant and broad aspect that they need to concern about. UX design not only includes HCI but also involves a large range of users’ feedback and multi-dimensions of enjoyment. It could be another issue that we might need to dive into.

Collective mutation, individual evolution

Before diving into the materials for this week, I did have a hard time figuring out why the computer would be regarded as “metamedium”. Now I probably still couldn’t say I’m 100% accurate on the topic, but I do feel the power that computer has in absorbing all kinds of media and mediums, simulating, and transforming them, integrating them to form synergy, while providing them back with new possibilities and potentials. It’s like an incubator and a black hole at the same time, and that’s the truly fascinating part about computer and its “meta” characteristic.

Here’s one of my favorite quotes of the week:” Another important type of software epistemology is data fusion— using data from different sources to create new knowledge that is not explicitly contained in any of them. For example, using the web sources, it is possible to create a comprehensive description of an individual by combining pieces of information from his/ her various social media profiles and making deductions from them. Combining separate media sources can also give additional meanings to each of the sources. Consider the technique of the automatic stitching of a number of separate photos into a single panorama, available in most digital cameras. Strictly speaking, the underlying algorithms do not add any new information to each of the images (i.e., their pixels are not modified). But since each image now is a part of the larger panorama, its meaning for a human observer changes. The abilities to generate new information from the old data, fuse separate information sources together, and create new knowledge from old analog sources are just some techniques of software epistemology.”

There is really a great indication of the vitality that computer and all the architectures on top of it could bring to the areas and disciplines involved.

From my perspective, the procedure of HCI evolution and the “metamedium” ecology around the computer is also a history of the mutual education of computer and human – Before we get used to a smartphone with built-in camera, people would question the necessity of this idea: why would I need a phone that could take pictures? But now we are so used to using phones as our primary photographing tools and even handle a great part of media production on it. Again – using smartphones for PS and video editing is something that didn’t happen until smartphone as a platform digested camera as an appropriate unit and the hardware development entitled the platform with the capabilities to do so. And this trend might have – to a great extent – led to the popularity of SNS like Instagram and Snapchat.

This helical trajectory also reflects the power of the idea “metamedium”, it empowers, and it gets empowered.

Photoshop and Micro-Management

First we need to think about: if micro-management of software kills creativity and productivity of developers?

The question I think is no. On the contrary, micro-management of software is convenient and still needs human control. And also, micro-management of software accelerates and extends human’s creativity.

One of the key uses of digital computers from the start was automation. Computers can be programmed to execute all the established instructions, aka algorithm, without human input. However, from the user’s point of view, this low-automation still requires human’s command of interface. Many software techniques which can simulate physical tools need the users to control them manually, but from a micro prospective. In order to produce the desired effect, human beings should micro-manage the tools in the software techniques and they can’t be seen as totally automated.

I used Photoshop for several years. For me, it is such a great software. It’s convenient not only for creating a picture, but also for retouching the pictures. When I didn’t know this software, I always use the drawing software in the Windows system. However, it is not very convenient to use and all the functions are simple. The whole pictures can only be seen as a whole. On the contrary, the pictures can be retouched layer by layer in the Photoshop and the functions are more complicated and convenient.

Photoshop can be an example of the micro-management of digital software. The user has to micro-manage the tool and direct it step by step to produce the desired effect. For example, the user need to control the cursor in a desired pattern to produce a particular brushstroke using a brush tool. This function is the same as the drawing software in the Windows system. The new digital technology and software is more convenient in its new functions and tools. It offers higher-level automation of creative processes. The user doesn’t need to control every detail. Instead, they can specify the parameters and controls and set the tools in motion. For example, when drawing a rectangle, the user needs to draw it in detail in the previous drawing software in the Windows system. However, with Photoshop or Illustrator, the user can set the length and width of the rectangle and the software will generate it automatically for the user.

My favorite tool in the Photoshop is the Magic Wand. Before Photoshop was invented, it’s almost impossible to move part of the picture to other places. However, with the Magic Wand, the user can easily choose part of one picture with the same or almost the same color. If the user set different tolerance parameter, the range is also different.

Photoshop can also be seen as an example of human-computer interaction. The users interact with computer over this software interface using the given input and output (I/O) hardware. As to the method of the interaction, it is about micro-management. The user interacts with computer by micro-managing the software and giving input to the hardware and then it will generate output automatically following established procedures, namely algorithm. During these processes, the human-computer interaction will give the users much more convenience.


  1. Manovich, L. (2016). Software takes command. New York: Bloomsbury.
  2. Kay, A. C. (1972, August). A personal computer for children of all ages. In Proceedings of the ACM annual conference-Volume 1 (p. 1). ACM.
  3. Yong, K. F. (2014). Emerging human-computer interaction interfaces: a categorizing framework for general computing(Doctoral dissertation, Massachusetts Institute of Technology).

iPhone: a non-stop developing metamedium – Wency

iPhone: a non-stop developing metamedium – Wency

It seems to be something happened just yesterday where people wake up in the morning, carrying an MP3 on their way to work, going all the way to a company 10 miles away from their office for a conference, buying a newspaper or magazine from a newsstand after business, call their friend on telephone and complaining about what happened in the workplace, turning on the TV and crying for missing their favorite show.

People at that time might never have a chance to imagine that less than 20 years later, all the things I mentioned above can be done on a single, portable, fashionable device: the iPhone.

Take myself as an example, I listen to music on Spotify, apple music on my iPhone where I can almost access all kinds of music I can imagine. I have video conference or class on some extremely bad weather where I can enjoy the real-time face to face conversation just like what I have in person. I check newsify, twitter, Facebook and even Instagram everyday to see if there’s something new happened in this world (of course, not only news but a ton of gossips from a lot of celebrities). I snapchat my friend everyday with photo, video and add some words and emojis to draw their attention. I never wait in front of the TV in case I might miss the TV series or reality show because I’m sure that the same version would still be release on, for instance, YouTube, within a few days.

Never would those people imagine that those applications existing today are releasing new version almost non-stop: New functions, new GUI (like one application I’ve been using has just changed the way of displaying users’ profile recently), new interfaces added to connect to other third-party applications and new competitors offering similar but extended or refined functions emerging every day.

The emergence of these new media (digital, or software-based media), thus brings out a new word, i.e., metamedium, while according to Manovich, computer is the first metamedium, all these digital devices including smart phones, PC, iPad, etc., can be defined as metamedium which is a combination of existing new, and yet to be invented media, a media for presenting, transforming other media (Manovich, 2016, p. 335; Irvine, 2014). The fourth mode of interface, i.e., graphic user interface (GUI and the first three are electrical, symbolic and textual), turns today’s digital computer into a remediation machine which continues the bridge connecting the previous media and computational media and further, users can control this graph to display different amount of information (Manovich, 2016, p.58-59, p.73).

The beauty of the idea to use software to implement and realize the function of media is the flexibility and high efficiency: we can add new properties or even invent new types of media by simply changing existing or writing new software, therefore, new media is new because new properties (i.e. new software techniques) can always be easily added to it (Manovich, 2016, p.92). Besides, the definition of medium, in terms of software, is now turned into the sum of algorithm and a data structure (Manovich, 2016, p.207), in other words, all previously existing and newly invented mediums share some common properties, they rely on a set of common software techniques for data management, authoring, and communication (Manovich, 2016, p.123), therefore, the material itself of different media becomes less important, it is rather the universal media machine that makes all seemingly different platform happen easily, just as in the conversation between Apple and Microsoft when they both share a same rich uncle, it is thus the same source which is later reimplemented that makes the magic happen (Manovich, 2016, p.70; Irvine, 2014). Besides, since the manipulation of data, code is much faster than that on the physical materials (although the actual operation of computer is relied on the physical hardware), the update and release of new version of media on the computing devices is thus much faster than it used to be, the whole sequence from user, user interface, algorithm and database is highly organized and connected which thus enables multiple factors and contributor to participate into the design and updating processes, such permanent extensibility thus enables software-based media to always be new (Manovich, 2016, p.156).

Besides, as Engelbart mentioned in the earlier years when he believed that properly trained and with the right computer tools, we could raise out collective IQ, namely, computer is not simply a tool, but also a medium through the interaction with which we are therefore able to gain an increased capability to approach a complex problem system (Englebart, n.d.). As a metamedium, computer not only offers us an integrated platform involving all the previous medium, but also, it detects the affordance latent in the environment which was used to be considered impossible. Meanwhile, with the constant update of technology and interface, users are also required to adjust and work with the technology which in turn helps them develop the augmentation means (Englebart, n.d.).

Another interesting point about computer as a metamedium is based on its technical aspect where the interaction between user and computer is becoming more and more important in that the application layer lies on the top of the hierarchy of computer. According to Murray’s explanation of HCI (human-computer interaction), users are not merely consumers or audience, but also are invited into the participatory design process (Murray, 2012, p.64, p.71). This, on the other hand, introduces the situated action theory where it is not enough to merely see users as interchangeable processors of information, who all behave the same way, but as uniquely positioned, complex actors whose frames of reference is shaped by the surrounding social and material world (Murray, 2012, p.62). Take Twitter as an example, while we are checking the news on Twitter, our opinions are not only easily affected by our friends, families, schools, workplaces, but also those real time interactive comments by other users which to some extents serves as another source of information and shapes our interpretation of the news. Thus, computing devices, together with the whole complex social, technical network, makes such metamedium not only an umbrella which contains and manipulates all other kinds of media, but also a hybrid one where information is exchangeable and inter-related (Manovich, 2016, p.170).


  1. Engelbart, C. (n.d.). Augmenting Human Intellect: A Conceptual Framework – 1962 (AUGMENT,3906,). Retrieved April 07, 2018, from
  2. Irvine, M. (2014, October 18). Retrieved April 07, 2018, from
  3. Manovich, L. (2016). Software takes command. New York: Bloomsbury.
  4. Murray, J. (2012). Inventing the Medium: Principles of Interaction Design as a Cultural Practice. Cambridge, MA: MIT Press, 2012.
  5. Iphone App Development India. (n.d.). Retrieved April 07, 2018, from
  6. (n.d.). Retrieved April 07, 2018, from images&source=lnms&tbm=isch&sa=X&ved=0ahUKEwjV1Mf6t6jaAhWKx4MKHf9yCa4Q_AUICigB&biw=1422&bih=629#imgrc=vESOgQx7QDx_gM:
  7. Explainer: What is Snapchat? -. (2018, February 20). Retrieved April 07, 2018, from
  8. Deorukhkar, P. (2017, September 22). At a Glance: Newsify’s New App Icon • Beautiful Pixels. Retrieved April 07, 2018, from
  9. (n.d.). Retrieved April 07, 2018, from images&source=lnms&tbm=isch&sa=X&ved=0ahUKEwi6-5LNuajaAhWJ0YMKHd_7D2MQ_AUICigB&biw=1422&bih=629#imgrc=5c-lHMgimz0SzM: