One of the more foundational ideas I’ve been playing with in my mind throughout this course has been the concept of the popular arbitrary separation between “man” and “machine.” This false dichotomy is everywhere in our culture, predicated on the belief that humans are “natural,” machines/computers/technology are “unnatural,” and that there is fundamental and necessary split between the two. Of course, as we’ve seen, this distinction is misguided, as the technology we create—even the more advanced kinds—is a product of our cognitive capabilities, our cultures, and our values.

As we saw a couple weeks back in the readings about the “extended mind,” one function of our technology, whether intentional or incidental, is to free up cognitive and/or physical “space” for us to focus on other things. We can “off-load” our thoughts into pen and paper, or typewriter and paper, or a word processor, allowing us to recall them and build upon these thoughts even further. This is similar to Licklider’s concept of “man-computer symbiosis”—since so much time would otherwise be spent completing calculations or engaging in technical thinking, it makes sense to have a machine that can, in a sense, do this thinking for you, while you focus on something else. [1] Because computers can easily be programmed to engage in mathematical thinking (as opposed to, say, replicating emotions), then naturally the computer should do this “heavy lifting” while humans press onward into new territory, and don’t need to get bogged down in the math. It is also similar to Englebart’s idea of “augmenting human intellect,” in that there is an interaction between human and computer to produce a quicker, easier, or more accurate solution to a pre-defined problem. [2]

It’s clear how this concept can be applied to problems involving mathematics, science, engineering, and other technical fields. But it can also be useful for non-computational elements, especially in a time of vast and overwhelming amount of information. One very simple example, drawing from my own personal experience, is how we use browser bookmarks to keep relevant links on our “home” Internet browser screens. As someone who’s just about surgically attached to my laptop (and metaphysically attached to the Internet) I keep a huge number of sites bookmarked. The “off-loading” function of this is two-fold. One, it is more convenient and saves time, off-loading the task of typing in a URL address and manually navigating to the website to the computer itself. Two, it off-loads the task of remembering all these relevant websites, which takes up valuable memory in my OS Alpha. Otherwise I might forget them, and even if not, I now theoretically have more “space” in my brain for other cognitive tasks. This kind of off-loading is more and more prevalent in a world of overwhelming amounts of information (see: filtering, curating, personalization, etc).

In a way, this is also like Bush’s vision of how intellectual knowledge could be collected and shared. In reading his description of a knowledge accumulation machine, as well as the pieces discussing his ideas (including the “memex”), I immediately thought of Wikipedia: a collaborative place to compare pieces of knowledge in a way that is associative rather than alphabetical like a typical encyclopedia, with hypertext links to other relevant pages. [3]

One last thought I had, tying this back to the “man/machine” dichotomy, was the futility of trying to find a clear separation between human and machine when using a piece of technology. In such an instance, how much work are you doing, and how much work is the technology doing? Who or what gets the credit for doing it? When you sit down at a computer to fill out an online form, you are providing the cognitive power to think through the questions, and the kinetic power of your fingers typing and hands moving. But it is the computer that processes all the information from the commands and keys you hit. Neither can exist without the other; without the computer, you simply do not have the technology to fill out the online form (no output). Without you, there would be no impetus for filling the form out, or cognitive/technical ability to do so (no input). Granted, there’s the potential for AI to provide the cognitive function of this equation, but that’s another story. In this instance, the computer is augmenting human intellect and capabilities. The computer and human are interfacing at the point at which they are able to “collaborate,” in a sense, on this project of completing the online form.

References

[1] Licklider, J.C.R. 1960. “Man-Computer Symbiosis”. New Media Reader. Wardrip-Fruin, Noah, Nick Montfort, ed.. 74–82. Cambridge, MA: The MIT Press, 2003.

[2] Engelbart, Dave. 1962. “Augmenting Human Intellect: A Conceptual Framework.” New Media Reader. Wardrip-Fruin, Noah, Nick Montfort, ed.. 93–108. Cambridge, MA: The MIT Press, 2003.

[3] Bush, Vannevar. 1945. “As We May Think.” The Atlantic, July.

This week’s readings were a good reminder that we’ve come a long way from the first days of computing. However, our products still hold on to concepts that were applied in the beginning – in other words, with each new piece of technology that comes out, it contains traces of everything that has come before it.

Our reading, particularly “Man Computer Symbiosis” by J.C.R. Licklider, made me think of the way that technology in airplane cockpits has changed over the years. I am familiar with smaller airplanes, and I often hear the “glass cockpit” vs. the conventional, analog, “round dial” cockpit argument.

Take, for example, the Grumman Tiger that was built in the 1980s. The cockpit looks a lot like the picture on the right. All of the instruments are analog, and there is a radio complete with a dial to change the frequency. That is about as high-tech as the cockpit gets. This is an older airplane, but it works just fine as long as the pilot knows how to function in the cockpit. While the instruments tell the pilot his/her speed, altitude, amount of gas, etc., the pilot must know how he or she is going to get to the desired destination because there is no built-in map system. The pilot must know how to turn various dials and in what direction they must go in. The pilot must manually change the radio frequency to communicate with the tower before takeoff and landing. The pilot is required to input a certain amount of information, and the instruments work with what they’ve been given.
On the other end of the spectrum, airplanes are now coming out with “glass cockpits,” which have been around for a while (the military used them in the 1960s), but are just now finding their way into small aircrafts. A glass cockpit features electronic (digital) flight instrument displays (typically LCD screens), as opposed to the traditional analog dials and gauges. Because these displays are driven by flight management systems, the aircraft operation is simplified because pilots only have to focus on the most pertinent information, such as the flight path. Numbers are punched and data is processed. Essentially, a new interface has been added to the cockpit – the LCD screen represents what used to be the analog instruments. It is helpful to note, however, that there are still some back-up dial instruments that are not computerized (so analog is not completely obsolete).

There is a lot of debate as to which cockpit is A. easier to work with, and B. safer. Many pilots do not believe that the glass cockpit is a safe option because it is all computerized.

Licklider’s notes on man-computer symbiosis made me think of how this would apply to both analog and digital instruments. It seems as though in both cases, the human operator must supply basic information. However, in an older airplane with analog instruments, everything must be inputted manually (which often results in a slower process). It is very much a man working with the machine process. With the digital instruments, a person must still work with the machine. But in this case, there is much more automation and computing done by the machine, and less done by the pilot. As Licklider writes, in the instance of some computer-centered systems, the human operators “are responsible mainly for functions that it proved invisible to automate” (75). I see this statement being true in the case of pilots who work with digital instruments – the cockpit may be easier to work with because less information is required from the pilot.

Arguments in the pilot community often arise when talking about what happens when the instruments fail. Many argue that pilots who fly with glass cockpits are “lazy,” or not prepared for an emergency situation. If the digital instruments fail on a pilot, there aren’t a lot of choices left. Many small planes with glass cockpits come with a parachute, perhaps because of that assumption. In comparison, if failure is experienced in an analog cockpit, the pilot still has a chance to work with all of the other still-working instruments. The entire cockpit doesn’t go blank. And this takes me back to the idea of man-computer symbiosis.

Aviation technology has certainly progressed from the time of the Wright brothers and other early pilots and engineers. However, as technology has progressed, the relationship between people and computers does change. It makes life easier, for the most part. However, I question whether – when the computer, or instrument, stops working – whether that idea could still be applied.

References:

Licklider, J.C.R. 1960. “Man-Computer Symbiosis”. New Media Reader. Wardrip-Fruin, Noah, Nick Montfort, ed.. 74–82. Cambridge, MA: The MIT Press, 2003.

Irvine, Martin. 2016. “Introduction to Affordances and Interfaces: The Semiotic Foundations of Meanings and Actions with Cognitive Artefacts”.

When thinking about the history of computer design, I’ve always just assumed that the primary goals were overtly technical, such as facilitating complex mathematics, decoding information, and so on. However, reading this week’s selections I was surprised to learn how many of the early computer design concepts were focused on the ideas of communication, organization and efficiency. It seemed that most of the early conceptual models were trying to establish a centralized tool for storing, consolidating and retrieving information in a manner that was easy for users to understand and access quickly.

However, what I found even more interesting is the way in which some of the fundamental design concepts, such as selection, recursion and indexing, serve both technical functions and practical functions at the same time. In other words, last week we learned how in coding we can program variables and create indices. From these indices we can select data to build complex abstractions that, through recursive actions, allow us to perform various computational functions. Those actions of course are often hidden from our view on the side of the interface with which we don’t always interact. However, between Bush’s description of selection (Bush), Sutherland’s description of recursion (Sutherland 118-9), and Engelbart’s description of indexing (Engelbart 99-100), I was able to see how we actually use these same concepts for organizing and manipulating data on the side of the interface with which we regularly interact.

As for continuing and improving computer design, the first thing that came to my mind in this regard is the mouse. On one hand, it forces users to perform an interaction that is relatively unnatural to the ways in which humans typically indicate things. Rather it is more like the motion of wiping down a surface than pointing to something. It’s only through the visual interface on which the mouse’s corresponding arrow is displayed that we are able to understand the significance of the motion used to move the mouse. Our reliance on this correspondence is most clearly revealed in the moments when the arrow doesn’t respond to the user’s movement, causing them to do things such as pick up the mouse, turn it over or click it repetitively.

Rather than performing than performing the odd motions that the mouse requires, I began to consider the possibility of using touchscreen technology as some tablet PCs have already begun to do, since using one’s finger to manipulate objects displayed on a digital interface provides stronger affordance to users (Irvine 1-2). Specifically, I wondered why, if touchscreens have already become the norm for cellular phone and tablet interface design, have we been so slow to standardize this technology for PCs and render the mouse obsolete ?

As far as I can tell, it seems to be a matter of precision, application development and ergonomics. To begin, even though it seems that the overall constraints of using one’s finger versus using a mouse to perform actions on a PC interface are the same (Irvine 2), the pointer to which a mouse corresponds allows more precision than our fingertips can. In particular, as the mouse’s pointer functions as an internal component of the interface, it can be significantly and consistently more precise than our fingers, as external tools acting on the interface. Of course, I don’t think that this is something that couldn’t be remedied; however, I think that in addition to improving the precision of touchscreen technology, applications would also require redesigns in order to facilitate touch activations more easily. A primary issue in this regard is that there is no standardized finger size, while the pointer for a mouse, on the other hand, is standardized. As a result, the dimensions of application buttons would need to be redesigned in order to accommodate the possibility that some people might have large fingers, which might mean that the look of a PC interface would need to change significantly.

There is also the possibility that a mouse is simply more comfortable to use because it doesn’t require the user’s arm to be elevated in order to reach the screen. Rather, the user can rest his hand on the desk while using the mouse. This also serves to stabilize the user’s hand and provide additional stabilization. When I think about the design of the mouse in this sense, I’m not sure that it should become obsolete or rather, it doesn’t seem that touchscreen indication is the most ideal design evolution for the PC.

So how could interaction with the PC interface be improved? The next thought that comes to mind is through voice commands. While I’m personally intrigued by the possibilities that advanced speech recognition holds for word processing, the potential it holds for interaction with the general PC interface seems more complicated. In particular, while it would surely relieve the user of performing functions with his hands, voice command control could potentially cause an unwanted burden of learning when interacting with the computer interface. For example, when using computer programs, we take for granted the large amount of data with which we interact, that we don’t really understand. For example, I know what all of the buttons in my word processor’s toolbar do, but I am not able to tell you what most of them are called. However, if I were to rely on voice commands in order to utilize any of these functions, I would be forced to learn the names of these functions in order to use them and communicate that to the computer. That may not be too daunting within one program, but think of all the different and new programs I may want to use and the changes that may result from any future updates. Utilizing an indicating tool, such as the mouse, allows me to understand and interact with these functions rather seamlessly without storing the additional information of their names.

Again, this issue could be solved by redesigning applications and internet browsers in order to facilitate voice commands. However, even if such redesigns were desirable, there are numerous social implications of using voice command for PCs . In particular, the office environment would become incredibly chaotic without developing some sort of barriers to the various voice commands that would be floating throughout the office. In an era in which we are finally moving away from the cubical, it seems that such a development would actually inhibit the advances in business communication and collaboration that have recently been acknowledged as beneficial. In this sense, the overall efficiency that is gained through voice command features for PCs might be less than the efficiency gained through open work environments.

All of that is to say that I’m not sure how the PC interface could be redesigned, unless we could develop a way to truly achieve “Man-Computer Symbiosis” (Licklider) and control computers through some sort of unspoken cognitive functions. Or, perhaps, there’s a way to interact with the computer via visual cues, such as installing a sensor on the screen that can track the focus of one eye and then respond to blinks in the same way we use clicks. However, that would probably require a large amount of additional buttons within programs that we would need to activate certain functions (I’m thinking of things like highlighting text with a cursor), so I don’t know if it would be more efficient or not. And while it may free users from developing carpal tunnel, I’m not sure if most people would trade that for a twitch!

Bush, Vannevar. 1945. “As We May Think.” The Atlantic, July.

Engelbart, Dave. 1962. “Augmenting Human Intellect: A Conceptual Framework.” New Media Reader. Wardrip-Fruin, Noah, Nick Montfort, ed.. 93–108. Cambridge, MA: The MIT Press, 2003.

Irvine, Martin. 2016. “Introduction to Affordances and Interfaces: The Semiotic Foundations of Meanings and Actions with Cognitive Artefacts”.

Licklider, J.C.R. 1960. “Man-Computer Symbiosis”. New Media Reader. Wardrip-Fruin, Noah, Nick Montfort, ed.. 74–82. Cambridge, MA: The MIT Press, 2003.

Sutherland, Ivan. 1962. “Sketchpad: A Man-Machine Graphical Communication System.” New Media Reader. Wardrip-Fruin, Noah, Nick Montfort, ed.. 109–26. Cambridge, MA: The MIT Press, 2003.

First of all, I am not saying Echo is a great product and I actually don’t have one because personally I don’t really see the point. But I have to note that Echo is definitely something I did not perceive as the next step in technical advancement. In my mind, the so called “next step” looks more like VR technology where humans directly interact with the symbolic environment, touch, draw, and create new layers of abstractions without any physical constraints like this video:

Voice control, on the other hand, is somewhat imperfect in my opinion because there are so many memes about “Siri fails,” and frankly it looks kind of dump when people are using it… like echo:

Anyways, back to this week’s topic. The readings reveal that the evolution of interface and interaction is not a matter of one way of mapping replacing another, rather a process of softening the divide between hardware and software. For instance, the keyboard we use to type on our smartphones are based on real life keyboard, and that keyboard is just a step forward from typewriters.

In the case of VR and voice control, both technologies fit the description yet have the opposite approach. VR maximizes its affordance by creating a neutral language between human and machine via a “friendly” interface. Voice control technology maximizes its affordance through making the machine function more like human by teaching them syntax and semantics. In my opinion VR is better than voice control because it’s more friendly and direct, which allows me to do more things. After this week’s readings, however, I think voice control might be the future.

Dictation technology exist long ago and I remember my old desktop PC has Dragon Dictation installed when I was learning English. It wasn’t a big deal for commercial purposes until Apple introduced “Siri,” then comes “Cortana” and “Alexa” (All have default female voices and feminine names btw). It seems that modern technology is able to make voice control software more reliable through a lot of engineering and semantic training. And Apple is pushing this idea by implementing Siri to laptop OS updates (Sierra) and creating Apple Watch, which has a small interface so that users are encouraged to give voice commands. Amazon’s Echo pushes this concept further by elimination a visual interface all together, forcing the users to directly “interact” with the machine.

This is where things get interesting, human language has historically been one of the reasons for the separation between hardware and software, because the human language is too nuanced for machine, and machine code is incomprehensible to regular humans. The voice interface between machine and human makes the dynamic between human and machine more natural because it allows machine and human to directly interact at the symbolic level without any physical constraints. With further development in voice control, we might be able to do so much more because we can offload all the complex syntax building labor to machine and directly engage new ideas. So imagine one day we can code using just our voice rather than earning a degreed in computer science or intensive training on Codeacademy.