Category Archives: Week 9

Dreaming, Gaming, and Design

Warning: Use of undefined constant user_level - assumed 'user_level' (this will throw an Error in a future version of PHP) in /home/commons/public_html/wp-content/plugins/ultimate-google-analytics/ultimate_ga.php on line 524

The link gaming and computing has been around since the early days of innovation in the computing field. The reading this week made me really sit back and think about the statement that “Before the 1980s, no one in the computer industry imagined that there could be a major consumer or small business market for computers.” (Irvine, 2018) This sentiment is something I have seen and heard many times over the years. Sure, breakthroughs in design had to occur for us to reach the ubiquitousness and power of the technology we use today. However, design is inherently not a practice one undergoes without a vision, goal, or at least motivation in mind.

Thinking back to the first truly significant compression of semiotic media into a quickly accessible format the creation of microfilm and microfiche came to mind. While initially a hobby form of photographic art, the reduction of visual images into small format that could be put behind a lens for consumption by the viewer was quickly recognized as a benefit by engineers, who frequently had to reference thousands of pages of engineering notes and specifications. In the 1920s and 1930s the practice of saving documents and particularly newspapers became institutionalized by the Library of Congress, the New York Times, and Harvard University, who all began creating repositories of important documents and newspapers on microfilm.

While Douglas Engelbart’s work to build a system that would allow for workers to augment their intellect was truly impressive in what it achieved, the underlying concept of amplifying the power of a single person to access and benefit from information without leaving their desk had already been a goal for many many years. The desire beget the design: access a great deal of information while sitting at a desk. This was true of the engineers and later of the libraries who installed very familiar looking readers:


(1) Ames Public Library, 1953. (2) 1956, Old Post Office building, microfilm reader, San Jose Public Library Collection

For me, the desire to have individual access to information at a desk, clearly laid the foundation for work to gradually improve the design of computers from the room sized early IBMs to the step by step, reduction of size and convenience. Early news pieces about microfilm in the 1930s talk about how one day we would all be able to have microfilm readers the size of a wristwatch through which we could read the newspaper… sounds like a clear goal for a current well known product in my opinion.

Another fuel for computer design and innovation I see is gaming. As mentioned some of the earliest games were in fact simply demos of computing hardware and concepts. The first computer games, Bertie the Brain (left), and Nimrod (right), seen below had little to do with gaming, they were demos of underlying computing technology on display at the Canadian National Expo in 1950 and the Berlin Festival in 1951, respectively.


While the companies behind them were set on demonstrating the power of the technology, spectators just wanted to play the game. Combine this focus by consumers on the power of technology to entertain, and we see the potential of consumers to desire electronic gaming in their homes. Early console systems like Magnavox Odyssey and Atari, brought computer based technology into the households of many consumers years before an affordable personal computer had been released. Soon the value of a computer for individual personal use beyond gaming was recognized and companies worked quickly to design the hardware and software needed to support that. The productivity was the goal, the personal computer was the design solution to the problem.

Looking ahead at the obstacles placed in front of users to improve on the design of just a few companies, I see most hope in the gaming community. I am fairly confident that an analysis of all custom built personal computers made this year would be overwhelmingly owned by gamers. These are your power users who demand more than what they can buy in a blackbox product off the shelf. They are the ones still creating components to give them an edge. But it isn’t just about competition.

Many gaming companies themselves have come to accept their own limits in designing games. Now most games on the popular Steam platform have communities of “modders” who design and code mods, or patches, for games that have been released. One of the most famous of these mods is for XCOM: Enemy Unknown by Firaxis Games. The Long War Mod, is a free add-on, yet is so extensive that it turns the underlying experience into an entirely new game. With over 840,000 downloads to date, the mod has greatly improved the sales success of the base game. In response, when releasing the sequel, XCOM 2, Firaxis Games included built in support for modding and modders. This includes access to base code and game algorithms.

While modding is still predominately restricted to the world of gaming, I could see its success resulting in more apps and companies opening up their software to user modification. This of course brings with it security and exploitation concerns, but could be an avenue for better products down the road.

Documentary: Alan Kay on the history of graphical interfaces:

Martin Irvine, Introduction to Symbolic-Cognitive Interfaces: History of Design Principles

Judging how the iPad App Flexcil Uses the Spatial Affordance of Interface Design

Warning: Use of undefined constant user_level - assumed 'user_level' (this will throw an Error in a future version of PHP) in /home/commons/public_html/wp-content/plugins/ultimate-google-analytics/ultimate_ga.php on line 524

As personal computing spreads from its PC origins and into tablets and smartphones, the common graphical design principles that governed early computing — notably the WIMP (windows, icons, menus, pointing device) model — are becoming less relevant when designing user experiences (Murray, pg. 73). As computing becomes more ubiquitous in daily life, designers do not need to be as reliant on a “desktop” metaphor when designing an application’s interface. In fact, when designers mix spatial affordance metaphors, the software’s interface can suffer because of it.

The spatial affordance of computing refers to how users feel as if the interfaces are not simply objects and images with which one interacts, but actual spaces and “sites.” (Murray, pg. 71). As mentioned before, the common way to represent this affordance on a desktop is through the WIMP model. However, the hardware constraints of a tablet make the WIMP model less relevant. While the WIMP desktop model and iOS tablet navigation share some navigational similarities, the main difference, to me, is that an application “takes over” the screen of the tablet within iOS tablet navigation. Once one opens an app, the app then becomes an entire environment in which you operate. Let’s take the Flexcil app for example, as its mixed spatial metaphors leave something to be desired.

After opening the app, the user feels like they are “in” the main screen of the app. This space of the app feels like the main hub, and anything you do within this space should reflect an operation upon this space.


To import files into this space, one navigates to a side menu and selects which file they want to import into the app environment. However, after one imports a file into the app space, it is nowhere to be found.

One has to navigate within the downloads file to find the imported file. When I write this process out, it  feels fine and not problematic. So, why does this process feel inorganic while using the app?


On a desktop, one would expect to navigate to a downloaded folder to find a downloaded file. But, because tablets so often use “encompassing environments” within their applications’ spatial analogies, it feels inorganic to have a file appear in a space that isn’t the one you’re currently visiting.

Design Structures and Computers

Warning: Use of undefined constant user_level - assumed 'user_level' (this will throw an Error in a future version of PHP) in /home/commons/public_html/wp-content/plugins/ultimate-google-analytics/ultimate_ga.php on line 524

Bolter and Grusin posit that the newest aspect of digital media is the strategies employed to remediate past media (Manovich 2013). This principle can also apply to computer design and the shift from governmental and business applications for computers to the layman application of computers and tablets. One clear design thinking step that impacted the shift in usage of computers was the recognition that computers were more than mathematics machines, they are symbol manipulating systems. This view, expressed by the Augmented Intellect Program, expanded the opportunity for computer processing, which lead to the production of  graphical interfaces that enabled manipulation of sociocultural signs and systems through computational interfaces and semiotics (Irvine). The perspective expressed by Bolter and Grusin enabled the practical and creative development of applications and tools for users to create film, music and other media. However, the change in purpose of computing does not address the shift in accessibility of computing from corporatized to personalized.The remediation of interface and application design in the 1960s was one update that has lead to the use of computing systems to be wide spread across industries and populations, but, cognitive-symbolic interfaces with human affordances have a long history. The physical design of modern computing is a revision of past systems like the Book Wheel, Memex and the NLS. The progression of information retrieval and processing tools clearly inspired the design for both Xerox and Macintosh devices which have been remediated into the kind of device that I am using today. The act of sitting at a desk to process information alone has remained the same, the device in front of the information gatherer has changed.  Specifically,  modern computational design has adapted from two structures 1: an internal logic for information processing and 2: an interpretable interface for humans (Irivine). The miniaturization of computing has caused the two essential structures of computing design to seem magical because the human facing interface has become simplistic and user friendly. Miniaturization has also resulted in increased portability of devices which in turn has lead to more access. When a computer is remediated from a room sized device, to one that fits a standard sized desk and eventually to one that fits into a backpack, more and more people have access to interact with device.The preservation of computational design, interface development and miniaturization are all design factors that have contributed to the current technological economy. Most obviously, these design factors can be seen in the smartphone which has benefited from remediations of the telegraph, the phone and the computer. The interface, physical design and even the way smartphones are used have histories that touch a variety of industries and creative developers.

Graphical User Interface

Warning: Use of undefined constant user_level - assumed 'user_level' (this will throw an Error in a future version of PHP) in /home/commons/public_html/wp-content/plugins/ultimate-google-analytics/ultimate_ga.php on line 524

Huazhi Qin

When first invented, the computer meant something too professional and hard for ordinary people to use. Nowadays, a computer can be operated by everyone without actually “knowing” what computer is and how it works. In other words, complicated functions can be achieved in some simple operations.

An easy-to-use interface is essential to make this happen. According to Engelbart, the development of human intellectual capability experienced four stages– conception manipulation, symbol manipulation, manual, external, symbol manipulation, and automated external symbol manipulation. (Engelbart) Computer lies in stage 4 and displays a close relationship with the prior three stage, especially symbolic system. For instance, a musical note icon represents a music software. A book with a letter refers to a built-in dictionary. The same as a microphone icon, a magnifying lens icon, a camera icon. It can be seen as a kind of simulation.

In addition, Current computers usually display the graphical user interface (GUI) or WIMP system. WIMP stands for “windows, icons, menus, pointer”, which provide ease of use to non-technical people. Beyond the use of icons (or symbols) mentioned above, a window shows what is running. A text or icon-based menu organizes and displays the function users could select. Also, a pointer visualizes users’ movements. In short, users can easily figure out how to do and what they are doing.

Furthermore, the expanding use of the computer, beyond the original boundary of military, government, and business, also help open computers to non-technical people. For instance, Hyperlinking was brought in to directly link to other documents. Photographs can be modified, combined, delivered or inserted in a computer. Video viewing, editing, and sharing are also included in a computer. It keeps computers evolving.

Meanwhile, simulation can also be seen in this progressing process. Manovich described a computer as a “remediation machine” in Software Takes Command. (Manovich) It means computer always imitate older. Including the symbol manipulation mentioned above, the adoption of common interface conventions and tools provides users with clues to operate something new. Users can operate new software based on their past experience. As to how Kay described Dynabook, “simulation is the central notion”.


Douglas Engelbart, ”Augmenting Human Intellect: A Conceptual Framework”

Lev Manovich, Software Takes Command, pp. 55-106, on the background for Allan Kay’s “Dynabook” Metamedium design concept.

Design Steps of Personal computers

Warning: Use of undefined constant user_level - assumed 'user_level' (this will throw an Error in a future version of PHP) in /home/commons/public_html/wp-content/plugins/ultimate-google-analytics/ultimate_ga.php on line 524

Zijing Liu

Computer development has a long history. Although there are different statements of the “first computer” due to different classifications of computer, it is true that earliest computers are super large, even occupying a whole room, which is hard for us to imagine. Similarly, it is also hard for the people who dedicated to inventing computers to imagine that computers become such “personal” and common use in decades later.

The initial step of personalizing computers, I believe, is to minimize its size, after all, no one would separate a room at home to store a computer. Even in the current computer industry, designers and engineers are still struggling to simplify the structure and appearance of PCs and laptops to make it lighter or portable. In short, reducing the physical characteristics of hardware, such as weight and volume, provide premise to personal computers.

The next step is concerned with the inner layer – GUI and software. One of the biggest progress was the development and advancement of software. For instance, in the 1980s to 1990s, the release of Adobe software brought a “remediation revolution”, which enabled people to use computers to complete daily projects, such as editing photos, reading digital documents, etc. The rapid development of software greatly expanded the user audience scale, from government and business officers to the public. Actually, I was pretty surprised when learning that Adobe Inc. has such a long history. I used to think that it was the widespread use of personal computers prompted the software development, but exactly the opposite, it was the growth of software that made computer personalization possible.

Another key reason is the stronger interaction between computers and users. Designs for an input-output interface with an encoding technique for enacting computations and returning human-interpretable results. (Irvine) Touchscreen is the best proof. Except for simply receiving displaying information, people were able to communicate with their computers and convey their orders positively and directly by “touching” the screens and get feedback immediately.

What Alan Kay did was not only “establish the computer as a comprehensive media machine” but also relate it to all possible artistic media. (Manovich) It was a continuing process of simulation, i.e. adding new features to old media, which proved that any new technology is the continuum of existing ones, whether adding new properties or rearrange in different ways, pretty much the same.



  • Martin Irvine, Introduction to Symbolic-Cognitive Interfaces: History of Design Principles.
  • Lev Manovich, Software Takes Command, pp. 55-106, on the background for Allan Kay’s “Dynabook” Metamedium design concept.

Week 9-Reading Response

Warning: Use of undefined constant user_level - assumed 'user_level' (this will throw an Error in a future version of PHP) in /home/commons/public_html/wp-content/plugins/ultimate-google-analytics/ultimate_ga.php on line 524

Similar to architecture design, the computer began with a concept—contributing a great user experience—that largely depends on its interface design. At the earlier stage, interface meant anything that physically connects different parts of a system—a point where two systems, subjects, organizations meet and interact. Today the concept basically developed into the interactive, software-controlled screen representations. A good interface design is about allowing for smooth completion of any task and making the experience enjoyable.

Here we take a brief look at how computer interface design has evolved over the past decades. Computers were first designed as numerical and logic processing. The user interface of Batch computers consisted of the input of punched card and users had no interaction with these earlier computers in real time. Since they are designed to utilize the processor to the maximum, it is unnecessary to spend the budget on user interfaces. Later on, computer design began to include graphical elements for cathode ray tubes as display and output interface. The next key interface progression was the introduction of video display terminals that could have users’ command inputs appear on the screen and let them modify those inputs. Then we embraced a new view of “augmenting human intellect” and the very birth of the first GUI developed by Xerox Palo Research Center which has let us to where we are today. The first generation of operating systems, including Xerox Star, Apple Lisa Office System 1, Mac OS System 1, etc., had a windows-based system with icons that had many features that are still in use on today’s systems. Files could be moved to different folders by dragging and dropping on the target icon. Windows could be moved around the screen with the cursor. Here we are in the new era of smartphones when everyone is the software user, interface design started to change significantly due to the popularity of handheld devices–smartphones, which are embedded with telephone and PDA features as well as sophisticated touchscreen and apps. What about the future? Keyboards, mouse, touch and voice inputs are common in both desktop and mobile devices now. Such functionality is likely to evolve with other interface design advancements such as gesture functionality, brain-computer and AR.

Figure1. The Early Mac OS Operating System

All of those future possibilities of futuristic computer interfaces were actually already depicted for million times in sci-fi films. Sci-fi movies are known to be sort of window into the future. Lots of fantastical technologies filmed only decades ago are now everywhere in our pockets or on our desks. Back in 1968, Kubrick made many predictions about man’s future in regard to technological. Despite a few variations, some of the interface designs are already existing in our lives. Glass cockpit displays, electronic tablet (iPad), in-flight personal television, video phone (Skype/Facetime). Hard to imagine the interfaces of those designs look so similar to what we use now in everyday life. I believe even though some of those technologies were not mature enough or even not invented yet, the audience at that time could easily tell what their functions were by simply garnering and interpreting information (text, icons, graphs, etc.) they got from the interface. After all, “the computer is no longer primarily a ‘mathematical instrument’ but a symbol-manipulating system for a human interpreting agent.”


Figure 2. Electronic Tablet in 2001: A Space Odyssey | Figure 3. iPad


Figure 4. Control Panel in 2001: A Space Odyssey | Figure 5. Screen Layout of iPhone


Figure 6. Warning in 2001: A Space Odyssey | Figure 7. Google Security Warning


Figure 8. Status Interface in 2001: A Space Odyssey | Figure 9. Electrocardiogram Monitor

There is a website called Kit FUI that gathers all this cinematic chrome into one place. Interfaces on this website don´t really exist—they are a subset of film’s visual effects—but the example they set about how digital devices could work is a fascinating barometer for user interfaces trends. It seems like all the designers focus on visual aesthetics, adding as many graphs and charts as possible while throwing away as much obscure text as they could. They focus on four things that define the WIMP paradigm (window, icon, menu, and pointing device). As stated in the reading, “the design principles are based on the affordances of pixel-grid screens for rendering interpretable symbols from defined software and hardware functions as either symbolic “content”(rendered text, images, video, etc.) or as action translators (icons, links, gestural controls) “. Most of the designs are appetizing but intuitive and clean because designers know that most web users love with their eyes.


Figure 10. Minority Report | Figure 11. Iron Man JARVIS Wallpaper

Credits to:

Martin Irvine, Introduction to Symbolic-Cognitive Interfaces: History of Design Principles (essay).

Bill Moggridge, ed., Designing Interactions. Cambridge, MA: The MIT Press, 2007.

Lev Manovich, Software Takes Command, pp. 55-106, on the background for Allan Kay’s “Dynabook” Metamedium design concept.

Weekly Writing for Week 9

Warning: Use of undefined constant user_level - assumed 'user_level' (this will throw an Error in a future version of PHP) in /home/commons/public_html/wp-content/plugins/ultimate-google-analytics/ultimate_ga.php on line 524

Banruo Xiao

The history of technology world is pretty like a process of sculpturing a piece of stone. Each scientist, engineer and developer works on one specific part, and they all together create a masterpiece. From a giant machine to an interactive artifact, the more the engineers and developers dig into the world of technology, the more hiding treasures they find.

According to Martin Irvine, the computer machine initially closely related to the idea of interface. The interface connects components together shown on one screen. Doug Engelbart creates a concept of symbol manipulation system to help people solve problem with the interface and the computer. Stu Card then comes up with an idea that design is to understand people. Instead of just focusing on developing hardware, people should pay more attention to the user side. The creation of mouse, desktop, system and many other advanced technologies are developed based on one idea which is how to make people interact with the machine smoother.

At the same time, Lev Manovich spends a whole chapter talks about how medium and metamedium help the development of computer and make it more interactive to nontechnical users. Many forms of mediums, such as text, image, music and video, make computer possible to provide more functions and solutions to people with various needs. Indeed, medium provides more ideas and possibilities to the designer and user. Later, the programming language, which Lev Manovich describes it as metamedium, can even help developers solve different layers of problems at the same time.

Bill Moggridge mentions in his paper that each conceptual idea should be tested a lot of times to see if it works for most users. Doug Engelbart illustrates the idea of demo for testing an idea for a better design. In fact, the idea of interface, symbol manipulation, interaction and metamedium all come out from a thousand times of trying. The capability of computer goes beyond the field of military and other earlier usage. The accomplishment should be credit to the designers’ persistent attempt and the ability to discover and to solve problem.


Week 9

Warning: Use of undefined constant user_level - assumed 'user_level' (this will throw an Error in a future version of PHP) in /home/commons/public_html/wp-content/plugins/ultimate-google-analytics/ultimate_ga.php on line 524

Sometimes when people are trying to compare the old/ first computer to the latest ones, they only see the huge difference in the actual sizes, but the transaction of its function and the design thinking behind them is merely considered. Back to the old days, the huge, room-sized computer was designed after war time for military and government use. The main purpose of computer was to calculate, which as Professor Irvine mentioned, only used for what we called “number crushing”. Though the interfaces and function of the first computer seems dumb, but these designs and ideas lead to the program innovation of “remediation machine”, which is basically the general structure of nowadays computers.

As introduced by Manovich, the “remediation machine” is what referring to “a machine that expertly represents a range of earlier media (Manovich)” that based on GUI software. It allows digital computers to integrated before-computer media such as video, audio and painting features while still keeps the same function and appearance. This development allows computer to be exposed to more nontechnical users, and turns it into “a personal machine for display, authoring and editing content in different media (Manovich)”.

Same idea is shared by Douglas Engelbart, who believes that modern computers are not simply used fro business or government, but should be used “as aids for core human cognitive tasks that could be open to everyone (Engelbart).” Which, as he also believes, that with the help of computer, human can face more complex problems and nailed it better than simply a computer alone. He states that “by ‘augmenting human intellect’ we mean increasing the capability of a man to approach a complex problem situation, to gain comprehension to suit his particular needs, and to derive solutions to problems (Engelbart).” I think this idea overlaps with the reading of computing thinking in some way, that both of the augmented intelligence and computational thinking are designed aiming to enhance human behaviors instead of replacing humans.

Engelbart, Christina, ed. “Augmenting Human Intellect: A Conceptual Framework – 1962 (AUGMENT,3906,).” Augmenting Human Intellect: A Conceptual Framework – 1962 (AUGMENT,3906,) – Doug Engelbart Institute,

Lev Manovich, Software Takes Command, pp. 55-106

Week 9_The Historical View of Computing with Symbolic-Cognitive Interfaces

Warning: Use of undefined constant user_level - assumed 'user_level' (this will throw an Error in a future version of PHP) in /home/commons/public_html/wp-content/plugins/ultimate-google-analytics/ultimate_ga.php on line 524

Tianyi Zhao

The historical track of conceptional designs of computer and interfaces is quite clear and behaves as continuum. We have been living in a digital environment where computers and human-computer interaction are ubiquitous so that we take them for granted. In retrospect of the history of the computing with symbolic-cognitive interfaces, there were many excellent designs. Besides Jean Mielot’s writer-scribe and Augustino Ramelli’s book wheel, I would like to begin with an example of Suanpan—a Chinese abacus firstly recorded in 190 CE. The design of two beads per rod in the upper deck and five beads per rod in the bottom deck is used for both decimal hexadecimal computing. Suanpan, as an initial computer device interface from ancient China, intuitively shows the process and final result of computing by toggling the positions of beads, which has been utilized widely in business accounting in China for centuries, even still used in some small pharmacies now. Without the electricity power and screen in the past, beads became the only interfaces to be interpreted to its own meaning system—calculate by moving the beads up and down and read the result though the final positions.  (Question: is Suanpan a kind of artefact of both output and input?)

Figure 1. Suanpan, China’s abacus. 


When it came to the twentieth Century, the ideas of modern computing gradually took shape. In the late 1930s, the militaries of the United States and United Kingdom began to seek for an electronic machine to calculate ballistic firing tables and to crack ciphers until the birth of ENIAC in 1944—the world’s first general-purpose computer. (Denning & Martell 5) Soon after, Memex debuted in Vannevar Bush’s “As We May Think” in 1945, which used the desktop metaphor and screens. Inspired by the invention of Bush, Douglas Engelbart furthered to the next level— “augmenting human intellect” that computers can aid people’s daily life for core human cognitive tasks by leveraging with CRT display, mouse, windows, network and memory storage as “a whole computer system.” (Irvine) Douglas’s contribution indicated a new direction which was the concept of screens serving as both output and input interfaces. So then came Alan Kay with his Dynabook and his coined term metamedium in the 1970s. Similarly, the Xerox PARC with its commercial implementation— such as graphical “desktop” with multiple “windowing,” icons and languages — guided Steve Jobs and Bill Gates for their further commercial computer applications.


Works Cited

Irvine, Martin. “Introduction to Symbolic-Cognitive Interfaces: History of Design principles.

Peter, and Craig H. Martell. Great Principles of Computing. Cambridge, MA: The MIT Press, 2015.

Bush, Vannevar. “As We May Think.” Atlantic. July, 1945. (2018) (2018)