My Television is a Computer: Design Thinking in Apple TV’s Interface

Abstract:

            For the past decade or so, television has been undergoing a shift  from a broadcast medium to a digitally networked one. And while such a shift entails certain technical advances and the designs therein, it also indicates a need for redesigning the user interface of the television. Unsurprisingly, Apple has engaged with the design of digital television interfaces from the very beginning, primarily through their Apple TV product. This paper takes the Apple TV – and the constituent parts which are involved in its human-computer interaction – as a case study in the design problems posed in applying computational media to television. Namely, it first considers the Apple Remote, and the challenges of using a remote as an input device for a computational machine, as opposed to a keyboard and mouse. Secondly, it considers the two graphical user interfaces designed for the Apple TV, the Front Row interface and the tiled grid. This section deals with the design principles common for 10-foot user interfaces while also dealing with the ways in which these two interfaces accomplished different purposes within the sociotechnical system built around Apple TV.

Introduction

In Apple’s never-ending quest to achieve unparalleled dominance in every segment of the media market, their forays into television proved multiple and varied in success. Even laying aside attempts which largely failed, such as Macintosh TV and the Apple Interactive Television Box, the history of Apple TV – or iTV as it was initially called – is convoluted and filled with eccentricities. The Apple TV as a product, initially introduced in 2007, underwent a number of iterations, even to the point that branding aside, the Apple TV which one might purchase today, in 2019, is designed to accomplish very different tasks than the one released in 2007. These iterations mirror trends in consumer computing and the rise of streaming video services. Even given these technical evolutionary stages, however, one persistent design problem (and its solution) separates out Apple TV as indicative of digital television and distinct from other technologies in the late 2000’s through present day: how to design a computer as a TV. From the perspective of 2019, this question may seem quaint, or even banal. After all, the answer seems straightforward enough, simply connect the TV to a laptop through an HDMI cord, or another technology to post-date Apple TV, Google Chromecast. But neither of these solutions actually solves the problem that Apple was tackling. Instead, they create systems where the television screen acts as a mirror for the PC or laptop monitor. For the user to manipulate the program, they turn to their laptop to do so. In other words, the television is a glorified monitor, and not a unique computing machine with both input devices and interfaces designed for the medium of the TV. The problem of TV interface design is one which defines the cultural moment of 2007 to 2009 where the role of the television decidedly changed from that of a broadcast medium to a medium which took full advantages of the affordances of networked computing (Braun 2013). These affordances, which according to Simon and Rose (2010) had been incorporated in limited ways with television technologies since the 1990’s, required distinct sorts of design choices for the expanded application of computational affordances in the last half of the 2000’s: designing the television as a system of information and designing the television as a user interface. The first goal of this paper, then, is to explore design choices which informed technical components of Apple TV employed in several iterations: data synchronization and data streaming. This section will also discuss the consequences of those choices in terms of design and user experience, as well as reasons for the ultimate triumph of streaming over synchronization. Second, this paper will seek to understand the design principles behind Apple TV as a computer designed for the television and according to its affordances and constraints; which is to say, without textual input or an X/Y pointing device (i.e. a mouse). This section will discuss the Apple remote as an input device and the principles of 10-foot user-interfaces at large, as well as the specific design of the Apple TV interface.

Computing TV

            It takes no great technologist to determine that the inputting mechanism for a television significantly differs from the inputting mechanism for a computer. To understand the exact interface problems which the Apple TV ultimately addressed, however, one must first understand the gap in the market Apple was attempting to fill through this product: on-demand television. To be fair, pay-per-view television was developed in the United States as early as the 1950’s (Smith 2001). Pre-digital on-demand television involved calling in to the pay-per-view provider by telephone in order to request the desired program. With the advent of digitally transmitted television in the 1990’s, as well as the boom in consumer computing in the late 20th century, the constraints of broadcast and cable television for providing on-demand video wavered in the face of new digital affordances. For Apple in 2007, during the incunabula stage of the Apple TV, great strides were being made toward designing an interface wherein users could take advantage of iTunes’ ability to offer video on demand not on their laptop, but on their television (Chamberlain 2010). In a certain sense, with digital television technologies taking the position of market dominance previously held by cable, Apple needed a way to compete with the affordance of digital TV to provide on-demand video within its own technical system. Apple had been providing on demand video rentals and purchases themselves for about 4 years, at the time, through their innovative iTunes platform, but the platform suffered severe limitations due to some of the design choices of other Apple products at the time. In other words, before the release of the iPhone in 2007, Apple’s only networked devices – that is, the only devices with the capability to access the Internet – were its laptop and desktop products. In those early days, the constraints of the iPod required a level of dedication from the user in order to purchase or consume on-demand music or video. Anyone who owned an iPod in its earliest iterations surely remembers having to buy a song or movie on the desktop or laptop iTunes interface, next plugging the iPod into the computer’s USB port, waiting for the iPod to sync, and finally dragging the music file from the iTunes library to the iPod device so as to signal the initiation of copying the song or movie file to the iPod from the personal computer. Only once this process was completed could the user consume their “on-demand” product away from their personal computer.

Now, this account of the iTunes store only has to do with the design problems of Apple TV insofar as it sets the scene by describing the state of Apple’s dealings with on-demand video at the time of the Apple TV’s release. Particularly, this early approach to using Internet-based marketplaces, accessed on personal computers and not media players themselves, explains Apple’s decision to design the first generation of Apple TV effectively as a storage and playback unit for media files bought on the iTunes store. Just as iPod users bought content from their personal computer, and then connected their iPod to the computer in order to copy files to the media player, first generation Apple TV users originally had no other option than to purchase media on a personal computer (Cheng 2007). Oddly, this unfortunate legacy from iPods and other Apple media players was not a technical constraint – as even the first Apple TV’s were networked computers with Internet access, which Apple capitalized on in promoting access to YouTube from the Apple TV – but instead a simple instance of technological “lock-in,” where a technological practice persists, not because it offers any advantages, but because it is an established method for doing something (David 1985). Fortunately for the user, this inconvenience lasted less than a year, as a software update in January 2008 provided users with the opportunity to access the iTunes store directly from the Apple TV interface, removing the inconvenient need for a personal computer to mediate this access.

Of course, this Apple TV differed significantly from the ones on the sterile white shelves of Apple stores today if not in its technical capacity, then in its technical function. In other words, even though Apple TV employed data streaming in order to play YouTube videos, it primarily relied on the technique of data synchronization for playing videos purchased from its own store (Pegoraro 2007). Unlike streaming, which involves sending media files from router to router as Internet packets, which are dissembled upon sending and reassembled upon receiving, data synchronization involves the data set on one device being exactly copied, or mirrored, on another. And while data synchronization offered some advantages, such as ensuring that a user’s iTunes library on a personal computer might match their library on their Apple TV, its major drawback, from a design perspective, was that it required the files to be downloaded to the Apple TV itself. In some respects, this constraint only becomes evident looking through the rearview mirror: the design of the fifth generation Apple TV, with only 32 and 64 GB’s of storage relies on its capabilities to stream music and video, and therefore to not require much space by way of internal storage. Comparatively, the first generation Apple TV’s basic model boasted 164 GB of storage. To be fair, the first generation Apple TV offered capabilities for both streaming and synchronization, however, synchronization was the default mode, as streaming was foreign to many users (Pegoraro 2007). Regardless, only after streaming made data synchronization largely obsolete would models with less storage space appear more tenable. The second generation of Apple TV, however, left behind the relic of data synchronization in favor of streaming for all time-based media.

Input and Interface Design for Apple TV

            Even with the technical capacity to use personal and consumer computing products as a gateway to content for the Apple TV, which a traditional television could then display on its screen, Apple still needed to overcome significant design challenges in terms of the input and interface of this media player. Particularly considering the emergence of on-demand movies and media on digital cable in the 1990’s and 2000’s, Apple needed a human-computer interface which simultaneously appealed to the brand’s design sensibilities, as well as provided users with straightforward navigation techniques where they could browse, locate, and access media easily and without frustration. In a word, Apple set out to design an interface and user input system which allowed for the user to take full advantage of the computational power of the Apple TV, while still perpetuating the user experience of using a television, and not a computer, which is to say, the primary input mechanism was a remote and not a mouse and keyboard. Because interfaces facilitate human-computer interaction, the following discussion of user-facing software and input devices often refer back to one another; the software must accommodate the constraints of the input device, and the input device should maximize the affordances of the software. At the same time, they can be more or less taken as individual modules in the combinatorial design of the Apple TV, and therefore I will address them as such. To be clear, by discussing the design of Apple TV’s interface, I am not only referring to the design of the Graphical User Interface (GUI), but the deep cultural history of the word as a meeting and the joining point for two disparate artefacts (Irvine n.d.). Specifically, Daniel Chamberlain’s definition of interface helpfully charts out the territory to be covered throughout this paper. He writes, “In a material sense we can think of those interfaces as consisting of three parts—a physical means of interacting with a screen-based display driven by dedicated software” (Chamberlain, p. 85, 2010, original emphases). Particularly, I am concerned with understanding the design of the GUI as interfacing the physical means of interacting (the Apple Remote) to the networked content (movies, music, etc.) on their screen-based display. The previous discussion took the dedicated software to task, albeit in limited scope, because the software, in the case of television’s adaptation of digital affordances, is not necessarily unique to the television medium, and therefore of secondary concern for this paper. In the following paragraphs, then, I consider the design problems and solutions of input and interface for Apple TV from both the design of the Apple Remote, as an input technology, as well as the Front Row interface and the subsequent tiled app interface of tvOS.

Apple Remote

Figure 1: First Generation Apple Remote

What’s a television without a remote? Ever since the television remote was invented in the latter half of the 20th century, its status as the ubiquitous mode for interacting with the television has remained unchallenged. For Apple to carry the experience of television watching through to Apple TV, they were wise to adopt this piece of hardware in their sociotechnical system – even if in the second generation and onward they enabled Apple TV with Bluetooth capabilities to connect with Bluetooth QWERTY keyboards for the convenience of those particularly fed up with the obstinance of “typing” with the Apple Remote. The Apple Remote was largely designed so as to allow a user to manipulate the Front Row interface – through which the user interacted with the Apple TV. Until a major design change in the fourth  generation of Apple Remote, its design visually borrowed from the iPod family, boasting only a minimalistic six buttons and the iconic “wheel” of the iPod (albeit this wheel did not function as a wheel and was simply four buttons positioned along the circumference of a circle). Because of their limited number, many of the buttons accomplished more than one feature, based on

Figure 2: Second Generation Apple Remote

whether the user was interacting with the Apple TV through the course of playing media or through navigating the menu. The fast-forward and rewind buttons, for example, doubled as left and right navigators, as the increase and decrease volume buttons doubled as up and down navigators. Through this technique, the design choice of positioning these buttons along the wheel proves more helpful than a simple perpetuation of Apple’s iconic brand and design philosophy. In other words, while the wheel did in fact preserve Apple’s visual aesthetic in the Apple Remote, it also allowed for these buttons to perform semiotic double duty. For example, if rewind/fast-forward and increase/decrease volume had been positioned in rows or columns, this implicit directionality of left/right and up/down would have been lost. For the buttons to perform double duty in the hypothetical column and row setup, they would need to be labeled as such so that they user might properly interpret their function. By positioning them along the wheel, the designers at Apple assume that their users have already been conditioned to understand the significance of directionality and design their hardware accordingly. Bruno Latour (under the pseudonym Jim Johnson) calls this phenomenon pre-inscription – the information or learning which the user is assumed to have before interacting with a technology (Johnson 1988).

The obvious limitation of such a remote, however, were its limited affordances for inputting textual data, which inevitably created problems for any sort of search functions. For a technology which organized entire catalogs of films and music, this constraint was fairly significant. Even with the adaptation of connectivity to Bluetooth keyboards with the second generation, Apple had designed the product so as to assume that the Apple Remote would function as the primary input device – any choice to allow the user to use a keyboard would be an extra flourish and affordance. For this reason, Apple had to ensure that whatever graphical user interface it designed or used would be entirely navigable through the affordances of the Apple Remote. This meant not only foregoing the QWERTY keyboard, but also abandoning any sort of mouse or cursor. Fortunately for Apple, interfaces which solved this problem proved not to be design impossibilities. And while, as we will see, the first interface did not endure until the present day, it characterized much of the early user experience of Apple TV. Furthermore, many of the principles of which it consisted still can be identified in the design of Apple TV’s current interface.

Front Row Interface

Figure 3: Front Row Interface

While the Front Row interface was always conceived of by Apple as a multimedia player, it pre-dated the Apple TV by around two years. And even while the user might have originally interacted with the Front Row interface through the medium of a Macintosh computer, Front Row’s release coincided with the release of the first generation Apple Remote, as the two were designed for one another. The Front Row interface was Apple’s first attempt at developing “10-foot user interface” (10-foot UI). 10-foot UI’s emerged with the rise of smart TV’s and the need to not only account for the manipulation of icons and symbols with a remote device and not a mouse and keyboard, but also the increased distance of the user from the screen when watching television as opposed to working on a computer (the 10 feet in 10-foot UI reference this distance) (Lal 2013). This distance generated several constraints of transitioning computing technology to the medium of the television (the first being the use of the remote as an input device): screens needed to be treated as single entities, and not windows, icons needed to be larger, and the interface should make it clear to the user which icons or symbols with which they were interacting by highlighting or otherwise indicating the icon at hand (Lal 2013).

According to Michael Moyer (2009), through the course of their development, 10-foot UI’s solved the problem by employing one of two major methods, the first of which quickly proved inferior to the second. The more obvious, but less successful design of 10-foot UI’s involved developing browsers for the television screen. As alluded to above, this involved enlarging the search bar and other icons, so as to ensure its visibility from the couch. Ultimately, however, it failed to provide a way to input text without requiring either the extreme patience of selecting every letter individually out of the entire alphabet for textual input or requiring the connection of a Bluetooth keyboard. For this reason, the second common design for 10-foot UI’s – a widget-based design – remains the industry standard. The widget system involves sorting out each program or feature into separate icons, not dissimilar to the manner in which apps are presented on smartphones, which the user can then sort through in order to select their desired function. Particularly in a digital economy where many Internet-based services are not housed by companies which manufacture communication technologies themselves, the widget design seems to offer many advantages. For example, instead of relying on a browser to mediate access to Netflix, Hulu, YouTube, Spotify, or any other number of media companies, the widget offers the user direct access to the content therein. Perhaps the greatest testimonial to these advantages, however, can be found in the fact that many of our touchscreen technologies, such as smartphones and tablets, employ the design of widget-based interfaces, at least in part, even when interfaces which rely on searching or textual input more heavily are not constrained by the physical and technical limits of their input systems.

Apple’s Front Row interface, however, needed to solve the problem of accessing the growing number of media types available for purchase or general consumption on Apple’s platforms. This included music, movies, TV shows, podcasts, and pictures. For all intents and purposes, the “widgets” which a user could choose between acted as access points, not for different media services or companies, but to discrete libraries for different types of media housed within Apple’s platforms. In other words, a user could toggle between text reading “Music,” “Movies,” “TV Shows,” and so on, while the correlating icon cycles along with the highlighted text to the left, in order to access the respective libraries. Upon accessing said libraries, the user would go through a similar process to select the actual file that they intended to stream or to play.

Leaving the Front Row and the Constraints of Widgets

Figure 4: tvOS Interface with “tiled widgets”

When Apple retired the Front Row interface in 2011 in favor of its OS X Lion for the Macintosh and tvOS for Apple TV (released the following year), it also abandoned the text-based widgets which defined the previous system. Instead, Apple opted to display software options through tiles of widget icons, with which the user interacted in essentially the same manner as he or she had grown accustomed to while using Front Row (i.e. using the remote to move vertically or horizontally, highlighting an icon to select along the way).

While this interface design afforded no new technical capacities for the Apple TV it signaled an important shift in the economics the product. As mentioned above, the Front Row interface was basically designed so that Apple users could organize and access a diverse number of media types all of which (or at least most of which) were under Apple’s umbrella. Of course, Apple could have simply kept the Front Row interface and added the products from new developers to the list of text which users could scroll through (i.e. Netflix, Spotify, and so on), but by choosing to design the interface with tiled widgets, which looked so similar to app icons on the ubiquitous iPhone, Apple subtly indicated a shift in their thinking about the Apple TV as a product. In other words, Apple TV was no longer a product which enabled consumption of media purchased through Apple’s platforms in the living room but was now a platform which facilitated the consumption of all digital and streaming content. From a developer’s point of view, this opened up the Apple TV from being an in-house Apple device to being one where products from a diverse number of developers could promote and distribute their products. However, this new focus of the Apple TV as a product did nothing to de-black box it from a consumer perspective. Meaning that even as Apple TV expanded the accessibility of its interface to non-Apple products which used Internet protocols to stream media, it still severely limited any access to the Internet writ large, particularly as opposed to browser designs for 10-foot UI’s. The interface constrains the user from accessing any aspects of the web other than those expressly designed to be accessed by the apps represented by the widget icons.

Conclusion

Apple TV proves an interesting case study as a smart TV technology not only because it serves as an archetypical example of the design problems all smart TV’s encounter – namely, the problem of distance and the problem of input – but also because of the interesting transition in this product’s specific history wherein it shifts from being designed as a product to mediate access to in-house Apple media, to being a meta-medium which facilitates access to non-Apple software and products. By considering the design problems encountered and overcome by the Apple TV, one discovers new answers to the old question, “why is this technology designed this way and not another way?” Particularly, Apple TV makes evident a unique symbiosis of the input technology and the interface design, where the interface accounts for the constraints of the input device.

Perhaps it is worth noting that the Apple TV product design only represents a single approach to reconciling television and personal (entertainment) computing, of which several product designs remain popular. Of course, there are other products on the market which solve these design problems in essentially the same way as Apple TV, such as Roku set-top boxes and the Amazon Fire TV Cube. Other products, however, like Google Chromecast, entirely sidestep design problems introduced by 10-foot UI and remote control input, and simply connects a personal computer to a TV, whereby the computer functionally acts as an input device and no new interfaces need to be designed. Comparatively, many companies now manufacture TVs which themselves possess computational power and can connect to the Internet. For smart TV products such as these, the design not only needs to accommodate for the limited user input of a remote and the 10-foot UI design, but also must consider how to seamlessly integrate the computational affordances of the smart TV with those of the cable TV system.

Considering the design of the Apple TV as primarily defined by input and interface problems, however, ultimately only considers a limited – albeit unique and important – set of principles guiding the design of this product and others like it. For example, it only deals with the combinatorial and modular design of Apple TV in an indirect and incomplete sense. And while an analysis which takes combinatorial and modular design into greater account might contribute a study which more fully de-black boxes this product, it would not necessarily address the unique design problems which we have explored above. By better understanding the input/interface system, the unique qualities of computers designed for video consumption on television monitors reveals an interesting moment of designing computational media both according to its own affordances and constraints, but also according to the affordances and constraints of input devices.

Bibliography

Braun, J. (2013). Going over the Top: Online Television Distribution as Sociotechnical System. Communication, Culture and Critique, 6(3), 432–458. https://doi.org/10.1111/cccr.12015

Chamberlain, D. (2010). Television Interfaces. Journal of Popular Film and Television, 38(2), 84–88. https://doi.org/10.1080/01956051.2010.483354

Cheng, J. (2007, March 27). Apple TV: An in-depth review. Retrieved December 14, 2019, from Ars Technica website: https://arstechnica.com/apple/reviews/2007/03/apple-tv-1st-generation.ars

David, P. A. (1985). Clio and the Economics of QWERTY. The American Economic Review, 75(2), 332–337.

Irvine, M. (n.d.). Introduction to Affordances, Constraints, and Interfaces.

Johnson, J. (1988). Mixing Humans and Non-Humans Together. 35, 298–310.

Lal, R. (2013). Digital Design Essentials: 100 Ways to Design Better Desktop, Web, and Mobile Interfaces. Rockport Publishers.

Moyer, M. (2009). The Everything TV. Scientific American, 301(5), 74–79. Retrieved from JSTOR.

Pegoraro, R. (2007). Apple Tries to Bridge Computer Desk, Living Room. Retrieved from http://www.washingtonpost.com/wp-dyn/content/article/2007/03/28/AR2007032802156.html

Simon, R., & Rose, B. (2010). Mixed-Up Confusion: Coming to Terms with the Television Experience in the Twenty-First Century. Journal of Popular Film and Television, 38(2), 52–53. https://doi.org/10.1080/01956051.2010.482907

Smith, R. A. (2001). Play-by-Play: Radio, Television, and Big-Time College Sport. Baltimore: John Hopkins University Press.