The Internet of Thinking: Problematizing the Semiotic Processes Behind Google Arts and Culture


Warning: Use of undefined constant user_level - assumed 'user_level' (this will throw an Error in a future version of PHP) in /home/commons/public_html/wp-content/plugins/ultimate-google-analytics/ultimate_ga.php on line 524

Abstract   

On February 1st, 2011, Google Culture Institute launched its long-awaited digital venture, the “Google Art Project”. Originally in partnership with 17 museums from Europe and the U.S., the platform offered a new perspective on how museums are used and will be used on the Web. Google’s endeavor, now titled “Google Arts and Culture” has become the epicenter of a debate within the professional museum field where scholars see a paradigm shift towards a destabilized way of apprehending works of art. I argue that the driving force behind such a shift are the design principles that are at play. By comparing the design of Arts and Culture to the Metropolitan Museum of Art’s website and the online database Artstor, I will extrapolate the poor implementation of Google’s design principles that facilitate user interpretation and access to understanding art and culture. I will end by using my analysis as well as my background in Art History to offer suggestions on how to improve the visualization for Google’s platform that incorporate such semiotic layers.

The success of Google Arts and Culture was perhaps unintentional. The enterprise was developed as a side project by Google’s Group Marketing Manager Amit Sood who envisioned a platform that provided a “number of digital reproductions of works from participating museum institutions, which can then be visualized in high resolution and explored through a drag-and-drop, zoom-in-and-out interface.”[1] High resolution images could easily be found online, yet Google created pictures composed of as many as seven billion pixels. Also dubbed “gigapixel” images, these provide the user an even better experience than the “real, breathing thing” because the quality allows the user see a “microscopic view of details in brushwork and surface condition that previously were difficult or impossible to see with the naked eye.”[2] Yet among these- and more- digital feats that characterizes Google’s platform, the problem arises as to where does the platform mediate and provide interpretable information for users. Establishing any sort of meaning and dialogue between works of art recalls Malraux’s idea of the musée imaginaire– an ‘imaginary museum’ or ‘the museum without walls’.[3] A concept on creating the ideal collection of all artworks seen in our imagination where dialogue rests on the view of art and art history as essential for establishing dialogue between works.[4] The Metropolitan Museum of Art’s website and Artstor are designed with that very intention in mind; providing access to material and information for users to construct their own meaning system.

The Metropolitan Museum of Art

Providing some background information on how both sites get and organize their metadata will determine how each institution designs their platform. Understanding data storage, creation and dissemination will become key points of comparisons to Google Arts and Culture. The Metropolitan Museum of Art, also known as “the Met”, has a very thorough website designed as a one-stop-shop for anyone who is curious about the museum’s vast collections, archives, exhibitions or visiting the Met Museums themselves. The homepage is divided into multiple sections that merge together when you scroll up or down the page. The Metropolitan Museum of Art’s metadata is from its own database through their digitization efforts of their collections. According to their website, “The Metropolitan Museum of Art creates, organizes, and disseminates a broad range of digital images and data that document the rich history of the Museum, its collection, exhibitions, events, people, and activities.”[5] Such an undertaking was managed by the Thomas J. Watson Library Office of Digital Projects that “provided access to research materials from the Libraries of The Metropolitan Museum of Art, selected materials from Metropolitan Museum of Art curatorial departments, and partner libraries and archives.”[6]

Don Undeen was the information architect at the Met for 4 years before coming to Georgetown University. In an email, he explains how the Met creates, stores and updates its metadata:

“The Met stores all the data about its collection in a proprietary database system called TMS (The Museum System, from Gallery Systems). It’s a big complicated relational Database that runs on an Oracle database. The collection managers and curators from all the museum departments keep this system updated and objects are acquired, new information is gained, they are placed in various exhibitions, travel on loan, etc. So the database is more than just the information that visitors to the website see, but is actually for managing the objects in the collection as well.”[7]

The museum’s collection falls into two categories; images of works believed to be in the public domain and those to be under copyright or other restrictions. To overcome this division, The Metropolitan Museum of Art implemented a new policy in February 2017 known as “Open Access” which allows artworks in the public domain to be freely available for unrestricted use as well makes available data from the entire online collection under the Creative Commons Zero (CC0).[8] Images of artwork believed to be in the public domain are also available on ITHAKA-Artstor and Google Cultural Institute, reinforcing a shared platform of metadata that also plays into Malraux’s concept of a museum without walls.

Artstor

Artstor is also committed to disseminating knowledge of scholarship and teaching through digital images and media. A nonprofit organization, Artstor is comprised of Artstor Digital Library that includes millions of high-quality images for education and research across disciplines from a wide variety of contributors around the world. They also developed JSTOR Forum, a software that allows institutional users to catalog, manage, and distribute digital media collections and make them more discoverable. In 2016, Artstor formed a strategic alliance with ITHAKA, a not-for-profit organization whose mission is for preserving and expanding access to knowledge and is home to other services for higher education such as JSTOR and Portico. Whereas the Met uses its own collection as metadata, Artstor relies on contributors to promotes and shares information. Contributors are listed as museums, artists, artists’ estates, photographers, scholars, special collections and photo archives on the website. As such, Artstor uses several different kinds of databases to get their information onto one platform. Artstor has specific guidelines on how to sort through, group, classify and organize the information they receive. According to the website, “…the Digital Library can be searched and browed by object-type classification (e.g. painting, architecture, etc.), country/ region, and earliest and latest date. The classification terms are applied from an in-house controlled list (painting, sculpture, etc.); the country terms are from the Getty Research Institute’s Thesaurus of Geographic Names (TGN); and numeric earliest and latest dates are created for each record. The Artstor Advanced Search and Browse functions depend on these consistent access points to improve access to all collections.”[9] To alleviate  problems associated with information overload, Artstor uses the following techniques:

  1. Clustering duplicates and details representing a unique work “behind” a lead image to represent the entire work and provide the highest quality
  2. Collaborative filtered groups determine which images are saved in conjunction with other specific images by users making groups
  3. A controlled vocabulary from Getty Research Institute’s Union List of Artist’s Names (ULAN) allows searching for works by any part or variant of an artist’s name to find images linked to the ULAN creator record

The Google Behind Arts and Culture

How do both of these platforms compare to Google Arts and Culture? In order to understand Google’s platform, one needs to understand how Google as an entire company thinks and creates its platforms. It is hard to image a life before Google. Indeed, Google is intertwined in almost anything if not everything we do. We retrieve information ranging from something trivial like DYI tutorials to helping students (like me) write papers like this one. Google was founded in the late 1990s along with the revolution to commercialize the Web. The rise of this mega-tech company started a new way of searching and categorizing ideas and issues through algorithms. In their book Google and the Culture of Search, Hillis et. al aptly state how “Google operates as a nexus of power and knowledge newly constituted through extremely rapid changes in networked media technologies…”.[10] Google is an online platform that is also a kind of interface that (should) facilitate meaning. An interface is a metaphor that describes discovering “important cognitive and technical patterns that apply to all kinds of symbolic artefacts: books, photographs, artworks, music, architecture (3D built space), and, more recently, the symbolic substrate of pixel-based screens that can represent any 2D digital object. “Interface” is also part of web of related conceptual metaphors: medium/mediation, affordance, window, node, link, relay.”[11] Google’s search model (PageRank) which is one component of its interface relies on relevancy, instantaneity and generic individualization which ultimately skews how they operate their database and more broadly speaking, their platforms. By creating algorithms that will preemptively select what you would search, Google is narrowing down the scope of your search results based on what you want to see. Such a concept ties into Manovich’s ‘paradox’ of personalized technologies, where in “…following an interactive path, one does not construct unique self but instead adopts already pre-established identities.”[12] Ultimately Google is telling a viewer information whereas a normal interface is giving a viewer access to materials and information so they create their own meaning system. Studying the search technology behind Google has broader implications on how users, searchers and viewers navigate, classify and evaluate Web content on all of Google’s platforms, including Arts and Culture.

In order to extrapolate Google’s design principles, one needs to look at the motive behind Arts and Culture as well as how it receives its data. First and foremost, Google Arts and Culture is a digitized open archive of sorts (the word ‘open’ complicates the matter and will be further extrapolated further down). The original 17 institutions that were part of the project have now expanded to over a thousand collections, museums and other cultural institutions since its initial launch 7 years ago. Don rightly proposes the following definition of the Google Arts and Culture project; “The Google Art project is more of a Content Management system for web-facing educational art websites. Its goal is to amass collections from as many sources as possible and present it nicely, at the expense of having complete records, complete collections, or making it available for re-use.”[13] Google therefore gets its data from institutions that would like to participate and ask for a kind of ‘membership’ for their artwork to be displayed on Google’s platform. Arts and Culture imports museum collection to its platform and has the liberty to interpret, place and treat these objects as it sees fit. Participating institutions provide their collection on this free platform and Google provides the technical services of archiving their data. Not only does it gain visibility for the institutions but also adds to Google’s branding. It’s a win-win for both parties involved.

The services provided on the Arts and Culture platform are also using Google’s current technologies, just bundled and combined in a different way. Combinatorial or cumulative designs are found on an interactive screen interface, where in using this interface “we become conductors of a complex, unobservable orchestra of actions for transforming signs and symbols through ongoing computation and combinations with other symbolic structures, and combinations with many other conductors.”[14] Google has bundled together technologies that have already existed before creating Arts and Culture. For instance, “Street View”, “Nearby” and all maps are from Google Maps, basic information on hours of the museums from Google Search and data on images fetched partly from Google Images. At an initial glances, the design of the platform seems transparent and natural but it’s not, given that Google’s software embedded in Arts and Culture just needs to fetch the basic scripted metadata from other platforms embedded into this platform. Arts and Culture therefore becomes an example of a ‘meta-medium’, a phrase coined by Alan kay to describe a computer with “no longer a single medium but a medium for other media processed by user-activated software in the same substrate used for display.”[15] The Metropolitan Museum of Art’s website and Artstor are also meta-mediums but for a different purpose. The motivation behind the Met collections database and Artstor rests on its priority to be a resource for scholars and for the public, being as open as possible and erring on the side of more sharing. The Met’s Open Access API further reinstates its commitment to Arts and Culture, however, doesn’t have a terms of service statement regarding re-use, and they’ve even made it hard to download the images on the site. Try right-clicking to save image; it doesn’t work anywhere that I can see. In a sense it’s a “Dead End.”

A Problem of Design

 Such back-end technologies impact every aspect of how Google designs and visualizes its metadata on Arts and Culture The Design of Arts and Culture is rather interesting if you look at other Google platforms in comparison. All of Google’s products and services follow a strict design guide called Visual Assets Guidelines. According to Google’s Art Director Christ Bettig, the guide “defines our visual design (across platforms)  and provides the assets / information needed in order to create any visuals for Google products or marketing collateral.”[16] Among the many guidelines in this comprehensive report, a few items stood out to me. Google’s core design philosophy has always been to create products that are built on a large scale and that are “mobile-first”; two concepts that are not embedded into Arts and Culture. Not only does Google violate their own design principles, the company fails to incorporate interactive design features that are intended to provide user information to facilitate interpretation and usable representation.

One of the key components of Arts and Culture is the continuous scroll feature that has one New York Times critic Roberta Smith saying “the images start sliding past like butter.” Each image is compressed into a flat tile that takes its place next to another flat tile, maybe of the same artist or a different artist? I don’t know. A featured theme of “Vermeer” is displayed on Arts and Culture’s home page. Does that have anything to do with the featured stories down below? I don’t know either- and that’s the problem. Arts and Culture is a database, a platform and most importantly, an interface. As stated earlier, an interface is intended to facilitate meaning to the viewer so they in turn can establish connections and create a nodes of meanings within their own information network. The visualizations of Arts and Culture inhibit such meaning-making. The flat tiles of images gives the viewer some information and no information at the same time. If you click the theme you are overburdened by information; Who was Johannes Vermeer?, “The complete works in augmented reality”, “The Mona Lisa of the North”, “Every painting in one place”, “Vermeer on screen,” “create your own masterpiece,” “Vermeer in pop culture,” “Justin Richburg x Vermeer,” “The Devil is in the Details”- the list goes on and on until you get to “explore more” which honestly is pointless given that Google has thrown so much information in your face already. By breaking up the information into seemingly “informational” nodes, Google is making it “difficult to identify the explicit transfer of knowledge because there is very limited interpretative text explaining the conceptual threads that tie items together.”[17] Barranha and Martins both argue that for virtual museums or collections such as the Arts and Culture, there is a need to “…opt (or should opt) for architecture which is flexible, transparent, distributed and open to collaboration and multiple realizations.”[18] The generalized “themes” created by Google also puts into discussion Malraux’s museé imaginaire, in which western culture has embedded a notion of stylistic continuities and generalities that result in a thematized style that ultimately leaves little to learn about.

The same search of Vermeer on the Met Collection and Artstor reveal a complete different perspective- and story behind both website’s design principles. The Met’s search results in 38 Vermeer paintings, that can be filtered by “object type/material”, “geographic location”, “date/era” and “department” as well as can be sorted by “Relevance”, “Title”, “Date”, “Artist” and “Accession Number.” The same search on Artstor has many more results and similar classifications with additional categories; “Contributor” and “Collection Type”. Such classifications draw a parallel to the three sign functions, icon, index and symbol, used in the semiotic process. As Irvine states, “they function as exemplary types (prototypes, ideal model forms) that the American viewer is given access to as part of a democratic cultural history, and thus open onto the whole network of symbolic values for art history.”[19] A viewer using the Met and Artstor can therefore understand the messages and meanings represented in the details of the artworks because of those categories of descriptions. Such connections are not made apparent on Google’s interface. It is unclear what concepts and relationships are being used or what organizing principles are being deployed when grouping the flat tiles into chunks on-screen or using the infinite scroll feature. When you click on Every Painting in one place, you see Vermeer’s paintings have been thematized one again according to “flirtation”, “Storytelling”, “Concentration”, “Correspondence”, etc. These themes created are further reaffirming Manovich’s concept of creating “pre-established identities.” The viewer is being told Google’s perspective on how to understand Vermeer whereas being shown the different types of interprenants that are available to their disposal. Google is actively changing how one interprets artists, paintings and art history in general when creating these preemptive themes and categories. Google becomes a ‘a zoo of oddities’ and an ‘endless seeing of the Internet’ where Arts and Culture represents ‘a kind of cultural illness’ that has infatuated the general public. [20]

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Another Perspective

There are other ways of designing and using Google’s technologies that can incorporate the missing semiotic layers apparent on the Met and Artstor’s website. The Met has gracefully and easily employed the basic meaning-making ‘system’ within their design of the website. While Google has shuffled its metadata in themes and groups, the Met re-organizes and interprets its various layers of data onto a comprehensive and interactive Heilbrunn Timeline of Art History. The timeline “pairs essays and works of art with chronologies telling the story of art and global culture through the Museum’s collection.”[21] While Arts and Culture is like Artstor in the sense that it is a database, it is also a quasi-archive. Alexandra Lussier-Craig speaks to how Arts and Culture (titled the Arts Project back in 2015) behaves as an archive:

“It behaves similarly to archives, in the plural, in the way that it treats the individual collections. The items in the Art Project are primarily arranged according to the institution that contributed the collection […] items are treated as though they originate in museum collections.”

Lussier-Craig adds that even though the Arts Project groups items according to the museum or institution that contributed them, “each of these collections the itemsʼ arrangement is far less structured . The arrangement of items within each collection is algorithmically generated[…]”[22] To combine the qualities of the Met’s archives and Artstor’s database with Google’s technologies, I propose that Arts and Culture creates a sophisticated timeline of sorts. Arts and Culture has already the metadata available to create this timeline. The timeline would be illustrated to incorporate the time period and geographic region together on one screen to create an indexical relationship among artworks. The contributing institutions would be another layer added onto the timeline, to locate where the artworks are in physical space. I would get rid of the Street View as it adds no information other than how you would see the artwork. Audio and videos that are already on the platform can become incorporated under each artwork on the timeline. Creating this chronological design and visualization will not only created usable information but also incorporate Google’s “large scale” design principle. Another feature that might be interesting to add is to take a picture of an artwork before you and the app will situate that within the timeline. This can create a real-time index to help the viewer understand the artwork before them.

Conclusion 

Google Arts and Culture is founded on the principle to “discover artworks, collections and stories from all around the world in a new way.” Yet  Google’s design, visualizations and organization of its metadata creates a “content management platform” for web-facing institutions. From the outset, the search engine was designed to avoid the subjectivity, maintenance expense, slow speed of indexing and limited scalability common to human-maintained directory sites.[23] Google’s focus on instantaneity and scalability neglects basic design principles that facilitate a user’s ability to understand and contextualize art and culture. The platform becomes a winding road of information with no directionality, that is rather funny for a search engine who prides itself on mastering the art of design. The Arts and Culture platform becomes an interesting case study of the power of design and the seemingly trivial details on our ability to form connections and create meaning. I wish to see the changes, if any, that Google will create to this platform that had such immense potential.

 

[1] Agostino, C. “Distant Presence and Bodily Interfaces: ‘Digital-Beings’ and Google Art Project.” Museological Review, no. No.19 (2015): p. 65.

[2] Beil, Kim. “Seeing Syntax: Google Art Project and the Twenty-First-Century Period Eye.” Afterimage: The Journal of Media Arts and Cultural Criticism. 40.4 (January/February 2013): 22-27.

[3] Irvine, Martin. “Malraux and the Museé Imaginaire: (Meta)Mediation, Representation and Mediating Institutions.” Google Docs. Accessed December 11, 2018.

file:///Users/adrianasensenbrenner/Downloads/Malraux-Imagined-Museum-Interface%20(1).pdf

[4]  Kristoffermilling. “Malraux and the Musee Imaginaire: The ‘Museum without Walls.’” Culture in Virtual Spaces (blog), June 17, 2014. https://culturalvirtualspaces.wordpress.com/2014/06/17/malraux-and-the-musee-imaginaire-the-museum-without-walls/.

[5] “Image and Data Resources.” The Metropolitan Museum of Art, i.e. The Met Museum. Accessed December 9, 2018. https://www.metmuseum.org/about-the-met/policies-and-documents/image-resources

[6] “TJWL Office of Digital Projects.” https://sites.google.com/site/departmentofcircumlocution/home.

[7] Undeen, Don. Dec. 10, 2018. Gmail Interview.

[8] https://www.metmuseum.org/about-the-met/policies-and-documents/image-resources

[9] “Metadata Policy & Standards | Artstor.” Accessed December 9, 2018. http://www.artstor.org/content/metadata-policy-standards.

[10] Hillis, Ken, Michael Petit, and Kylie Jarrett. Google and the Culture of Search. 1st ed. New York, NY, 10001: Routledge, 2012.

[11] Manovich, Lev. “The Language of New Media.” MIT Press, 2001, p. 226.

[12] Manovich, p. 129.

[13] Undeen, Don. Dec 10, 2018. Email Interview.

[14] Irvine, Martin. “Introduction to Affordances, Constraints, and Interfaces.” Google Docs. Accessed December 9, 2018. https://drive.google.com/file/u/1/d/0Bz_pbxFcpfxRaGprQ3BWY3VXQmc/view?usp=sharing&usp=embed_facebook. Page 7.

[15] Irvine, p. 7.

[16] “Google Visual Asset Guidelines.” CB. Accessed December 12, 2018. https://chrisbettig.com/google-visual-asset-guidelines/.

[17] Lussier-Craig, Alexandra. “Googling Art: Museum Collections in the Google Art Project,” n.d., 7.

[18] Barranha, Helena, and Susana Martins. “Beyond the Virtual: Intangible Museographies and Collaborative Museum Experience.” Reasearchgate.net, January 2015. https://www.researchgate.net/publication/280919854_Beyond_the_virtual_intangible_museographies_and_collaborative_museum_experience.

[19] Irvine, Martin. Art and Media Interfaced: From Studying Interfaces to Making Interfaces. Accessed December 11, 2018. file:///Users/adrianasensenbrenner/Downloads/Irvine-Making-Interfaces-Project%20(1).pdf

[20] “Is Google Bringing Us Too Close to Art?” The Daily Dot, March 21, 2013. https://www.dailydot.com/via/elkins-is-google-bringing-us-too-close-to-art/.

[21] “Home | Heilbrunn Timeline of Art History | The Metropolitan Museum of Art.” The Met’s Heilbrunn Timeline of Art History. Accessed December 13, 2018. https://www.metmuseum.org/toah/.

[22] Lussier-Craig, p. 8.

[23] Hillis, p. 36.

 

Bibliography 

“About the Google Cultural Institute.” Accessed December 12, 2018. https://www.google.com/culturalinstitute/about.

Agostino, C. “Distant Presence and Bodily Interfaces: ‘Digital-Beings’ and Google Art Project.” Museological Review, no. No.19 (2015): 63–69.

Beil, Kim. “Seeing Syntax: Google Art Project and the Twenty-First-Century Period Eye.” Afterimage: The Journal of Media Arts and Cultural Criticism. 40.4 (January/February 2013): 22-27.

Barranha, Helena, and Susana Martins. “Beyond the Virtual: Intangible Museographies and Collaborative Museum Experience.” Reasearchgate.net, January 2015. https://www.researchgate.net/publication/280919854_Beyond_the_virtual_intangible_museographies_and_collaborative_museum_experience.

“Google Visual Asset Guidelines.” CB. Accessed December 12, 2018. https://chrisbettig.com/google-visual-asset-guidelines/.

Hillis, Ken, Michael Petit, and Kylie Jarrett. Google and the Culture of Search. 1st ed. New York, NY, 10001: Routledge, 2012.

“Home | Heilbrunn Timeline of Art History | The Metropolitan Museum of Art.” The Met’s Heilbrunn Timeline of Art History. Accessed December 13, 2018. https://www.metmuseum.org/toah/.

“Image and Data Resources.” The Metropolitan Museum of Art, i.e. The Met Museum. Accessed December 9, 2018. https://www.metmuseum.org/about-the-met/policies-and-documents/image-resources.

Irvine, Martin. “Introduction to Affordances, Constraints, and Interfaces.” Google Docs. Accessed December 9, 2018. https://drive.google.com/file/u/1/d/0Bz_pbxFcpfxRaGprQ3BWY3VXQmc/view?usp=sharing&usp=embed_facebook.

Irvine, Martin. “Malraux and the Museé Imaginaire: (Meta)Mediation, Representation and Mediating Institutions.” Google Docs. Accessed December 11, 2018. file:///Users/adrianasensenbrenner/Downloads/Malraux-Imagined-Museum-Interface%20(1).pdf

Irvine, Martin. Art and Media Interfaced: From Studying Interfaces to Making Interfaces. Accessed December 11, 2018. file:///Users/adrianasensenbrenner/Downloads/Irvine-Making-Interfaces-Project%20(1).pdf

“Is Google Bringing Us Too Close to Art?” The Daily Dot, March 21, 2013. https://www.dailydot.com/via/elkins-is-google-bringing-us-too-close-to-art/.

Kristoffermilling. “Malraux and the Musee Imaginaire: The ‘Museum without Walls.’” Culture in Virtual Spaces (blog), June 17, 2014. https://culturalvirtualspaces.wordpress.com/2014/06/17/malraux-and-the-musee-imaginaire-the-museum-without-walls/.

Lussier-Craig, Alexandra. “Googling Art: Museum Collections in the Google Art Project,” n.d., 114. https://spectrum.library.concordia.ca/980818/1/LussierCraig_MA_S2016.pdf

Manovich, Lev. “The Language of New Media.” MIT Press, 2001, 68.

“Metadata Policy & Standards | Artstor.” Accessed December 9, 2018. http://www.artstor.org/content/metadata-policy-standards.

Proctor, Nancy. “The Google Art Project: A New Generation of Museums on the Web?” Curator: The Museum Journal 54, no. 2 (April 1, 2011): 215–21. https://doi.org/10.1111/j.2151-6952.2011.00083.x.

“TJWL Office of Digital Projects.” Accessed December 9, 2018. https://sites.google.com/site/departmentofcircumlocution/home.

Undeen, Don. Dec. 10, 2018. Gmail Interview.