In this paper, I am going to analyze an online exhibition of the painting Landscape with the Fall of Icarus as:
- A digital artefact mediating the social-cultural function of museums
- An interaction design as a cultural practice
The paper will include:
- An illustration of the multilayer of interfaces embedded in this digital artefact and the affordances implemented by this digital exhibition;
- An analysis of how this online exhibition remediated the social-cultural function of museums; what changes brought by the digital museum idea to on-site museum-visiting experience and human cognition of art history.
Digitization has made it possible for human to tour the world without leaving their desk. Digital representations can remediate almost every material artefact bit by bit. Remediation, in essence, is a process of creating an embedded structure of meaning systems as time goes by. Therefore, in order to de-black the remediating process, we need to figure out what’s going on in each layer of interfaces and how they connect to the larger meaning systems. The digital exhibition I want to discuss in this paper is a case of mediating a material painting in a physical-existed museum, a meta-(already)-meaning system. Therefore, before I go further into looking at the remediation of the social-cultural function of museum (as suggested in the title), I would do an visual illustration of symbols remediated and affordances implemented in each layer of interface. Then, based on the detailed illustration, I will do an analysis on how this online exhibition remediated the social-cultural function of museums. Done with every piece of detail about this digital exhibition, I want to compare it to another form of digital representation within a larger realm of virtual experience, to see what might be a special application of the technical mediation technologies for this instance. Finally, I will extrapolate from the case to discuss the differences between the virtual experience and on-site experience of visiting museum, and how digital exhibition changes human cognition of the art history.
Multilayer interfaces of the Digital Exhibition
“All human symbolic activity and representation in any material form are artefacts of symbolic cognition in a long cumulative continuum of technical mediations for human sign systems.” – Martin Irvine
Computation, software, digital media are artefacts of human symbolic cognition, and thus all computing is fundamentally “human(istic) computing. Human sign system is the fundamental meaning system underlying all of the technology mediated platform. Therefore, I will start off from the “human” interpretation of the painting and go further into the computer-mediated meta world.
First of all, for background information about the painting, please click here for details.
- Layer 1 The Painting: Landscape with the Fall of Icarus
The painting Landscape with the Fall of Icarus, featuring a subject from Greek mythology, depicted the fall of Icarus described by Ovid in his Metamorphoses. The painting included almost all the elements in the story, but at the same time, it is a very personal interpretation of the story.
The composition suggests that what Ovid accentuated in his passage, the fall of Icarus, was understated by Bruegel (the painter). In the bottom right-hand corner of the painting, the legs of Icarus himself can be seen desperately flailing in the air. What’s more, the rest of the world remains unperturbed; the ploughman, the shepherd, even the fisherman were showing indifference to the fall of the mythical hero.
According to Peirce’s triadic model of the sign as a process with material forms, the following diagram presents and explains the Object, Interpretant, Representamen, and the Semiosis.
The Object in this symbolic productivity process is the original Greek mythology, which is already a meaning system organizing assorted folklores into a formalized framework. Interpreted by Bruegel, he decomposed the established institutions and relationships of the characters and elements (the symbols) in the story and reconstruct the symbols into a personal interpreted meaning system via the proportion, composition, palette, etc. Understating the fall of Icarus, Bruegel emphasized the landscape and other characters in the painting. In terms of proportion, Icarus takes up only a small space of the painting while the other characters in the foreground and the panorama of the port in the background take up a large proportion of the whole painting. In terms of composition, the character of Icarus was depicted in the shadow on the right-bottom end of the diagonal – the least conspicuous position. In the foreground, Bruegel depicted a tangible reality that ordinary people were minding their own business: ploughman was steering his plough, shepherd was gazing at sky while grazing his sheep, and fisherman was engrossed in his toil; none of the characters pay attention to Icarus flailing in the water. In the background, Bruegel delineated a panorama of the port, island and the surrounding town, which shift audience’s attention to the marvelous landscape of the vast ocean. In terms of palette, Bruegel used three hues one after the other (browns, greens and blues) to create the impression of depth. Icarus was on the blurred boundary of two hues, where hardly can audiences notice. These correlation-making components reflected Bruegel’s responses to the story, which function as the Interpretant in the symbolic activity. The painting, the one hanging on a wall in the Royal Museums of Fine Arts of Belgium is the Representamen. It is the “sign vehicle” representing the material-perceptible structure with the interpretable features in the painting realm.
The painting provided a personal interpretation of the Ovid’s passage, which is a special genre of the Greek mythology. It is not a direct interpretation of the original story, but it serves to present an interface for audience to understand the Fall of Icarus story via a medium of painting.
- Layer 2 The Painting in the Museum
In the room where exhibit Bruegel’s painting artworks, the paintings are decorated (or protected) by wooden frames hanging on the wall. The walls are painted in peaceful green, providing a harmony tone for better visual experience for audiences while watching the artworks. Paintings are hang on the wall aligned with a horizontal line at a suitable height according to people’s watching habit. The paintings are not tightly attached to the wall; instead, in order to solve the problem that the painting may reflect light, there is certain degrees between the plane of the painting and the wall. In terms of location of each work, the most famous and delicate artwork of Bruegel, The Fall of Rebel Angel is placed at the center of the wall, which is faced with the designated auditorium (the deck in the middle of the room). Beside this painting, there are Landscape with the Fall of Icarus (right) and Winter Landscape with Bird-trap (left). These paintings work together to serve to map audiences into another meaning system to interpret any one of these paintings. The painting as a whole is a single symbol of the “room meaning system”. For instance, Landscape with the Fall of Icarus functions as a related artwork to the middle painting for the same topic of the fallen angel, at the same time, serves to guide audience through the room to the next painting, leaving room for audiences to imagine the possible correlation between these two artworks.
The painting Landscape with the Fall of Icarus with the wooden frame is a part of the Bruegel’s room, and functions as an interface for audiences to understand the larger meaning system of Bruegel’s artwork collection. Observed from the photograph (in fact, a digital copy) above, we have a general feeling of the installations of the Bruegel’s artwork room. Unconsciously, we have interpreted the painting through a digital interface, the computer screen. This brings us to the next layer of interface – digital representation.
- Layer 3 The Digital Exhibition: Landscape with the Fall of Icarus (… and the surrounding controversy)
There are so many possible ways to present a painting in a digital form, photograph, video, even audio commentary. The digital exhibition Landscape with the Fall of Icarus (… and the surrounding controversy) combines several media together to present a in-depth interpretation of the painting.
“Drawing on a wide spectrum of virtual and on-site experiences, this unique initiative offers everyone the chance to immerse themselves in Bruegel’s works by honing in on the details of each painting and accessing expert knowledge. By delving deeper into the artist’s world, the viewer will discover the unexpected elements in Bruegel’s works which constitute the pinnacle of the Flemish master’s craft. … This innovative concept is the fruit of in-depth thinking on current transformations in the field of museology as it adapts to the digital era.“
From the description, we instantly know that we have immersed in a 3-step meta process when we watch the painting (layer 1) that resides in the museum (layer 2) on the computer screen (layer 3) in the context of the digital exhibition as a part of the whole project.
Accessing the Bruegel. Unseen Masterpieces virtual exhibitions on the Google Arts and Culture platform, we are interact with a material screen which is a pixel-mapped substrate. What makes the digital representation different is the difference in material representamen. When visiting the on-site museum, what we see using our naked eyes is an artwork painted on a canvas using oil painting. However, when we watch the painting in the online exhibition, we are looking at a set of pixels, which remediate the original painting into a digital copy. These pixels are organized in a specific way (abstraction, recursion, …) to resemble the original painting.
Apart from the painting itself, other functions of museum are also imitated by the virtual online museum. The commentary text and video helps explain the composition and the dynamic path that eye follows across the composition, which is also an example of imitating the description in the card and the curators in the physical museum:
The sequence of the exhibition and other paintings involved in the digital presentation is an example of imitating the installations in the on-site museum as well. In the physical museum, paintings are organized by author, at the same time, paintings of the same authors are placed according to specific logic. In the digital exhibition, paintings that are related to the presented painting, according to assorted needs, will be added into the exhibition:
Therefore, the digital representations are taking advantage of preexisting interfaces.
But it also has something new, which an actual museum cannot do. In Alan Kay’s vision, this is a win for the virtual museum.
The high resolution of the photographic copy of the painting allows people see even more details than what we can see using our naked eye in the museum. With the zoom-in function, we can even see the little cracks of the oil paints:
Also, the digital exhibition enable to guide audiences to specific part (often the neglected details) of the painting:
The technical mediation enable audiences to see more than we could see: the head of a man lying in the undergrowth, the horse equipped with blinkers, a knife and a sword on the same rock, a seed bag leaned against a rock, the partridge, the fisherman, the mythology hero falling into the water, the dark ewe stands amongst the sheep, the mediating shepherd, the wind-filled sails and masts, the remote island, the surrounding town, and the sun disappearing over the horizon – all these details are emphasized by zooming in, which enable audiences to get a detailed appreciation of the intricate painting.
Affordances of the Digital Exhibition
"Think of the computer not as a tool but as a medium." - Brenda Laurel
The platform for the digital exhibition resides in computer. “All digital artifacts are made of a common substance: programmable bits that can be used for symbol manipulation.” In essence, computer is a common medium of representation. According to Murray(1997), computer is encyclopedia, spatial, procedural and participatory. Based on the four representational affordances, the following analysis of the digital exhibition will focus on the interaction design as a cultural practice.
- Procedural Affordances
The digital exhibition is able to represent and execute conditional behaviors. The experience of visiting the digital exhibition of the painting takes the format of sequential slideshow.
The progress bar at the bottom is based on the metaphor of any sequential visual medium, exploiting the procedural (and participatory) affordances of the medium:
The progress bar is the embodiment of the abstraction and algorithm of the dynamics of this digital presentation. Interacting with the bar, audiences are interacting with a conceptualized model of executing conditional command initiated by the audiences. The jump-off window suggests the flexibility of the program, providing possibilities for audiences to jump back and forth between sections in a unisequential design. Also, the jump-off window is an abstraction of that specific slides, which signifies that specific slide and gives audiences a preview of the content.
- Participatory Affordances
“The relationship between the interactor and any digital artifact is reciprocal, active, and open to frustrating miscommunication.” The participatory design concept of this digital exhibition is displayed by the automatic language switch based on the language settings of different login accounts. This is one of the preset algorithm.
Sometimes the script is more flexible. Some digital conventions are so familiar that they script us in a transparent way. For example, on the first page of the digital exhibition, there is an arrow on the right of the screen:
To start off, audiences automatically relate the arrow with the starting: the arrow at the right edge of application windows cue us to scroll the screen; when we put the cursor on this symbol, the arrow turns into a hand, which cues us to click on the symbol.
The same logic, the restart arrow serves to the restarting of the procedural affordances.
The loading circles signifies “please wait”:
These digital conventions aids the participatory design; the interactive design employs these conventions to stimulate human actions to realize their expectations.
- Encyclopedic Affordances
The word encyclopedic emphasizes on computer’s capacity of storage and transmission, and its inheritage of the tradition of knowledge collecting, preserving, and transmitting.
The digital presentation of this painting is a part of a larger project remediating Bruegel’s artworks. Within the large project, exhibitions are linked one another according to meticulous segmentation and classification. For example, exhibitions can be classified by stories:
Videos embedded in a specific exhibition could be reorganized into a video section:
Landscape with the Fall of Icarus, together with other paintings consist of this collection of paintings with the label of Bruegel’s artworks. However, this is not the only way for Landscape with the Fall of Icarus to present among numerous paintings. For example, this painting can be assigned to another project collecting all the adaptions of the Greek mythology, which can include music works, poems, passages, etc. The flexibility in positioning the paintings in the huge collection, displays the inclusiveness of the encyclopedic affordances provided by the computer.
- Spatial Affordances
Computer creates virtual spaces that are navigable by the interactor, which rests upon the procedural and participatory affordances of computation.
Visual design manipulate the space to represent the hierarchy of the items. By looking at the horizontal-placed paintings, we know the equal relationship among the paintings; the consistency in design suggests the equal level in the structure. We are not going to lose navigation during the browsing experience, because the coherent spaces and the scripted digital conventions.
A remediation of the social-cultural function of museums
The Museum is an organizational system. How museums organize the paintings is highly correlated to the social and culture institutions. Museums can be considered as implementations of human cognitive art history. This predominant museum idea “preceded and pre-interpreted any artefacts selected for representation”. Museum establishes the standard: it set up limited categories for selecting and assigning artefacts to some established categories (which we take for granted now): periods, styles, genres, cultures, etc. The institutionalized idea of museum makes it properly to suggest that museum is an ideal meaning system: it turns the actual museums into embodiments of its conceptual model, in the way of determining what to present and how to present.
In terms of the social-cultural functions of museum, the most primary goal for museum is to provide a ordered representation of art history. Most of the museums are divided into relative independent spaces in order to enable visitors to make sense to the installations and organizations of the artworks. Each space creates a real space for a meaning system, in which establish correlations among artworks within the same space. The meaning system is open, allowing unlimited interpretations of each artwork itself and the relationship between any pair of artworks. The meaning system is going through an on-going changing process, for the possibilities of occurrence of further interpretants expressible in new or additional signs.
How to evaluate the performance of the digital exhibition in fulfill the social-cultural functions of museums? The digital exhibitions earn advantages in flexibility, accessibility, and compatibility.
Flexibility. “Artworks continue to be receives as art works by means of further technological mediation and representation, but are also continually reinstantiated as art works by the institutional framing of “art history” and the museum function in culture.” There are so many ways of presenting only one painting. For Landscape with the Fall of Icarus, it can be assigned to the collection of Bruegel’s artworks, it can be assigned to the collection of Renaissance period paintings, it can be assigned to the collection of artworks adapted from Greek mythology. What digital museum can do is to reorganize the collections by manipulate symbols on the website, totally getting rid of the trouble caused by the material artworks.
Accessibility. The virtual exhibition helps the artworks go beyond the boundaries of time and space. Make the artwork accessible to people with a terminal device with Google Arts and Culture platform.
Compatibility. With multimedia, text, image, audio, video, involving in the digital exhibition, these media also present an embedded relationship in the exhibition. A text annotating the image, a video commentary interpreting the character in the picture … all these work well together thanks to the inclusiveness of the semiosis – an emergent process.
Concerns. Digital museums exist in an intricate network connecting to unlimited cultural symbols and dynamic meaning systems. When we depict the digitalizing process of museum, we always use the word “simulation”. Even with the high level of resemblance, when visiting virtual museum, we are still conscious about something different. It suggests the deficiency in the current condition how human manipulate symbols in an opening and emerging interface. Computing is fundamentally “human computing”, the reason we feel it is “non-human” is related to our current ideological, political-economic conditions, and processes of education and socialization about computers. Even though we have concerns, we are always open to the possibility of remediating human meaning system into computer version. Digital museum is never an enemy, so does computer.
Clark, A., & Chalmers, D. (1998). The Extended Mind. Analysis, 58(1), p. 7-19.
Janet Murray, Inventing the Medium: Principles of Interaction Design as a Cultural Practice. Cambridge, MA: MIT Press, 2012.
Martin Irvine (2016), André Malraux, La Musée Imaginaire (The Museum Idea) and Interfaces to Art. Communication, Culture & Technology Program, Georgetown University.
Martin Irvine (2016), Introduction: Toward a Synthesis of Our Studies on Semiotics, Artefacts, and Computing. Communication, Culture & Technology Program, Georgetown University.