Author Archives: Alexander Macgregor

Do Robots Speak In Electric Beeps?: Artificial Intelligence & Natural Language Processing (Alexander MacGregor)

Abstract

When we think of the term “artificial intelligence”, a certain array of images often comes to mind. Be it humanoid robots doing housework, or sinister machines ruthlessly stepping over humans on their path to dominance, much of the public discourse surrounding this term has been driven by our media and arts, and all the artistic license that comes with them. But if we explore the history of artificial intelligence and its applications, we see that the tasks we have traditionally attempted to offload onto AI are less whimsical, but perhaps just as fundamental to our experience as semiotically capable cognitive beings. In this paper, I will trace this history while focusing in on the specific task of natural language processing (NLP), examining the models we use to offload the linguistic capabilities that we, as humans running OS Alpha, obtain at birth.

Brief History of Artificial Intelligence

Although artificial intelligence only became a formal academic discipline at the Dartmouth Conference of 1956, the ideas and concepts that came to shape the field were present as far back as Ancient Greece. It was Aristotle’s attempts to codify “right-thinking” that first laid ground for much of the logic-based framework that AI philosophy resides within (Russell & Norvig, 17). In what was perhaps the first step in the history of AI related cognitive offloading, 14th century Catalan philosopher Ramon Llull conceived of the mechanization of the act of reasoning in his Ars generalis ultima. In the 16th Century, Leonardo da Vinci outlined the designs for a mechanical calculator, which was eventually realized in 1623 by a German scientist by the name of Wilhelm Schickard, although it was Blaise Pascal’s “Pascaline” calculator built 20 years later that is more widely recognized (Russell & Norvig, 5).

Leonardo da Vinci's sketch of a mechanical calculator

Leonardo da Vinci’s sketch of a mechanical calculator

Around the same time, English philosopher Thomas Hobbes, German philosopher Gottfried Leibniz, and French philosopher Rene Descartes were each advancing the discourse on this topic. In the introduction to his seminal book, Leviathan, Hobbes asks the reader “For seeing life is but a motion of limbs, the beginning whereof is in some principal part within, why may we not say that all automata (engines that move themselves by springs and wheels as doth a watch) have an artificial life? For what is the heart, but a spring; and the nerves, but so many strings; and the joints, but so many wheels, giving motion to the whole body” (Hobbes, 7). Leibniz was attempting to discover a “characteristica universalis”, which would be a formal and universal language of reasoning allowing for all debate and argument to be unambiguously reduced to mechanical operation (Levesque, 257). It is impossible to ignore the impact of Rene Descartes on the formation of this field. While he is perhaps most well known for his theory of mind-body dualism, he also developed more directly automation based observations, such as conceptualizing animals as machines (Russell & Norvig, 1041).

In the 19th and 20th Centuries, we began to see attempts to build machines capable of executing on the ideas promoted by previous philosophers. Charles Baggage’s Difference Engine was an attempt to mechanize computational work previously done by “human computers”. Babbage also designed, but was never able to build, what he called an “Analytical Engine”, which was really the first design for what we now know as a “general purpose computer” (Dasgupta, 27). The code breaking frenzy of the Second World War provided an environment in which many computational advances were made, and there was perhaps no more influential figure to emerge from this era than Alan Turing. Considered by many to be the father of modern computing, Turing’s work during this era was crucial to the prismatic explosion of AI and computing advancements that we saw in the latter half of the 20th century.

London Science Museum's model of Charles Babbage's Difference Engine

London Science Museum’s model of Charles Babbage’s Difference Engine

Brief History of Natural Language Processing

In 1950, Alan Turing published his paper “Computing Machinery and Intelligence”, which proved to be pivotal in the yet-to-exist field of artificial intelligence. In the paper, Turing contemplated the possibility of building machines that are capable of “thinking”, and introduced the concept of the Turing Test as a way to determine whether a machine was exhibiting such traits to the extent they were indistinguishable from a human being (Dennett, 3-4). When we, as humans, engage in the exchange of ideas, symbols and messages that assure us of our respective intelligence and “personhood”, we do it through the interface of language. It is truly one of the key enablers of our semiotic skill-set. So if we are to create artificial intelligence, then the ability to communicate signs, symbols and meaning through language is a top priority.

Four years after Alan Turing published his seminal paper, IBM and Georgetown University held a joint exhibition to demonstrate their fully automatic machine translation capabilities. Using an IBM 701 mainframe computer, the operator was able to translate Russian sentences comprising of 250 words and six grammatical rules into English (Hutchins, 240). This “breakthrough” was highly publicized, and led the authors and press to make bold predictions about the immediate future of artificial intelligence and machine translation, but the reality of the situation was much less grandiose. The program was only able to seem successful by severely restricting the grammatical, syntactic and lexical possibilities far short of any realistic conceptions of a truly artificial intelligence.

Newspaper clipping detailing the IBM-Georgetown Machine Translation Experiment

Newspaper clipping detailing the IBM-Georgetown Machine Translation Experiment

Successes & Limitations

This was, in fact, the story of most of the NLP attempts made during the early days of AI. Although successful, programs like Daniel Bobrow’s STUDENT, designed to solve high school level algebraic word questions, and Joseph Wizenbaum’s ELIZA, famously used to simulate the conversation of a psychotherapist, were still operating within very strict linguistic constraints. ELIZA, for example, wasn’t capable of analyzing the syntactic structure of a sentence or deriving its meaning, two elements that are crucial for true language comprehension. Nor was it able to extrapolate on its linguistic inputs to explore its environment. ELIZA was, in fact, the first chatterbot, designed only to respond to certain keyword inputs by with a pre-set answer. (Bermudez, 31)

The limitations of these early NLP systems gave rise to the micro-world approach of MIT AI researchers Marvin Minsky and Seymour Papert, most famously exhibited in fellow MIT researcher Terry Winograd’s SHRDLU program. Micro-worlds were problems that would require real intelligence to solve, but were relatively simple and limited in scope (Russell & Norvig, 19-20). Papert initially saw micro-worlds as a way to connect computing to the hard sciences, where simple models were often used to derive fundamental scientific principles (Murray, 430). Winograd’s SHRDLU program put this approach to test, and was one of the earliest attempts to get machines to do true natural language processing, which meant the system would “report on its environment, plan actions, and reason about the implications of what is being said to it” (Bermudez, 32). SHRDLU was a success and prompted a lot of excitement around the field of NLP, but because it was so dependent on syntactic analysis and was operating within a micro-world, many of the same limitations from the early machine translation attempts were present in SHRDLU (Russell & Norvig, 23). The simplicity of the micro-world constraints meant SHRDLU’s language was correspondingly simple, as it could only talk about the events and environments of the micro-world it inhabits.

The micro-world of Terry Winograd's SHRDLU program

The micro-world of Terry Winograd’s SHRDLU program

Even with these constraints, SHRDLU did contribute to three major advancements in the evolution of NLP. Firstly, it displayed that the conceptual and theoretical rules of grammar could be practically executed in an NLP program. Secondly, it showcased the approach of breaking down cognitive systems into distinct components that each executes a specific information-processing task. Thirdly, it was built on notion of language as an algorithmic process (Bermudez, 33). These factors would set the stage for how NLP programs would be built moving forward.

In many ways, machine translation and NLP followed a similar historical trajectory as speech recognition attempts. The early excitement prompted by the information theory and word-sequencing models of the 1950s would be tempered in favour of the highly knowledge-intensive and specific micro-world approach of the 1960s. The 1970s and 1980s saw a push for commercialization of these previously academically restricted programs, but also, more importantly, the rise of the neural network approach. (Russell & Norvig, 26)

This prompted a civil war of sorts between the “connectionist” camp advocating for approaches like neural networks, the “symbolic” camp advocating symbol manipulation as the best frame to understand and explore human cognition, and the “logicist” camp advocating a more mathematical approach. Even to this day there has been no truly definitive resolution to this conflict, but the modern view is that the connectionist and symbolic frameworks are complementary, not incompatible. (Russell & Norvig, 25)

Enter Stage Right: Artificial Neural Networks

When looking at the modern natural language processing landscape, one sees that artificial neural networks (ANNs), particularly recurrent neural networks, are the en vogue computational approach (Conneau, Schwenk, Barrault & LeCun, 1). Originally inspired by the mid-20th century neuroscience discovery that mental processes are composed of electrochemical activity in networks of brain cells called neurons, artificial intelligence researchers aimed at modeling their approaches after this system (Norvig, 727). Two of the most prominent early advocates of this method were Warren McCulloch and Walter Pitts, who in 1943 designed a mathematical and algorithmic computational model for these neural networks. Yet due to a lack of research funding and the publication of an influential research paper by Minsky and Papert in which they detailed the limitations of the computational machines being used to run neural networks at that point in time, the neural network approach was sidelined until the late 1980s when the neural network back-propagation learning algorithms first discovered in 1969 by Arthur Bryson and Yu-Chi Ho were reinvented (Norvig, 761). The very influential textbook “Parallel Distributed Processing: Explorations in the Microstructure of Cognition” by David Rumelhart and James McClelland also helped to reinvigorate the AI community’s interest in this approach.

Model of a feedforward neural network

Model of a feedforward neural network

Yet a lack of computational power would mean other machine learning methods, such as linear classifiers or support vector machines, would hold precedence over neural networks. That was the case until the computational processing hardware landscape evolved to a state where technologies such as GPUs and computational approaches like distributed computing made it possible for neural networks to be deployed on the scale necessary to handle tasks like natural language processing.

So How Exactly Do These Neural Networks Work?

In a nutshell, a neural network is a collection of computational units connected together that “fires” an output when its inputs cross a predefined hard or soft threshold. (Russell & Norvig, 727-728) The earliest models were designed with only one or two layers, but they ran into limitations when it came to approximating basic cognitive tasks. Later models would solve this problem by adding a layer of “hidden units” and giving the nets the ability to adjust the connection weights.

This video is a helpful visual introduction to the concept:

What Does Linguistics Have To Say About All This? 

Due to the brain being a massively parallel organ with neurons apparently working independently of each other, artificial neural networks have been used as an approach to computationally offload many of the cognitive functions the brain performs, such as pattern recognition, action planning, processing and learning new information, and using feedback to improve performance (Baars, 68). Language is an inherently symbolic activity, so if we are to offload the task of natural language processing to artificial intelligence, the capability of neural nets to be translated into symbolic form, and for symbolic forms to be translated back into neural nets, is a feature that makes this approach very attractive.

In addition to being symbolic, language is also a practical, rule-governed activity. It was Noam Chomsky, often considered to be the father of modern linguistics, who first attempted to discover why it is that language operates in the manner it does (Bermudez, 16). In his groundbreaking book Syntactic Structures, Chomsky makes a distinction between the deep structure and surface structure of a sentence. The former is referring to how the basic framework of a sentence is governed by phrase structuring rules operating at the level of syntactic elements such as verbs, adjectives and nouns. The latter refers to the organization of words in a sentence, which must abide by the sentence’s deep structure. The important point to note here is that language is conceived of as hierarchical, algorithmic, and rule-based. The rules extend to not only grammar and syntax, but also individual words and contextual meaning.

Chomsky's famous grammatically correct, yet semantically unintelligible sentence.

Chomsky’s famous grammatically correct, yet semantically unintelligible sentence.

Adding onto Chomsky’s insights was his student at MIT in the 1960s, Ray Jackendoff, whose “parallel architecture” linguistic model sought to debunk the syntactocentric models of old and promote a framework positing the independent generativity of the semantic, phonological and syntactic elements of language (Jackendoff, 107). From Jackendoff, we can conceptualize language as a combinatorial structure in which elements work in a parallel fashion to produce expression and meaning. Again, a processing architecture is at the basis of this framework.

Jackendoff's model of Parallel Architecture

Jackendoff’s model of Parallel Architecture

ANNs and NLP, Live Together in Perfect Harmony? 

While artificial neural networks do not have linguistic rules inherently built into them like the human brain is thought to, they have been shown to capable of modeling complex linguistic skills. The simple recurrent neural networks designed by Jeff Elman have been successful trained to predict the next letter in a series of letters, and the next word in series of words. Studies done by developmental psychologists and psycholinguists that examine the patterns children display when they learn languages have shown that in many features of language acquisition, human beings follow a very archetypal trajectory. One example would be making similar types of grammatical construction mistakes at similar learning stages. When artificial neural network researchers analyzed the degree to which their models can reproduce these features of language processing, they found similarities between how the neural networks learn and how children learn. (Bermudez, 246)

Verb tense is another specific area in which much research has been conducted testing the natural language processing capabilities of artificial neural networks. While the actual computational process is quite complex, it essentially boils down to the theory that children learn the past tense in three distinct stages. In the first stage, they use only a small number of verbs in primarily irregular past tenses. In the second stage, the number of verbs in use expands and they formulate past tense in the “standard stem + -ed” format. In the third stage, as they learn more verbs, they correct their “over-regularization errors”. Where artificial neural nets come in is in their ability to develop a similar learning pathway without needing to have linguistic rules explicitly coded in them. (Bermudez, 247)

*Record Scratch* Let’s Pump The Brakes A Bit

It is important to note at this juncture that artificial neural nets are nowhere close to mirroring the brain’s ability to perform these tasks, and neither is that the goal. The aim is to enable machines to engage in natural language processing, regardless of the similarity of method to how humans engage in natural language processing. There is no imperative to follow the same rule-based framework for language that humans use, because artificial neural networks are not attempts to reconstruct the human brain or even mirror its intricacies, but rather to behave in accordance with rule-governing aspects of linguistic understanding, even though they do not represent those rules. They are simply an approach inspired by this one element of how our brains process information. Compared to the massively complex brain, most of the simulations run through artificial neural nets are relatively small-scale and limited. But for certain cognitive tasks, neural nets have proven to be more successful than programs using logic and standard mathematics (Baars, 68). The neural network approach provides certain affordances that make computation of this scale and nature more effective, such as its ability to handle noisy inputs, execute distributed and parallel computation, and to learn. It is not imperative to resolve any conflicts between the way we believe the brain to be operating and the way neural networks are architected. As Texas A&M University Professor of Philosophy Jose Bermudez states:

 “The aim of neural network modeling is not to provide a model that faithfully reflects every aspect of neural functioning, but rather to explore alternatives to dominant conceptions of how the mind works. If, for example, we can devise artificial neural networks that reproduce certain aspects of the typical trajectory of language learning without having encoded into them explicit representations of linguistic rules, then that at the very least suggests that we cannot automatically assume that language learning is a matter of forming and testing hypotheses about linguistic rules. We should look at artificial neural networks not as attempts faithfully to reproduce the mechanics of cognition, but rather as tools for opening up new ways of thinking about how information processing might work.” (Bermudez, 253-254)

What Does The Future Hold?

The future of natural language processing and artificial intelligence is sure to be shaped by the tech giants currently absorbing research talent at a vociferous rate. Companies like Google, Facebook, Microsoft, Amazon, and Twitter have all identified businesses uses for this technology. For Facebook, it’s their DeepText engine that filters unwanted content from their users’ newsfeeds. Google’s uses for this technology are varied, but include user experience in apps, search, ads, translate and mobile. Microsoft’s research team is looking to this technology to design and build software.

This corporate takeover has not gone without concern. For the majority of the history of AI research, universities and public research institutions have been the incubation chambers for breakthroughs, and they have a far more transparent culture than corporations driven by profit maximization and incentivization towards harbouring trade secrets. In order to assuage this concern, many of these companies have embraced an open source culture when it comes to their findings. They have encouraged their researchers to publish and share their work (to an extent) with the broader community, under the rationalization that a collegial atmosphere will create gains that everyone can utilize. Bell Labs and Xerox PARC have become the aspiration models, as it was precisely the accessibility and open environment of these institutions that allowed innovation to thrive.

Xerox PARC's infamous beanbag chair meetings

Xerox PARC’s infamous beanbag chair meetings

This is surely one of the main reasons we’ve witnessed an exodus of academic researchers into these companies. Two of the most prominent names in the field right now are Geoffrey Hinton and Yann LeCun. Hinton, a former University of Toronto professor considered to be the godfather of deep learning, was scooped up by Google to help design their machine learning algorithms. LeCun, a former New York University professor, is now the Director of AI Research at Facebook. Undoubtedly, the extremely large data sets these companies have collected are also a powerful draw, as they allow for training bigger and better models. When asked what he perceives the future of NLP and artificial neural nets to be, Hinton answered:

“For me, the wall of fog starts at about 5 years. (Progress is exponential and so is the effect of fog so its a very good model for the fact that the next few years are pretty clear and a few years after that things become totally opaque). I think that the most exciting areas over the next five years will be really understanding videos and text. I will be disappointed if in five years time we do not have something that can watch a YouTube video and tell a story about what happened” (Hinton).

A similar question was posed to University of Montreal professor of Computer Science Yoshua Bengio, also considered to be one of the preeminent figures in the field right now, to which he responded:

“I believe that the really interesting challenge in NLP, which will be the key to actual “natural language understanding”, is the design of learning algorithms that will be able to learn to represent meaning” (Bengio).

Where Does “Meaning” Fit Into The Equation? 

If meaning-making is the ultimate purpose of language, then the true holy grail of natural language processing through artificial neural networks is unsupervised learning. The majority of the current models being employed utilize a supervised learning technique, meaning the network is being “told” by the designers what mistakes and errors it is making (Bermudez, 220). With unsupervised learning, the training wheels come off and the network receives no supervisory external feedback, learning on its own (Arbib, 1183). According to University of California, Berkley Professor Michael I. Jordan, one of the leading researchers in the fields of machine learning and artificial intelligence, unsupervised learning is “presumably what the brain excels at and what’s really going to be needed to build real “brain-inspired computers”” (Jordan).

Conclusion

Journeying through the history of artificial intelligence, we saw just how broad and deep the philosophical roots of this field are. From canonical figures like Aristotle and Descartes to modern heavyweights like Turing and Chomsky, the scope of thinkers contributing to artificial intelligence advancements is proof positive of its interdisciplinary nature. The problems posed by the quest to cognitively offload key human faculties require answers drawing from such diverse fields as computer science, neurology, linguistics, mathematics, physics, and engineering. Out of all the cognitive tasks we have attempted to offload to AI, natural language processing is perhaps the most important. As the renowned cognitive scientists, linguist and psychologist Steven Pinker has stated:

“For someone like me, language is eternally fascinating because it speaks to such fundamental questions of the human condition. Language is really at the center of a number of different concerns of thought, of social relationships, of human biology, of human evolution, that all speak to what’s special about the human species. Language is the most distinctively human talent. Language is a window into human nature, and most significantly, language is one of the wonders of the natural world.” (Big Think)

It is only natural that in the quest to technologically mediate this uniquely human skill, we looked to our own brain for inspiration. But while certain neurological features have surely inspired artificial neural networks, the dominant natural language processing model, AI designers, researchers and architects are not bound by them. The goal is to get computational machines to process natural language. How one gets there is relatively inconsequential. Due to the exponential increase in size and quality of the data sets used to train artificial neural nets, we are sure to see some exciting advances in natural language processing over the next few years, but as of now, the ultimate goal of a “strong AI” capable of dealing with the concept of linguistic meaning remains behind the “wall of fog”.

Works Referenced

  1. Russell, Stuart J., and Peter Norvig. Artificial Intelligence: A Modern Approach. Third ed. Upper Saddle River, NJ: Prentice Hall, 2010.
  2. Hobbes, Thomas. Leviathan. Urbana, Illinois: Project Gutenberg, 2002. Web. 2 December, 2016.
  3. Levesque, Hector J. “Knowledge Representation and Reasoning.” Annual Review of Computer Science1 (1986): 255-87. Web. 6 Dec. 2016.
  4. Dasgupta, Subrata. It Began With Babbage: The Genesis of Computer Science. Oxford: Oxford UP, 2014.
  5. Dennett, Daniel C. Brainchildren: Essays On Designing Minds. Cambridge, MA: MIT, 1998.
  6. Hutchins, John. “From First Conception to First Demonstration: the Nascent Years of Machine Translation, 1947-1954. A Chronology.” Machine Translation, vol. 12, no. 3, 1997, pp. 195–252. Web. 5 Dec. 2016.
  7. Bermúdez, José Luis. Cognitive Science: An Introduction to the Science of the Mind. Cambridge: Cambridge UP, 2010.
  8. Murray, Janet. Inventing the Medium. Cambridge, MA: MIT, 2012.
  9. LeCun, Yann, et al. “Very Deep Convolutional Networks for Natural Language Processing.” ArXiv: Computation and Language, 2016. Web. 15 Dec. 2016.
  10. Jackendoff, Ray. Foundations of Language: Brain, Meaning, Grammar, Evolution. Oxford: Oxford UP, 2002.
  11. Baars, Bernard J., and Nicole M. Gage. Cognition, Brain, and Consciousness: Introduction to Cognitive Neuroscience. Burlington, MA: Academic/Elsevier, 2010.
  12. geoffhinton [Geoffrey Hinton]. “AMA Geoffrey Hinton.” Reddit, 10 Nov. 2014, https://www.reddit.com/r/MachineLearning/comments/2lmo0l/ama_geoffrey_hinton/clyjm11/. Accessed 13 Dec. 2016.
  13. yoshua_bengio [Yoshua Bengio]. “AMA: Yoshua Bengio.” Reddit, 24 Feb. 2014, https://www.reddit.com/r/MachineLearning/comments/1ysry1/ama_yoshua_bengio/cfpmo29/ . Accessed 13 Dec. 2016.
  14. michaelijordan [Michael I. Jordan]. “AMA: Michael I. Jordan.” Reddit, 11 Sep. 2014, https://www.reddit.com/r/MachineLearning/comments/2fxi6v/ama_michael_i_jordan/ckep3z6/. Accessed 13 Dec. 2016.
  15. Big Think. “Steven Pinker: Linguistics as a Window to Understanding the Brain.” Online video clip. Youtube. YouTube, 6 October 2012. Web. 15 Dec. 2016
  16. Arbib, Michael A. Handbook of Brain Theory and Neural Networks. 2nd ed. Cambridge, MA: MIT, 2003.
  17. Wilson, Robert A., and Frank C. Keil. The MIT Encyclopedia Of The Cognitive Sciences. Cambridge, MA: MIT, 1999.
  18. Murphy, Kevin P. Machine Learning: A Probabilistic Perspective. Cambridge, MA: MIT, 2012.
  19. Marcus, Gary F. The Algebraic Mind: Integrating Connectionism and Cognitive Science. Cambridge, MA: MIT, 2001.
  20. Frankish, Keith, and William M. Ramsey. The Cambridge Handbook of Cognitive Science. Cambridge: Cambridge UP, 2012.

The Girl With The Pearl Selfie Stick – Ojas + Alex

Google Art as Software Remediation

When art is remediated digitally in computer software, the medium opens up editing affordances, blurring the line between type and token. That is to say the painting, itself a token, is typified when it is represented digitally. The digital painting is neither the same as, nor different from the original. In the case of the Vermeer work above, though Girl with a Pearl Earring is a token, it functions as a type for the token of the second image. The second image, in addition to the humor, blends the medium of painting with photography. Though the painting becomes one of the subject taking a mirror selfie, the image, in retaining the visual qualities of a painting, draws a relationship between photography and painting. In this way, it is a token for the types of artistic representation being engaged; however, it also adds new sign processing potential to the original painting. And this is one of the key transformations of artistic representation in Google Art and Culture. The pieces in the museum, when represented digitally, in being remediated and re-represented go through a process of re-typification and potential for re-tokenization.

Parallel Architecture

The three images above represent three layers of abstraction, all of which are in a greater network of sign systems. The sign systems work concurrently in a parallel architecture in which all three layers are engaged simultaneously in the sign processes of any one of images individually. This is because the layers of abstraction and representation are in an evolutionary ecosystem of sign systems. Any changes or re-representations of previous works affect the other representations. So in the case of Venus of Urbino, it is re-instantiated as a symbol in Tribuna of the Uffizi, and again in an extra meta layer of abstraction in Google Art and Culture, engaging and transforming the original.

Meta-Information & The Dialogic Process

We were interested in exploring the notion of the museum as an inherently dialogic institution. Malraux seems to place a primacy on the intertextual and inter-cultural dialog occurring when you juxtapose works of art in the context of a museum. By housing these pieces of art in the same physical and conceptual construct, we necessarily situate them in a historical, yet living, lineage. We are asked to compare, contrast, critique and engage with this lineage as we experience the museum. But meaning doesn’t arise in a vacuum. We need access to the meta-information of each work of art to effectively dialog. This is why tagging is such a crucial component in the digital mediation of Malraux’s “museum without walls”. It provides us with access to the stylistic, temporal, historical and otherwise contextual information surrounding each work of art. The potential for dialogic exploration is thereby enhanced, allowing the viewer to engage with the artistic web of meaning and analyse the art beyond a superficial level.

 

Google Art in relation to Malreaux’s “museum without walls”

As an interface, the Google Art Project is not only following in Malraux’s footsteps, it is expanding the size of the imprint. The constraints of the art history textbook and its photographic representation of the artistic work are loosened by digital mediation. Malraux conceives of the museum as a conceptual information system for the classification and organization of artefacts, and through its ability to delve deeper into various classifications, the Google Art Project provides us with not only more specificity, but also a more personalized and individualized framework of curation. It is important to remember that with the Google Art Project, we are still operating within the “museum” construct that provides a framework within which we engage with the artwork. We can expand the realm of what constitutes “art”, but we are still confronted by many of the same problems Malraux encountered, such as the reconceptualization into generalized abstractions and the alteration of historical contexts. A museum without walls is still a museum.

References

  1. Irvine, Martin. 2016. “From Samuel Morse to the Google Art Project: Metamedia, and Art Interfaces.”
  2. Irvine, Martin. 2016. “André Malraux, La Musée Imaginaire (The Museum Idea) and Interfaces to Art.”
  3. National Gallery of Art, background on Samuel Morse’s painting, The Gallery of the Louvre.
  4. Irvine, Martin. 2016. “The Museum and Artworks as Interfaces: Metamedia Interfaces from Velázquez to the Google Art Project“

  5. Proctor, Nancy. 2011. “The Google Art Project.” Curator: The Museum Journal, March 2.

     

 

Dream Machine – Alexander MacGregor

“Computer science is no more about computers than astronomy is about telescopes.” – Edsger W. Dijkstra

Coming into this course, the above quotation would have been incomprehensible to me. How could computer science not be about computers? These intricate, abstruse, blackboxed machines need a discipline as rigorous as a science to understand and interrogate. While I haven’t been completely divested of the latter sentiment, I am now far more confident in the universality of computational concepts than I was before. I believe it was receiving a grounding in semiotic concepts that set the stage for this transition in thinking. The ideas found in Peirce’s writings were absolutely instrumental in understanding the basic processes that are taking place whenever we interact with a computer. As was the information we gained about Morse and the history of binary. It now seems to me as though computers are just devices that we, as humans running OS Alpha, use to augment functions and processes we’ve been performing since time immemorial.

This is not to understate the importance of the mechanization of computers, particularly the “microcomputer revolution”. As we have tracked the history of computing, from the human computers of the Napoleonic Wars that inspired Babbage’s Difference Engine to the iPad, we see that the technological mediation of computational devices has been prismatic. The ability to cognitively offload tasks to machines capable to executing them at a far faster and more powerful rate has been crucial to constructing our present. The history of interface design has also been imperative in making use of these computational machines as widespread as it has become. From Vannever Bush’s Memex to Ivan Sutherland’s Sketchpad to Douglas Engelbart’s WIMP innovations, we see how design concepts like affordance and extended cognition have played a vital role in shaping our computational landscape.

One word I keep coming back to is abstract. I have found the process of de-blackboxing the “computer” to necessarily be an act of abstraction. Going from thinking of computers as simply mechanical devices to a “universally applicable attitude and skillset”, as Jeanette Wing puts it, has been enlightening for me, and has helped to expand the realm of what I considered to be computationally possible. I now consider computational thinking as more of a philosophy than a strict set of concrete rules governing inputs and outputs to machines. This pliability is important when thinking of possible phenomena arising out of the dissipation of computers into our surroundings. As society moves from conceiving of computers as metal encased box of wires and chips to potentially every item we see around us, computational thinking will need to be applied in order to tackle to the problems of tomorrow.

One last point I wanted to make was related to this quote from the Dennings reading:

“Many of us desire to be accepted as peers at the ‘table of science’ and the ‘table of engineering’. Our current answers to this question are apparently not sufficiently compelling for us to be accepted at those tables.”

It seems to me as though a natural result of the spreading of computational thinking would be the dissolution of this ossified hierarchy that seems to be implicit in these “communities of practice”. We learned how important these distinct communities were during the early years of computing, and how their fingerprints can still be seen on our modern devices, but I believe that once we gain a deeper grasp on the ever-present computational processes surrounding us, “computers” will no longer be seen as being native to the disciplines of science or engineering, but will rather be as intrinsic to all fields as reading and writing are.

In conclusion, as I try to synthesize all the concepts we’ve learned so far in this class, I come to the question of how will the computational interface and system designs of tomorrow integrate historic design concepts, and interpret semiotic concepts such as extended cognition, affordance, and distributed cognition, to create a more intuitive computational relationship between the user and the machine in order to meet our new computational desires and needs? I’m particularly excited to see these developments as they apply to Artificial Intelligence via concepts like parallel computing and artificial neural networks. As computing continue to become a ubiquitous presence in our lives, breaking down that prevailing “man-machine” illusion may produce even more radical consequences to the way we not only perceive computers, but the very world around us.

References

  1. Irvine, Martin. “Introduction: Toward a Synthesis of Our Studies on Semiotics, Artefacts, and Computing.“
  2. Simon, Herbert A. ” The Sciences of the Artificial“. Cambridge, MA: MIT Press, 1996.
  3. Denning, Peter. “What Is Computation?” Originally published in Ubiquity (ACM), August 26, 2010, and republished as “Opening Statement: What Is Computation?” The Computer Journal 55, no. 7 (July 1, 2012): 805-10.
  4. Murray, Janet. “Inventing the Medium: Principles of Interaction Design as a Cultural Practice.” Cambridge, MA: MIT Press, 2012.

The Cognitive Leap of Faith (Ojas + Alex)

When analysing the process through which computing became the dominant metamedia platform of the symbolic and cognitive technologies we utilize everyday, we were able to identify two main concepts; permanent extendibility and extended cognition.

Looking at permanent extendibility, what stands out to us is the transience of digital media. Through computing, we are able to mimic the ephemerality of much of our cognitive process. For example, just as the concept of “dog” or “book” exists in a cognitively incorporeal space, the files we save, edit, delete, or otherwise alter exist on our computers are impermanent. This extendibility is an incredibly powerful tool allowing for the multitudinous levels of abstraction necessary for modern cognitive and computational tasks. The Ship of Theseus paradox is an interesting thought experiment that we found pertinent to this particular concept. Namely, what or where is the cultural artifact if it is permanently extendible? It’s one thing in the triadic model of semiosis to be dealing with physical, unchanging artifacts, but where do we place artifacts that are by nature always in flux?

It seems to us as though extended cognition is the guiding principle by which advances in computational and software interfaces evolve. Last week, Ojas had a blog post on the “phenomenological trickery” of the mouse, which was a great example of closing the cognitive-technological gap. It’s very easy to take these interfaces for granted, but when compared to older forms of computers, such as Babbage’s Analytical Engine, we really see how much more intuitive modern interfaces have become. These advances have led to a lowering of the threshold for computational learning and interaction, which was crucial for getting to the levels of widespread adoption we’re seeing now.

As far as technologies go, we believe that the innovations in computer networking born out of XEROX PARC and ARPA have been hugely influential in our modern computational landscape. This is especially clear when we consider that one of the major features of computer networking that made it commercially successful was e-mail (Campbell-Kelly et al 284). Combining the affordances of word processing with networked computers across space allowed for conversations to take place over this network. Of course the lineage from Vannaver Bush’s Memex to Ivan Sutherland’s Sketchpad to Alan Kay’s Dynabook is well noted within the readings, but perhaps most importantly, Doug Engelbart’s GUI innovations were critical in bridging the cognitive-technological divide. Creating graphical interfaces that are intuitive and factor in the concept of affordance was an enormously important step in getting computers to become effective symbolic representation tools.

When reading about Alan Kay and his Dynabook, we were amazed at how many of the concepts and innovations he developed that have actually been implemented in our modern computational technology; most obviously the iPad. But there was also a fundamental distinction between Kay’s work and the technologies we have now. Kay’s designed the Dynabook to be used for “symmetric authoring and consuming”. The concept of sharing and the spirit of openness seem central to Kay’s design process. We surmised that this came from his desire for this technology to be used as a learning tool, or as Kay says “qualitatively extend the notions of reading, writing, sharing, publishing, etc. of ideas”. Our modern computers are often designed to be inherently siloed devices, which most likely comes from the particular incentives of commercialization. So we suppose one could say the differences a culture clash took place between the Kay/Xerox PARC/ARPA communities of innovation and the Microsoft/Apple corporations that were took those inventions and made them available to the masses.

An interface feature we propose to realize Alan Kay’s vision for the dynabook is the integration of image recognition in the cameras on our phones. Much like we can use natural language processing to interact with computers with textual search engines and virtual assistants, an image recognition feature in cameras allows us to interactively engage with our environments. For example, if someone were to use this feature on a piece in an art museum, they could scan the piece, and this would automatically hyperlink to relevant articles in art history, criticism, other museums the piece has been in, etc. This could lend itself to an open-source, networked, encyclopedic database of image-entries which students actively engage with by consuming, editing, and producing knowledge on the subject.

References

  1. Manovich, Lev. Software Takes Command, pp. 55-239; and Conclusion.
  2. Campbell-Kelly, Martin and William Aspray. Computer: A History Of The Information Machine. 3rd ed. Boulder, CO: Westview Press, 2014.
  3. Greelish, David. An Interview with Computing Pioneer Alan Kay
    . Time Magazine, 2013.
  4. Alan Kay’s original paper on the Dynabook concept: “A Personal Computer for Children of all Ages.” Palo Alto, Xerox PARC, 1972).

“The two most significant events in the 20th century: Allies win the war…and this.”

I found this week’s readings to be some of the most fascinating we’ve done so far.

One of the main takeaways I had from the Mahoney reading is that the history of computational and software development was not singular and fixed, but rather collective and malleable. As Mahoney himself says “The computer thus has little or no history of its own. Rather, it has histories derived from the histories of the groups of practitioners who saw in it, or in some yet to be envisioned form of it, the potential to realise their agendas and aspirations.” (Mahoney 119). The various “communities of computing” that were present in the formative years of machine based computing, such as the science and engineering community, the data processing community, the management science community, and the industrial engineering community all have their fingerprint on our modern computational systems. They each had unique expectation and needs from the computer, and as such, the product we have now is an amalgamation of these distinct cultures.

I was interested in how we got from the relatively esoteric and community-based uses of computers to the general-purpose PCs we have now. A key step in this process seems to be the universalization of computational affordance via the evolution of GUI design. Going from Vannevar Bush’s Memex to Douglas Engelbart’s work at the SRI labs, to the innovations that came out of Xerox PARC, we can see an active thread linking much of our contemporary conception of computing to designs decades in the making. But while Xerox PARC may have been the incubation chamber for much of our modern GUI design, such as the WIMP UI that we all know and love, companies like Apple and Microsoft played a crucial role in taking these breakthroughs and disseminating them to the public at large.

As for the development of interfaces and interactions, I was stunned by that side-by-side comparison of writer-scribe Jean Mielot in his 15th century library and Ivan Sutherland demonstrating the Sketchpad in 1963 (Irvine 7-8). Yet again, we see that the gods didn’t gift us these technologies. Rather, they are extrapolations and advances in the existing technological lineage. The mediatory role of interfaces is something I find highly fascinating. In fact, to bring it back to semiotics, one can think of interfaces as languages that allow us to communicate with the underlying technology. A good UI designer always keeps this concept in mind. Realization of good interface design also requires knowledge of contextual use. What is the ultimate purpose of the underlying technology? Is it to read text, like Ramelli’s 14th century “Book Wheel” and Bush’s Memex? That context has a particular affordance history that will inform the design of that technology’s interface.

Cubs are one out away! 

Thinking of a high-profile interface development in recent times, my mind goes to Google Glass.

Ubiquitous computing was Google’s goal in producing the headset, and this context informed their interface design. The product was a commercial failure, but one thing the readings have taught me is that timing is a crucial component of public adoption. Many of the historical antecedents to the current interface designs didn’t catch on in their time. But they played a crucial role in laying down the framework that successfully adopted interfaces have utilized. So even though right now everyone isn’t walking around with Google’s sleek, metallic eyeglasses, one day we may be.

If ubiquitous computing is to be the dominant computing context, then I believe the future of interface design will be shaped around seamless, recognition-based non-events. Augmented reality devices are also a distinct possibility, and will require a whole new set of design principles as we learn to rethink not only traditional computational affordance, but the affordance of everyday objects we map computational abilities onto. Diversification of design is another concept that should play a large role, especially as computers become less physically constrained, and more multifarious (think IoT). This shifting paradigm will require a new interface and interaction framework, but perhaps we will look to the past to help define our future.

CUBS WIN! I’M GOING TO BED! 

References

  1. Mahoney, Michael S. “The Histories of Computing(s).” Interdisciplinary Science Reviews 30, no. 2 (June 2005): 119–35.
  2. Engelbart, Dave. 1962. “Augmenting Human Intellect: A Conceptual Framework.” New Media Reader. Wardrip-Fruin, Noah, Nick Montfort, ed.. 93–108. Cambridge, MA: The MIT Press, 2003.
  3. Vannevar Bush, “As We May Think,” Atlantic, July, 1945.
  4. Sutherland, Ivan. 1963. “Sketchpad: A Man-Machine Graphical Communication System.” New Media Reader. Wardrip-Fruin, Noah, Nick Montfort, ed.. 109–125. Cambridge, MA: The MIT Press, 2003.
  5. Irvine, Martin. “Introduction to Affordances and Interfaces: Semiotic Foundations”

Coding: Ain’t Just For Nerds Anymore! – Alex MacGregor

Having little to no experience with coding before I came to CCT, I found this week’s exercise to be very informative. Being given the opportunity to test the parameters and abilities of coding was helpful in seeing how it is truly a language, replete with syntactic and grammatical rules. Misplacing a certain element of code or confusing one variable for another resulted in errors, much the same way our brain has trouble processing nonsensical linguistic structures. One cannot escape the computational history of coding, either. Concepts we’ve discussed in class that have computational uses, like “Boolean values” and binary, also have fundamental roles in the architecture of coding. By “de-blackboxing” the coding process, you’re able to see how coding is not some arduous task requiring specialized knowledge, but rather just an extension of our own language system and ways of thought. Yet again, we see that there is nothing alien about computing; it came from, and belongs to, us.

This was also evident from the readings this week, particularly Campbell-Kelly’s “Origin of Computing” and Dasgupta’s “It Began with Babbage: The Genesis of Computer Science”. Plotting the computational lineage from the literal human computers of the 19th century to the exponentially smaller and faster machines we’re dealing with in this era was really cool. Seeing the various disciplines and influences that shaped the history and future of computing made me wonder what the computational landscape would look like if certain events, like WW2 or Babbage abandoning the Difference Engine, had never happened.

After reading the article “Computational Thinking” by Jeanette Wing, I started to think of coding as a mechanism through which computational thinking can be wired and framed on our minds. The absolutely imperative role that computers play in not only our daily lives, but also the long-scale trajectory of our species, has been more or less accepted by the public at large, yet coding is still seen as a relatively esoteric field. The bottlenecking of the functional knowledge required to operate these incredibly important cognitive technologies seems to me an undesirable situation. So I share Wing’s insistence on placing computational thinking on the same level as the traditional Three R’s of education. That model is from the turn of the 19th Century, and we’ve quite clearly gone through multiple socio-technological revolutions since then, so the de-blackboxing of these systems and technologies should be an educational imperative. I believe coding should be to the 21st century what literacy was to the 20th.

References

  1. Jeannette Wing, “Computational Thinking.” Communications of the ACM 49, no. 3 (March 2006): 33–35
  2. Martin Campbell-Kelly, “Origin of Computing.” Scientific American 301, no. 3 (September 2009): 62–69.
  3. Subrata Dasgupta, It Began with Babbage: The Genesis of Computer Science. Oxford, UK: Oxford University Press, 2014.

Mad Max 3: Beyond the Infosphere (Two Messages Enter, One Message Leaves!) – Alexander

When first engaging with this course’s subject material, I often found myself getting lost in the thickets, so to speak. Perhaps it’s due to the unfinished/unpolished nature of Peirce’s writing, but I had a hard time wrapping my mind around some of the connection we were trying to establish between semiotic theory and its practical deployment. But this week’s reading made a lot of things click into place for me.

One of the concepts I’ve found most interesting is the dialogic and communal nature of the meaning-making process. As Floridi says, “In many respects, we are not standalone entities, but rather interconnected informational organisms or inforgs, sharing with biological agents and engineered artefacts a global environment ultimately made of information, the infosphere.” Interconnected is a term we’ve encountered many times throughout this course, but this was the first I’d heard of the infosphere. Analyzing now familiar semiotic concepts through Floridi’s epochal lens was fascinating. This computer science and ICT induced fourth revolution has had very significant ramifications for our self-conception as semiotic beings. I’m personally interested in the Internet of Things, so the idea of rendering inanimate objects animate struck a chord with me. What effect does this elevation of information and data, which is decidedly non-alive in the conventional sense, have on us as a society? If information and data shifts to the centre of our ideological framework, supplanting the outmoded anthropocentric (for lack of a better term) model, then anything that can be merged or interact with data and information deserves a seat at the table. This would include traditionally “dead” objects, like cars, clothes, computers and even cities. These objects are now “speaking” to us. This relates back to semiotics because we’re dealing with an inflection point of meaning. Our preexisting conception of the subjects and objects of communication, how messages are transmitted, the “language” of transmission, and the very framework with which we undertake communicative acts are all under review in this new informational age.

Allowing my mind to run free, I started to think of what the next step of this information revolution would look like. Perhaps the next generation will adhere to a neo-animist philosophy based on RFID style implantation. A sort of technological paganism. The pendulum swings back. We can already see this generational rift taking place. A smartphone means different things to a digitally native child than to their digital immigrant parent. Now imagine growing up communicating with Siri, or Alexa, or Cortana. As a tool, or a relation to the world, your way of viewing traditionally inanimate objects is going to be radically different. I was (pleasantly) surprised to see Floridi greet me at the precipice of this cliff when he said “This animation of the world will then, paradoxically, make our outlook closer to that of pre-technological cultures, which interpreted all aspects of nature as inhabited by teleological forces.”

To tie this back to our prompt, cultural context and linguistic literacy are required to effectively derive meanings from messages. I found the reading’s example of the Rosetta Stone to be particularly instructive. Even before the discovery of the Rosetta Stone, Egyptian hieroglyphics were always information. The Rosetta Stone was simply an interface through which their meanings could be rendered accessible to the hieroglyphically illiterate symbolic agent. This is also a useful illustration of meaning not being contained within any specific individual’s mind, as the last person to know Egyptian hieroglyphics was long dead by the time the Rosetta Stone was discovered. Meaning was created within the symbolic system once the requisite elements were integrated. The actual shapes and forms that made up hieroglyphics were more or less inconsequential. They didn’t change pre-to-post Rosetta Stone discovery, yet the ability to derive meaning from them did.  Additionally, situating the transmitted symbols in the correct meaning register was necessary, which the Rosetta Stone did by placing the hieroglyphs next to Ancient Greek and Demotic language. So if I were to text someone a fully functional code to which they did not have the decoder to, meaning could not be derived because they would lack the necessary contextual or linguistic literacy.

Lastly, something that caught my attention from the readings was the question of identity in a world of mass production. When Floridi talked of “the metaphysical drift caused by the information revolution”, I was reminded of this passage from “The Conscience of the Eye” by Richard Sennett, in which he discusses the symbolism of our post-modern architectural landscape and urban planning:

“The ancient Greek could use his or her eyes to see the complexities of life. The temples, markets, playing fields, meeting places, walls, public statuary, and paintings of the ancient city represented the culture’s values in religion, politics, and family life. It would be difficult to know where in particular to go in modern London or New York to experience, say, remorse. Or were modern architects asked to design spaces that better promote democracy, they would lay down their pens; there is no modern design equivalent to the ancient assembly. Nor is it easy to conceive of places that teach the moral dimensions of sexual desire, as the Greeks learned in their gymnasiums—modern places, that is, filled with other people, a crowd of other people, rather than the near silence of the bedroom or the solitude of the psychiatrist’s coach. As materials for culture, the stones of the modern city seem badly laid by planners and architects, in that the shopping mall, the parking lot, the apartment house elevator do not suggest in their form the complexities of how people might live. What once were the experiences of places appear now as floating mental operations.” – Richard Sennett

Apart from the term “floating mental operations” reminding me of the location of meaning, the concept of identity is one I believe may hold some semiotic importance. Philosophers from Nietzsche to Georg Simmel to Louis C.K. have all discussed about the effects of modern (urban) living on an individual’s relation to not only the world around them, but to themselves. It seems to me as if a necessary part of semiotic communication and meaning-making is identity, as a person sending a message must have some conception of themselves and the identity of the intended recipient. So I believe it is worthwhile to examine how these large scale revolutionary changes to the very foundation of our society are affecting our self-perception, and how that might alter the ways we utilize language and conceive of its meaning.

References

  1. Floridi, Luciano. Information: A Very Short Introduction. New York: Oxford University Press, 2010.
  2. Irvine, Martin. Introducing Information and Communication Theory: The Context of Electrical Signals Engineering and Digital Encoding. Google Doc.
  3. Sennett, Richard. The Conscience of the Eye: The Design and Social Life of Cities. New York: Knopf, 1990. Print.

Parallel Computing and Surrealist Syntax – Alex MacGregor

I had first encountered the idea of “parallel architecture” in a computer science environment. As the amount of data we’re collecting continues to grow exponentially, the traditional method of linear computing has become ineffective. We simply could not build processors big enough to handle the sheer size of data we’re now dealing with. Parallel computing, also sometimes referred to as “distributed computing” has provided a solution to this problem by distributing the data load across multiple processors and thereby divvying up the workload. Linguistics, with its ability to produce an infinite number of possible sentences, can be seen as the ultimate workload, so using parallel architecture in a cognitive setting makes sense.

Visualization of Parallel Computing from WikipediaTripartite Parallel Architecture from Jackendoff’s Foundations of Language

I’ve previously touched upon it in my older posts, but the convergence of semiotic/linguistic concepts and computational concepts continues to excite me.

As for analyzing symbolic genres through the lens of a sign system, surrealist art provides an excellent avenue to do so. If we take a look at Salvador Dalí’s The Persistence of Memory, we can see iconic resemblances, albeit distorted, throughout.

Soft Construction with Boiled Beans (Premonition of Civil War) by Salvador Dalí

The head smiling in the clouds, the hand grasping a breast, the leg, the man in the bottom of the picture, and the town in the background are all recognizably connected to their respective signified, and as such are operating within the iconic mode. But when looking at this painting in a more abstract, meta sense, we can see how it is also operating in the symbolic mode. The scene is obviously one that does not, and cannot, exist in reality, so in order to analyze and derive meaning, we must utilize learned references. For example, Dali meant this painting to be a symbolic critique of the Spanish Civil War. The beans in the foreground were meant to augment the mass of flesh in the painting and represent war as a devourer of life. The skyline was meant to evoke that of Catalonia, which was a major revolutionary hub during the war. The signifier (painting) does not resemble that signified (Spanish Civil War), except when we utilize these agreed upon heuristics and abstractions. As such, the painting can be viewed as icons within a symbol.

What makes Surrealism interesting is by distorting the image on the iconic level and “playing” with the visual syntax, you can derive an interesting semantic result. Just like with linguistic grammar, we notice an awkwardness when the various elements, be they word classes or visual features, are out of their order. We call the former bad grammar, but the latter has been made into an art form.

References

Jackendoff, Ray. Foundations of Language: Brain, Meaning, Grammar. Evolution. New York: Oxford University Press, 2003.

Chandler, Daniel. Semiotics: The Basics. 2nd ed. New York, NY: Routledge, 2007.

 

Linguistics, Chemistry and Computing – Alex MacGregor

I wanted to see how my conception of “language” changed as a result of the readings and video this week, so I first wrote down some keywords I associated with the term. I came up with: culture, history, speech, understanding, communication, and transmission.

After doing the readings, I must admit that I was guilty of the sociological problem Jackendoff describes of laymen being especially susceptible to the Dunning-Kruger effect when dealing with the field of linguistics. I think this most likely comes from the fact that, as Jackendoff and Pinker both mention, we acquire and comprehend these incredibly elaborate linguistic skills from such a young age, so we’re never really cognizant of the process. Even more so if, as Chomsky proposes, we’re hard-wired for it. I was really blown away by the sheer amount of behind-the-scenes cognitive work (or as Jackendoff calls it, “f-mental” or “f-language”) that must be taking place in the child learning her or his way around language. Pinker’s segment on structure dependent rules was a great illustration of this. I personally had no idea these rules even existed, yet I immediately knew something was off when they were altered. It was intuitively clear to me, and reminded me of this post I saw a few weeks ago:

Another thing I learned from the Jackendoff reading is just how scientific and complex this area of study is. I was struck by the similarity of the illustrations of linguistic rules and structures with the illustrations of chain reactions you’d find in a chemistry textbook. Which, I suppose, is what sentences are: chain reactions that prompt understanding and meaning. Sequencing and hierarchy of the elements within both linguistic and chemical chains are imperative to their outcome. The wrong linguistic sequencing will lead to an illogical or awkward sentence, whereas the wrong chemical sequencing will lead to totally different formula or element (I think…I’m not a chemist).

I’m interested in the efforts to mirror or mimic these linguistic structures in computers and AI, so when Jackendoff talked of “structures built of discrete combinatorial units”, I was reminded of the discussion we had a couple of classes ago about Morse and the foundation of modern computing. Stripping computing down to its fundamentals in binary was illustrative of how crucial chaining and coupling are to computing. It seems as though there may be some kinship between this conceptualization of computing and linguistics.

So after going through the assigned material, I would add the words science, structure, innate, and computing to my previous list.

A few questions I had:

  • One of the critiques of Chomsky was that universal grammar may not be specific to language, but what else could it be applied to? Pinker mentions vision, control of motion, and memory. Is this implying that there could be hard-wired ways in which we physically move, see and remember?
  • Is the solution to solving the issue of pragmatics in AI linguistics to program for existing social interactions and contexts? Is there some “learning” ability that AI should be able to exercise if they come across unaccounted for situations and contexts?

References

  1. Ray Jackendoff, Foundations of Language: Brain, Meaning, Grammar, Evolution. New York, NY: Oxford University Press, USA, 2003.
  2. Steven Pinker, Linguistics as a Window to Understanding the Brain. 2012.

Our Cognitive Past and Our Cognitive Future – Alex MacGregor

In last week’s journal response I posed some questions regarding the historical timeframe of our symbolic cognitive evolution, as well as the how non-human animals fit into this discussion, so I was glad to see both of these issues addressed in this week’s readings.

In “The Morning of the Modern Mind: Symbolic Culture“, Wong uses Neanderthals as a reference point. She brings up “systematic ochre processing” and “manufacturing of body ornaments” (Wong, 95) as evidence of their symbolic capabilities, which I found particularly interesting because we know that biological ornamentation is a feature utilized by many different animals, so is the manufacturing the key distinction that takes us from non-symbolic to symbolic? I was also interested in her point about how both humans and Neanderthals came to exhibit the ability to think symbolically. She raises the possibility of independent evolution, as well as the possibility of a “primeval common ancestor” (Wong, 95). Should it be the latter, I wonder how far back this common ancestor is located?

One thing that really struck me from “The Symbolic Species: The Co-evolution of Language and the Brain” was how interdisciplinary this field of study is. Deacon mentions philosophers, neurologists, biological scientists, psychologists, archaeologists and linguists as all having a role to play in exploring and understanding symbolic cognition. To me, that signifies (heh) just how fundamental this topic is to our existence. I was also interested in the point Deacon made about the human brain being overbuilt for learning symbolic associations (Deacon, 413). The amplification of emergent consequences in other mental domains that results from this overbuilding is something he didn’t go into depth about, so I was wondering about examples of these supramodal adaptations.

In the Donald reading, yet again we come across the “symbolic cognition as network” analogy, which i’m really loving. The way he framed the uniqueness of the human mind as being in the ability to imagine or think about things outside our immediate environment also made a lot of sense to me, especially when analyzing these issues through a semiotic lens, as much of it is based on non-immediate mental processing. Donald also talks about the difference between literary cultures and oral cultures (Donald 220), and while he discusses how those differences manifest in the realm of overt culture, I wonder about any neurological differences between cultures that may have sprung up as a result.

So I suppose my question is where is all of this leading to? The readings this week did the anthropological work of tracing our cognitive history, but what will our cognitive future look like? The “computer as mind, mind as computer” concept seems to be quite important, so perhaps the field of computer science will become an increasingly critical slice of this interdisciplinary pie. Is it possible we will reach a kind of cognitive inflection point once the technology advances enough? In the “Mind and Matter: Cognitive Archaeology and External Symbolic Storage ” reading, Renfrew brings up the four transitions of cognitive phases (Renfrew, 4), so is it possible we’re due for a fifth transition?

References

Wong, Kate. “The Morning of the Modern Mind: Symbolic Culture.” From Scientific American. 292, no. 6: 86-95. 2005.

Deacon, Terrence W. The Symbolic Species: The Co-evolution of Language and the Brain. New York, NY: W. W. Norton & Company, 1998.

Donald, Merlin. “Evolutionary Origins of the Social Brain,” from Social Brain Matters: Stances on the Neurobiology of Social Cognition, ed. Oscar Vilarroya, et al. Amsterdam: Rodophi, 2007.

Renfrew, Colin. “Mind and Matter: Cognitive Archaeology and External Symbolic Storage.” In Cognition and Material Culture: The Archaeology of Symbolic Storage, edited by Colin Renfrew, 1-6. Cambridge, UK: McDonald Institute for Archaeological Research, 1999.