Author Archives: Deborah Oliveros Useche

Gender Bias & Artificial Intelligence: Questions and Challenges

Gender Bias & Artificial Intelligence: Questions and Challenges

By Deborah Oliveros

  • Abstract
  • Introduction
  • A (Unnecessarily) Gendered Representation of AI
  • A ‘Knowing’ Subject
  • Replicating Human Biases
  • Challenges of Imbedded Biases in AI
  • Possible Approaches to Addressing Bias in AI
  • Conclusion
  • Bibliography

Abstract

This essay aims to analyze the different ways in which bias is imbedded in AI, machine learning, and deep learning, addressing the challenges from a design perspective to understand how these systems succeed in some respects and how they fail in others. Specifically, this paper focuses on gender bias and the significance of a gendered representation of technology in mass media and marketing as an obstacle to not only understand the process of human interaction with these technologies but to replicate historical biases present in society. Lastly, this analysis strives to present possible approaches necessary to address gender bias in AI.

  • Introduction

Over the last couple of years, prominent figures and big companies in Silicon Valley have participated in public debate around the benefits and concerns about artificial intelligence and machine learning and its possible consequences for humanity. Some embrace technology advancement openly, advocating for a non-limiting environment arguing that, otherwise, it would prevent from progress and innovation in the field. Others offer warnings similar to those of a sci-fi dystopian film; they argue that artificial intelligence could be an existential threat to humanity if -or more likely when- machines become smarter than humans. Although the potentiality of a world controlled by machines as the ones presented in The Matrix (Wachowski sisters, 1999), Blade Runner (Scott, 1982), and 2001: A Space Odyssey (Kubrick, 1968) is unsettling and borderline terrifying, there are more urgent and realistic questions to address around the issues of AI and its impact on society.

2001: A Space Odyssey (Stanley Kubrick, 1968)

Machines learn based on sets of data that humans ‘feed’ them. If machines are learning how to imitate human cognitive processes, what kind of human behaviors, social constructions, and biases are these machines picking up and replicating based on the data provided?

There is a long history of cases in which technology has been designed with unnecessarily deterministic biases on it: the famous low bridges in New York preventing minorities from using public transportation to go to the beach; the long-time perpetuated ‘flesh’ labelling of crayon colors, also on band aids, paint, and ballerina shoes; the famous case of Kodak’s Shirley Cards used by photo labs to calibrate skin tones, shadows and light during the printing process of color film, making it impossible to print darker skin facial expressions and details, among others.

Kodak Shirley card, 1974

We couldn’t expect different than the replication of these patterns when it comes to artificial intelligence and machine learning. In this case, both the design of the technology and the set of data that we are feeding into the machines are primary factors of this issue. There is a systemic, systematic, racist, sexist, gendered, class-oriented -and other axes of discrimination- bias embedded in most data collected by humans, and those patterns and principles are being picked up and replicated by the machines by design. Therefore, instead of erasing divisions through objectivity in decision making, this process is exacerbating inequality in the workplace, the legal and judicial systems, and other spaces of public life in which minorities interact, making it even more difficult to escape from it.

The data fed to the machines is diverse: images, text, audio, etc. The decision of what data is fed to the machine and how to categorize it is entirely human. Based on this, the system will build a model of the world accepted as a unique and stable reality. That is, only what is represented by the data have the meaning attached to it, without room for other ways of ‘being’ in the world. For example, facial recognition trained on data of overwhelmingly white men as successful potential candidates for a job position, will struggle to pick up others that don’t fit into those categories.

Police departments have also used data-driven systems to assess the probability of crime occurring in different areas of a city and, as it was discussed before, this data is polluted with systemic racism and class discrimination of minorities. Therefore, the immediate consequence is over policing of low-income areas and under policing of wealthy neighborhoods. This creates and perpetuates a biased cycle but, more importantly, it creates a false illusion of objectivity and shifting of responsibility from the human to the machine. Crawford says, “predictive programs are only as good as the data they are trained on, and that data has a complex history” (Crawford 2016, June 26).

  • A (Unnecessarily) Gendered Representation of Technology

There is a challenge to analyze how we perceive something that is invisible to us, not only physically but also cognitively. Two aspects need to be taken into account to get to the root of why the general public does not fully understand how these systems work: the lack of transparency from companies to reveal how these systems make data-driven decisions due to intellectual property and market competition; and the gendered marketing of these technologies to the users in combination with a gendered representation in pop culture media that is not only inaccurate but misleading. Let’s start by addressing the latter.

For decades, visual mediated spaces of representation such as movies and tv in the genre of sci-fi, have delved into topics of technology and sentient machines. Irit Sternberg states that these representations tend to ‘gender’ artificial intelligence as female and rebellious: “It goes back to the mother of all Sci-Fi, “Metropolis” (Lang, 1927), which heavily influenced the futuristic aesthetics and concepts of innovative films that came decades later. In two relatively new films, “Her” (Jonze, 2013) and “Ex-Machina” (Garland, 2014), as well as in the TV-series “Westworld” (2016), feminism and AI are intertwined.” (Sternberg 2018, October 8).

Alicia Vikander

Ex Machina (Garland, 2014)

These depictions present a gender power struggle angle between AI and humans, which is at times problematic and at others empowering: “In all three cases, the seductive power of a female body (or voice, which still is an embodiment to a certain extent) plays a pivotal role and leads to either death or heartbreak” (Ibid). This personification of AI invites the viewer to associate the technology with a power struggle that already exists in our own historical context, which in turn makes it difficult for the general public to go beyond the superficial layers of explanations of a regular tech news article that fails to address how these technologies work from a conceptual level. The over-generalizing paranoid headline seems to be catchier than an informative analysis in those cases. On the other hand, the representation of the level of agency in a female-gendered AI offers the imagined possibility that, through technology, systematic patriarchal oppression can be challenged and surpassed by the oppressed.

In spite of these manifestations of gender roles combined with AI, the reality is far from empowering: gender discrimination in algorithms is present in many spaces of social life. Even more problematic are the non-fictional representations of technology, in particular AI, as gendered.

AIs are marketed with feminine identities, names and voices. Examples such as Alexa, Siri, Cortana demonstrate this: even though they enable male identities, the fact that the predetermined setting is female speaks loudly. Another example is the female humanoid robot Sophia, developed by Hanson Robotics in Hong Kong, built as a representation of a white slender woman with no hair (enhancing her humanoid appearance) and, inexplicably, with make up on her lips, eyes and eyebrows. Sophia is the first robot to receive citizenship of any country (Saudi Arabia), it was also named United Nations Development Programme’s first ever Innovation Champion, making it the first non-human to be given any United Nations title.

Sophia The Robot.

These facts are mindboggling. As Sternberg asks, “why is it that a feminine humanoid is accepted as a citizen in a country that would not let women get out of the house without a guardian and a hijab?” (Sternberg 2018, October 8). What reaction do engineers and builders assume the female presence and identification generates during the human-machine interaction?

Sternberg says that, fictional and real decisions of choosing feminine characters are replicas of gender relations and social constructs that already exist in our society: “does giving a personal assistant feminine identity provide the user (male or female) with a sense of control and personal satisfaction, originating in the capability to boss her around?” (Ibid). As a follow up question, is that what we want the machines to learn and replicate?

  • A ‘Knowing’ Subject

Artificial intelligence (and machine learning and deep learning as subcategories) is built and designed to acquire and process human knowledge and improve its decisions on categorization over time.

Gary Marcus says, “Deep learning systems are most often used as classification system in the sense that the mission of a typical network is to decide which of a set of categories (defined by the output units on the neural network) a given input belongs to. With enough imagination, the power of classification is immense; outputs can represent words, places on a Go board, or virtually anything else. In a world with infinite data, and infinite computational resources, there might be little need for any other technique” (p. 4).

However, the data in our world is never infinite and does not necessarily have a definite and unchanging meaning or interpretation, which limits the scope of AI and machine learning and its accuracy on representing the reality of said world, “Instead, systems that rely on deep learning frequently have to generalize beyond the specific data that they have seen, whether to a new pronunciation of a word or to an image that differs from one that the system has seen before, and where data are less than infinite, the ability of formal proofs to guarantee high-quality performance is more limited” (Ibid).

As stated before, these systems will know what we teach it, and the nature of that knowledge and the power dynamics surrounding it are inherently problematic. Early feminist theorists and social critics raised questions about how the knowledge will inform the identity and ‘world view’ of the ‘knowing subject’, offering contrasting takes on gender, class and racial determinism while also presenting the possibility of “un-situated gender-neutral knowledge (“a view from nowhere”) or lack thereof” (Sternberg 2018, October 8).

Critics also pointed out how ambitious projects designed around mastering expertise and knowledge about a topic might be tainted in said ‘expertise’, taking into consideration the origin of the ‘expert’ knowledge being fed to the machines: “the role of the all-male-all-white-centuries-old-academia in defining what knowledge is valuable for a machine to master and what is expertise altogether” (Ibid).

All of these characteristics have to be put in conversation with the fact that we are at the very early stages of AI. However, even at its infancy, AI and machine learning are already impacting the way we function as a society, not only in the technological aspect but social, health, military and employment as well.

  • Replicating Human Biases

A group of researchers from Princeton University and University of Bath conducted a study in which they tested how ordinary human language applied to machine learning results in human-like semantic biases. For this experiment, the authors replicated a set of historically known biased dichotomies of different terms, “using a […] purely statistical machine-learning model trained on a standard corpus of text from the Web.” (Caliskan, A, Bryson, JJ & Narayanan, A 2017, p. 2). “Our results (fig. 1) indicate that text corpora [the machine learning system that was tested] contain recoverable and accurate imprints of our historic biases, whether morally neutral as towards insects or flowers, problematic as towards race or gender, or even simply veridical, reflecting the status quo distribution of gender with respect to careers or first names” (Ibid).

They tested various dichotomous terms that are considered systematically stereotypical, demonstrating that the terms have an underlying historical -and contextual- understanding that the machine might not be processing it as such but is replicating: “these results lend support to the distributional hypothesis in linguistics, namely that the statistical contexts of words capture much of what we mean by meaning. Our work suggests that behavior can be driven by cultural history embedded in a term’s historic use. Such histories can evidently vary between languages” (p. 9).

(Fig. 1) Caliskan, A, Bryson, JJ & Narayanan, A (2017)

Machine learning technologies are already being used in many contexts in which these biases deeply impact minorities, and specifically women. They are used for resume screening resulting in cultural stereotypes and prejudiced outcomes around gender-professions perception. Another study from Carnegie Mellon University found that women were less likely than men to be shown ads on Google for highly paid jobs (Amit Datta, Michael Carl Tschantz, Anupam Datta, 2015).

Karen Hao, for the MIT Technology Review, looks at a study performed by Muhammad Ali and Piotr Sapiezynski at Northeastern University, analyzing the impact of variations on ads in regards to their target audience based on data, finding that those variations have an impact on the audience that is reached by each ad. Unsurprisingly, the decision of who is shown each ad is biased.

Hao says, “bias occurs during problem framing when the objective of a machine-learning model is misaligned with the need to avoid discrimination. Facebook’s advertising tool allows advertisers to select from three optimization objectives: the number of views an ad gets, the number of clicks and amount of engagement it receives, and the quantity of sales it generates. But those business goals have nothing to do with, say, maintaining equal access to housing. As a result, if the algorithm discovered that it could earn more engagement by showing more white users homes for purchase, it would end up discriminating against black users” (Hao 2019, February 4).

However, Hao also explains that the problem cannot be generalized to a biased data issue, “bias can creep in long before the data is collected as well as at many other stages of the deep-learning process” (Hao 2019, April 5). She specifically refers to three stages:

  1. Framing the problem: the goal that the designer plans to achieve and its context might not take into account fairness or discrimination
  2. Collecting the data: “either the data you collect is unrepresentative of reality, or it reflects existing prejudices” (Ibid)
  3. Preparing the data: “selecting which attributes you want the algorithm to consider” (Ibid)
  • Challenges of Imbedded Bias in AI

Gary Marcus offers a very detailed critique of the field of AI in “Deep Learning: A Critical Appraisal”. His article is presented as an intentional self-introspective snapshot of the current state of deep learning. It looks not only at how much has been accomplished but also how it has failed and what that presents for different approaches to deep learning in the future.

He says, “deep learning currently lacks a mechanism for learning abstractions through explicit, verbal definition, and works best when there are thousands, millions or even billions of training examples, as in DeepMind’s work on board games and Atari. As Brenden Lake and his colleagues have recently emphasized in a series of papers, humans are far more efficient in learning complex rules than deep learning systems are (Lake, Salakhutdinov, & Tenenbaum, 2015; Lake, Ullman, Tenenbaum, & Gershman, 2016).” (p. 7)

As it was mentioned many times before, deep learning struggles to offer outputs that accurately reflect complex human concepts that are difficult to represent as computational, as a set of yes and no answers. It can go from translation of languages, to more abstract concepts such as justice. Referring to my personal favorite example of open-ended natural language, Marcus says, “In a problem like that, deep learning becomes a square peg slammed into a round hole, a crude approximation when there must be a solution elsewhere.” (p. 15)

Another observation present in Marcus’ analysis refers to the approach of a real world taken as a ‘set in stone’ reality: “deep learning presumes a largely stable world, in ways that may be problematic: The logic of deep learning is such that it is likely to work best in highly stable worlds, like the board game Go, which has unvarying rules, and less well in systems such as politics and economics that are constantly changing.” (p. 13)

Not only the world and our knowledge of it is constantly changing, but our representation of that reality through data is most of the times inaccurate at best, skewed at worst. To what extent and what are the different ways in which we can see the impact of such flawed outputs? Sternberg presents two aspects to take into consideration:

  • What exists in the data might be a partial representation of reality:

Even that partial representation might not be entirely accurate. For example, the previously mentioned case of Kodak’s film being unable to efficiently capture non-white tones of skin is also present in facial recognition systems. Other more recent controversial cases are of systems mistaking pictures of Asians as ‘blinking’ and identifying black people as gorillas. The social cost of a mistake in any AI system being used by the police for decision-making is higher and more likely to present results that are less accurate with minorities since they were underrepresented and misrepresented in the data-set: “This also calls for transparency regarding representation within the data-set, especially when it is human data, and for the development of tests for accuracy across groups” (Sternberg 2018, October 8).

  • Even if the data does represent reality quite truthfully, our social reality is not a perfectly-balanced and desired state that calls for perpetuation:

Gender and racial biases present in the binary terminology is, after all, based on statistics present in the off-line world, well-documented in history. However, here Sternberg presents an optimistic perspective, “our social reality is not a perfectly-balanced and desired state that calls for perpetuation” (Ibid). This meaning that we are giving a deterministic characteristic to the data in this process when these are not the ideas, concepts and human values we should be preserving or basing our technology on.

In regards to that, Sternberg criticizes the absolute faith in the outcome of these systems regarding them as more objective than humans: “Sexism, gender inequality and lack of fairness arise from the implementation of such biases in automation tools that will replicate them as if they were laws of nature, thus preserving unequal gender-relations, and limiting one’s capability of stepping outside their pre-defined social limits” (Ibid).

Marcus’ premise agrees with Sternberg, focusing more on the problem of thinking about deep learning as the only tool available to understanding and digitalizing the world when, in reality, this tool might not fit every problem we want to fix: “the real problem lies in misunderstanding what deep learning is, and is not, good for. The technique excels at solving closed-end classification problems… And some problems cannot, given real world limitations, be thought of as classification problems at all.” (p. 15)

What is most thought-provoking about Marcus’ article is the proposal to see deep learning beyond this set box in which every human problem/though must be filtered through. To understand that we have to develop other hybrid ways in which we can analyze these problems beyond classifications instead of trying to make the “square leg fit into the round hole” (p. 15).

  • Possible Approaches to Addressing Bias in AI

Now we’ll address the remaining challenge presented a few sections above: it is difficult to analyze our perception and consequently actions regarding something that is invisible to us. The lack of transparency from companies to reveal how these systems make data-driven decisions due to intellectual property and market competition, is one of the main reasons why we don’t have access to this knowledge.

However, even if companies were coerced into sharing this information with the general public or authorities, the reality is that artificial technology is not only extremely young but evolving as we speak. Therefore, it can be said that the use of these technologies is both a process of creation and discovering at the same time. Based on what is public, engineers don’t fully know or understand how artificial technology learns and evolves with time. And whatever they know, they are not willing to share because of the conditions of the market in which they operate.

Crawford explains in regards of the case of women not seeing ads for high-paying jobs, “the complexity of how search engines show ads to internet users makes it hard to say why this happened — whether the advertisers preferred showing the ads to men, or the outcome was an unintended consequence of the algorithms involved. Regardless, algorithmic flaws aren’t easily discoverable: How would a woman know to apply for a job she never saw advertised? How might a black community learn that it was being overpoliced by software?” (Crawford 2016, June 26).

In terms of social actors that are invested and can influence how these technologies are managed, we can find that governments, NGOs and other entities have a stake into the outcomes of artificial intelligence and machine learning. Unfortunately, in the environment that was previously described of lack of information, they all pretty much operate ‘in the dark’ or, at least, at various levels of ‘darkness’.

A great example of how unprepared our politicians are to deal with this reality and attempt to hold tech companies accountable, happened a few months ago in the House Judiciary Committee. During the hearing of Google CEO Sundar Pichai, the members of the committee spent more time on passive-aggressively asking embarrassingly ignorant questions, with a clear partisan tone, than asking urgent, and appropriate questions around Google’s data policies and privacy practices. At one point, Pichai had to repeatedly explain that iPhone was a product of Apple, a different company than Google, and the collective groan of humanity could be heard across the globe.

It is clear that regulation and outsider audits are necessary to address the issue of gender bias in AI. However, it seems unlikely that something even remotely close to a proposal will make its way to congress anytime soon, let alone pass as a bill. Therefore, there is a need to find alternative ways in which actors can collaborate and share information towards the common goal of fixing and preventing the perpetuation of historical bias in AI. Evidently, the ones who have more possibilities of enacting a change are the engineers and companies themselves.

The authors of the language-based study from Princeton University and University of Bath offer: “we recommend addressing this through the explicit characterization of acceptable behavior. One such approach is seen in the nascent field of fairness in machine learning, which specifies and enforces mathematical formulations of non-discrimination in decision-making (19, 20). Another approach can be found in modular AI architectures, such as cognitive systems, in which implicit learning of statistical regularities can be compartmentalized and augmented with explicit instruction of rules of appropriate conduct (21, 22)” (Caliskan, A, Bryson, JJ & Narayanan, A 2017).

However, how can a solution such as this one be enforced and regularly supervised? We need organizations that address issues of technology and human rights to serve as intermediaries with the companies and the civil society, as they have done in the past since the creation of the internet.

If machines are going to replicate a human, what kind of human do we need them to be? This is a more present and already underway threat than a dystopian apocalypse in which humanity is decimated by their own creation, the Frankenstein old tale. As Kate Crawford wrote in the New York Times, the existential threat of a world overtaken by machines rebelling against humans might be frightening to the male white elite that dominates Silicon Valley, “but for those who already face marginalization or bias, the threats are here” (Crawford 2016, June 26).

 Conclusion

            Gender bias in AI, machine learning, and deep learning is the result of the replication by design of a deeply systemic, systematic, racist, sexist, gendered, class-oriented -and other axes of discrimination- bias embedded in most data collected by humans. Instead of erasing divisions through objectivity in decision making, this process is exacerbating inequality in the workplace, the legal and judicial systems, and other spaces of public life in which minorities interact. This happens in combination with an inaccurate and gendered representation of technology both in pop culture media as in marketing, making it more difficult for the general public to become aware and understand how these technologies work and their impact in our day-to-day lives. Bias can be introduced in the process by how the problem is framed, how the data is collected, and what meanings are attributed to that data (Hao 2019, April 5). Fixing gender bias in AI is a complex issue that requires the participation of all stakeholders: the companies, the designers, the marketing teams, tech reporters, intermediary collective organizations that advocate for civil society, and politicians. However, the major challenges boil down to the lack of transparency on how these systems make decisions and regarding them as the only filter through which every abstract human problem can be solved.

Bibliography

Ethical Implications of Advertising and Big Data

Ethical Implications of Advertising and Big Data

By Proma Huq and Deborah Oliveros

 

Talking points of the group presentation

  • Introduction
  • Video
  • AWS and overarching implications
  • How does this influence activity/who is privy to the data
  • Decision making due to surveillance
  • Interpretations and consequences
  • Implications of targeted marketing with case studies
  • Closing arguments

Presentation slides here

 

References:

Challenges of (not)Fitting into the Deep Learning Model

Gary Marcus offers a very detailed critique of the field of AI in “Deep Learning: A Critical Appraisal”. What caught my attention regarding this article was the fact that it was presented as an intentional self-introspective snapshot of the current state of deep learning. It looks not only at how much as been accomplished but also how it has failed and what that presents as different approaches to deep learning in the future.

He says, “deep learning currently lacks a mechanism for learning abstractions through explicit, verbal definition, and works best when there are thousands, millions or even billions of training examples, as in DeepMind’s work on board games and Atari. As Brenden Lake and his colleagues have recently emphasized in a series of papers, humans are far more efficient in learning complex rules than deep learning systems are (Lake, Salakhutdinov, & Tenenbaum, 2015; Lake, Ullman, Tenenbaum, & Gershman, 2016).” (p. 7)

As we have mentioned many times before, deep learning struggles to offer outputs that accurately reflect complex human concepts that are difficult to represent as computational, as a set of yes and no answers. It can go from translation of languages (I never get tired of using this very convenient example) to more abstract concepts such as justice.

Referring to my personal favorite example of open-ended natural language, Marcus says, “In a problem like that, deep learning becomes a square peg slammed into a round hole, a crude approximation when there must be a solution elsewhere.” (p. 15)

Another observation present in Marcus’ analysis refers to the approach of a real world taken as a ‘set in stone’ reality: “deep learning presumes a largely stable world, in ways that may be problematic: The logic of deep learning is such that it is likely to work best in highly stable worlds, like the board game Go, which has unvarying rules, and less well in systems such as politics and economics that are constantly changing.” (p. 13)

Not only the world and our knowledge of it is constantly changing, but our representation of that reality through data is most of the times inaccurate at best, skewed at worst. To what extent and what are the different ways in which we can see the impact of such flawed outputs? Sternberg presents two aspects to take into consideration:

  • What exists in the data might be a partial representation of reality:

Even that partial representation might not be entirely accurate. For example, the famous case of Kodak’s film being unable to efficiently capture non-white tones of skin is also present in facial recognition systems. Other controversial cases were those that mistook pictures of Asians as ‘blinking’ and identified black people as gorillas. The social cost of a mistake in any AI system being used by the police for decision-making is high and more likely to present results that are less accurate with minorities since they were underrepresented and misrepresented in the data-set: “This also calls for transparency regarding representation within the data-set, especially when it is human data, and for the development of tests for accuracy across groups” (Sternberg 2018, October 8).

  • Even if the data does represent reality quite truthfully, our social reality is not a perfectly-balanced and desired state that calls for perpetuation: 

As an example, gender and racial biases present in binary terminology is, after all, based on statistics present in the off-line world, well-documented in history. However, here Sternberg offers an optimistic idea, “our social reality is not a perfectly-balanced and desired state that calls for perpetuation”. This meaning that we are putting so much value on the data and deep learning in this process, giving it a deterministic characteristic, when in reality, these are not the ideas, concepts and human values we should be preserving or basing our technology on.

In regards to that, Sternberg criticizes the absolute faith in the outcome of these systems regarding them as more objective than humans: “Sexism, gender inequality and lack of fairness arise from the implementation of such biases in automation tools that will replicate them as if they were laws of nature, thus preserving unequal gender-relations, and limiting one’s capability of stepping outside their pre-defined social limits” (Sternberg 2018, October 8).

Marcus’ premise agrees with Sternberg, focusing more on the problem of thinking about deep learning as the only tool available to understanding and digitalizing the world when, in reality, this tool might not fit every problem we want to fix: “the real problem lies in misunderstanding what deep learning is, and is not, good for. The technique excels at solving closed-end classification problems… And some problems cannot, given real world limitations, be thought of as classification problems at all.” (p. 15)

These ideas are not entirely new to me. However, what I found the most thought-provoking about Marcus’ article is the proposal to see deep learning as this set box in which every human problem/though must be filtered through, but to understand that we have to develop other hybrid ways in which we can analyze these problems beyond classifications instead of trying to make the square leg fit into the round hole.

 

References:

 

Challenges of Interpreting Big Data

We live in a highly digitalized world that requires constant interaction with technology, creating massive amounts of data related to many aspects of society, from human behavior to the human body, among others. Some data is being collected on purpose with or without consent, and some is a result of digital interaction. The challenge is not necessarily about the size of the data sets but about how to process and interpret them to make sense of the world. To design and represent systems that can put process the data in spite of its big volume, the velocity to which it is acquired, the variety of data/information/meaning, and the veracity of the data in relation to how accurately does it represent the real world (Denning and Martell, 2015).

Johnson et al. challenge the accessibility of data on the internet, presenting various cases with different levels of access to the public that called my attention, “In principle, this means that huge amounts of data are available to anyone in the world over the internet. In practice much data are private and not available” (Johnson et al, 2018). I started to wonder about personal experiences in which my data was collected with or without my consent, and for what purposes.

I deleted my original Facebook years ago and just recently opened a new one as a way to stay connected with my graduate student peers and stay on top of events in my city. I rarely interact directly (like, share, comment, etc) but my less direct interaction (watching a video, expanding an article, etc.) is still collected and somehow interpreted with the interactions of my friends in order to make sense of who I am and what are my interests to provide attention-grabbing content on my page.

In the security page of your Facebook you can see the patters/labels/categories Facebook has placed you in and you can also see the interests associated with you that inform targeted adds into your page. When I checked mine I was surprised by the stark contradictions of the categories I was put in. Somehow Facebook had labelled me as both an “Venezuelan ex-pat” and a “new American”, whatever that means. It placed me as “extreme liberal” but also “extreme anti communism/socialism”. No wonder I was being shown ads for guns and to sign up to the NRA while also getting articles anti “the Israeli agenda”. My data and interactions are being interpreted in a binary way “if she’s anti-socialism she must like guns” “if she’s liberal she must be pro-Palestine”.

You can view, organize, add, delete, modify these categories to better fit your needs and interests. You can also report/hide adds and articles that you don’t want to see. I’ve done both to no change on my timeline. I’ve deleted all categories and associated interests by interaction, and also manually reported NRA adds in vain.

What is interesting to me is that clearly the large sets of data are being interpreted in a way that is not veridic with my experience. However, the data that I am willingly providing is not being taken into account in that process. To try to unpack all the issues behind that fact will require a lot more space than what I have in this post.

Johnson et al (2018) raise concerns regarding the over optimistic approach to big data while dismissing its challenges and misuse, “One can easily imagine what would happen if medical, financial, and behavioral data fused for the targeted individual fell into hands of bankers, insurers, politicians, or criminals. Mayhem would follow, no doubt.” (Johnson et al, 2018)

I have another personal example to illustrate this. A few years ago I randomly started receiving packages, directly addressed to me by name and address, containing information and free samples of products related to motherhood and babies: boxes of baby formula, pacifiers, pads, among others. This was a surprise to me since a) I’m not a mother, b) I was not planning to be at the time, c) no one in my household was pregnant, d) there were no babies in my house. The fact that every correspondence was addressed to me showcased that this was not a mistake.

I couldn’t figure out how this company had gotten hold of my address or why they thought I was a target for these products. When I visited the website, the only contact available was filling a form (and never got a reply from them). I received another package on my birthday with a letter saying something along the lines of “another year, it’s time to start planning on expanding your family” while also addressing the efficiency of birth control methods.

After much thinking, I remembered taking a survey related to birth control practices and medical conditions around chronic ovary illness and reproduction. After much scrolling down on my email, I found the survey link and realized that it was hosted by Amazon MTurk (although I didn’t take the survey on the MTurk website). I figured that’s how they got my address, birthday, age and other extremely detail information about me. It seems Amazon knows my habits very well, it knows my medical conditions and thinks it’s time for me to have a baby. 

I wish I had the time to unpack how the processing of all these sets of data about me determined that as a woman of my age I must absolutely either have babies or be thinking about having babies. Maybe I’ll save it for a final project.

References:

Gender Bias & AI: Questions and Challenges

Over the last few years, prominent figures and big companies in Silicon Valley have participated in public debate over the benefits and concerns over artificial intelligence and machine learning and its possible consequences for humanity. Some embrace technology advancement openly, advocating for a non-limiting environment arguing that, otherwise, it would prevent from progress and innovation. Others offer warnings similar to those of a sci-fi utopia film; they argue that artificial intelligence could be an existential threat to humanity if -or more likely when- machines become smarter than humans.

The defenders of the latter insist that, as much as unrealistic as it sounds, it is a very possible future. As unsettling as it is, they focus on a hypothetical threat forcing us to rethink and assess how are we managing the process of machine learning. However, more than looking into how fixing a utopic future before it happens, there are other questions that arise during this process:

Machines are learning how humans think and behave based on sets of data that humans ‘feed’ them, therefore, what are we feeding to these machines? If machines are learning how to imitate human cognitive processes, then what kind of human behaviors, social constructions, and biases are these machines picking up and replicating by design?

There is a long history of cases in which technology has been designed with unnecessarily deterministic biases on it: the famous case of the low bridges in New York preventing minorities from using public transportation to go to the beach; the for-long-time perpetuated ‘flesh’ color of crayons, band aids, paint, and more recently ballerina shoes; the famous case of Kodak’s Shirley Cards used by photo labs to calibrate skin tones, shadows and light during the printing process of color film, making it impossible to print darker skin facial expressions and details, among others.

We couldn’t expect any less than this pattern of embedding biases in technology being replicated when it comes to artificial intelligence and machine learning.

There is a systematic racist, sexist, gendered, class-oriented, and other axes of discrimination bias embedded in the data that has been collected by humans, and those patterns and principles are being picked up and replicated by the machines. Therefore, instead of erasing divisions through objectivity in decision making, this process is exacerbating inequality in the workplace, the legal and judicial systems, and other spaces of public life in which minorities interact making it even more difficult to escape from it.

The data fed to the machines is diverse: images, text, audio, etc. The decision of what data is fed to the machine and how to categorize it is entirely human. Based on this, the system will build a model of the world accepted as a unique reality. That is, only what is represented by the data have the meaning attached to it, without room for other ways of ‘being’ in the world. For example, facial recognition trained on overwhelmingly categorizing white men as successful potential candidates for a job position, will struggle to pick up others that don’t fit into those categories or labels.

(Unnecessarily) Gendering technology

There are two aspects that need to be taken into account to get a broader perspective: 1) the lack of transparency from companies to reveal how these systems make data-driven decisions due to intellectual property and market competition; and 2) the gendered somewhat contradictory representation of these technologies to the users and in pop culture media as well. Let’s start by addressing the latter.

For decades, visual mediated spaces of representation such as movies and tv in the genre of sci-fi, have delved into topics of technology and sentient machines. Irit Sternberg states that these representations tend to ‘gender’ artificial intelligence as female and rebellious: “It goes back to the mother of all Sci-Fi, “Metropolis” (1927), which heavily influenced the futuristic aesthetics and concepts of innovative films that came decades later. In two relatively new films, “Her” (2013) and “Ex-Machina” (2014), as well as in the TV-series “Westworld”, feminism and AI are intertwined.” (2018, October 8).

These depictions present a gender power struggle between AI and humans, which is sometimes problematic and others empowering: “In all three cases, the seductive power of a female body (or voice, which still is an embodiment to a certain extent) plays a pivotal role and leads to either death or heartbreak”. However, the representation of the level of agency in a female-gendered AI offers the imagined possibility that, through technology, systematic patriarchal oppression can be challenged and surpassed by the oppressed.

AIs are marketed with feminine identities, names and voices. Examples such as Alexa, Siri, Cortana demonstrate this; even though they enable male identities, the fact that the predetermined setting is female speaks loudly. Another example is the female humanoid robot Sophia, developed by Hanson Robotics in Hong Kong. Sophia is clearly built as a representation of a white slender woman with no hair (enhancing her humanoid appearance) and, inexplicably, with heavy make up on her lips, eyes and eyebrows.

Creator David Hanson says that Sophia uses artificial intelligence, visual data processing and facial and voice recognition. She is capable of replicating up to 50 human gestures and facial expressions and is able to hold a simple conversation about predetermined simple topics, but she is designed to get smarter over time, improving her answers and social skills. Sophia is the first robot to receive citizenship of any country (Saudi Arabia), she was also named United Nations Development Programme’s first ever Innovation Champion, making her the first non-human to be given any United Nations title.

These facts are mind-boggling. As Sternberg asks, “why is it that a feminine humanoid is accepted as a citizen in a country that would not let women get out of the house without a guardian and a hijab?” (2018, October 8). What reaction do engineers and builders assume the female presence and identification generates during the human-machine interaction?

Sternberg says that, fictional and real decisions of choosing feminine characters are replicas of gender relations and social constructs that already exist in our society: “does giving a personal assistant feminine identity provide the user (male or female) with a sense of control and personal satisfaction, originating in the capability to boss her around?” (2018, October 8). As a follow up question, is that what we want the machines to learn and replicate?

If machines are going to replicate human behavior, what kind of human do we need them to be? This is a more present and already underway threat. As Kate Crawford wrote in the New York Times, the existential threat of a world overtaking by machines rebelling against humans might be frightening to the male white elite that dominates Silicon Valley, “but for those who already face marginalization or bias, the threats are here” (2016, June 26).

References:

Google Assistant

Her (Jonze, 2013) installing OS1/Samantha

This week’s focus is on A.I. and specifically virtual assistants. As a fan of cinema, sci-fi, and representation of technology in the moving image, I can’t help but think of a few examples such as Her (Jonze, 2013), A.I. Artificial Intelligence (Spielberg, 2001), Ex-Machina (Garland, 2015), Blade Runner (Scott, 1982), Minority Report (Spielberg, 2002), 2001: A Space Odissey (Kubrick, 1968), and the list goes on and on.

I must confess that I’m not a big fan of voice recognition virtual assistants. I don’t have an Amazon Echo, Google Home and I’ve deactivated the “listen for Hey Siri” option on my iPhone. Digging deeper into the reasons for my dislike, I’ve come to the conclusion that it has to be because I was first exposed to all these dystopian films before being given the tools to actually understand how do the technology works. These fictional representations often present these technologies exaggerated/distorted with some ‘truth’ at its core. Watching these films doesn’t necessarily prevent me from de-blackboxing AI or voice recognition virtual assistants, but it definitely provides a filter through which we can understand not only how they work but how users understand and interact with them

While reading through the Google Assistant patent I was surprised at finding that, although most of the specifications are too technical for my understanding, the main description of its use and purpose was very accessible and even more clarifying than most attempts from articles to ‘unveil’ the mystery to the reader.

The patent reads:

“According to various embodiments of the present invention, intelligent automated assistant systems may be configured, designed, and/or operable to provide various different types of operations, functionalities, and/or features, and/or to combine a plurality of features, operations, and applications of an electronic device on which it is installed.”

Based on this excerpt, the patent describes the system as an intermediary between the user and many possible outcomes/actions that are already available in the devices, accessible through different modes of interaction.

If we look into the different levels/layers/steps into how Google Assistant works, the patent describes:

  • “…actively eliciting input from a user,
  • interpreting user intent,
  • disambiguating among competing interpretations,
  • requesting and receiving clarifying information as needed,
  • and performing (or initiating) actions based on the discerned intent.

Those actions can vary from activating and/or interacting with other applications and services already on the device, or accessible through the Internet: it can perform a google search on your question and provide answers, it can activate google maps or Spotify, it can perform e-commerce interactions such as buying things on Amazon, among others.

Some of the language used through the description in the patent was interesting to me. At one point it says, “[thanks to the assistant] The user can thereby be relieved of the burden of learning what functionality may be available on the device and on web-connected services, how to interface with such services to get what he or she wants, and how to interpret the output received from such services; rather, the assistant… can act as a go-between between the user and such diverse services.”

Oh to be relieved of the burden of learning how something works. This [insert any technology here] makes life so much easier we shouldn’t concern ourselves with the technicalities of how does it work.

I will admit that the benefits of voice recognition virtual assistants are massive for different communities and fields of work. The patent describes in detail how this serves people with disabilities and users who work handling machinery and cannot interact with devices at the same time without shifting their attention, which could be possibly dangerous. Not just for work, a great example is making a call or searching for something while driving.

Although all of this is true and valid, it must be acknowledged that it also opens the door to many vulnerabilities and security issues for users, as many technologies do. Cases of stolen identity, e-commerce fraud, home security, children protection, scams, etc. Last year, the New York Times published an article regarding research studies from various US and China universities on malicious use of these technologies, specifically “Berkeley researchers published a research paper that went further, saying they could embed commands directly into recordings of music or spoken text. So while a human listener hears someone talking or an orchestra playing, Amazon’s Echo speaker might hear an instruction to add something to your shopping list.

Therefore, there should be concern. Not the dystopian sci-fi movie’s fear around technology taking over, but about humans using these technologies to take advantage of the users. As much as I love/hate the greatest villain in film (in my humble opinion) Hal9000, I admit the threat of an embedded hidden command that I cannot hear but Echo can, seems exponentially more terrifying.

2001: A Space Odyssey (Kubrick, 1968). Hal9000

 

References:

Information theory: sending a digital image through the Internet

In this week’s readings we explored fundamental concepts around information and data. The nuances of the terminology used in different fields and how the meaning changes in between them: meaning, value, symbol, information, among others. Let’s attempt to de-Blackbox the process of sending a picture from one computer to another computer through the internet.

When making the argument of information theory, Shannon makes a distinction:

“The fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at another point. Frequently the messages have meaning; that is, they refer to or are correlated according to some system with certain physical or conceptual entities.” (Shannon, 1949)

Applying this to the case of sending a picture, Shannon differentiates between the picture as we see it and understand it as a concept and the information being sent and received. He is saying that communication’s only concern is how to move the bits from one computer to the other one. In this particular case those bits represent an image, but it can be other types of data. To that sense he says:

“These semantic aspects of communication are irrelevant to the engineering problem. The significant aspect is that the actual message is one selected from a set of possible messages. The system must be designed to operate for each possible selection, not just the one which will actually be chosen since this is unknown at the time of design.” (Shannon, 1949)

Therefore, the meaning of those bits does not change or impact the action of selecting sending a package of information and receiving it on the other side. The way I envision it is as a series of translations of different types of D-information in order to be exchanged and reproduced through E-information. In that sense, E-information is the bits, D-information is the type of data that those bits represent.

  1. The picture is encoded into a certain type of structured data that is stored and processed by the computer: 

    “Computational devices (large or small) are designed to store and process bits (binary units) — millions of on/off switches — which are encoded, at a different design level in the system, as units of the data type that they represent.” (Irvine, Intro, pp.3)

    Let’s say the size of our picture is 24mb. Those 24mb need to be received at the destination in order to be decoded again into the actual picture. In order to do that:

  2. The file (picture) is divided into packages of encoded bits that will be sent through the network. The router, as the name suggests, send the packages through different routes. Every package has encoded information that states its origin, its destination, its number in the total of packages of the file, and the actual encoded information that is a partial size of the file. 

    “Programming “code,” when translated into these mathematical-electronic mappings, is designed to “encode” our symbolic “data” structures together with the logical and mathematical principles required for transforming them into new structures” (Irvine, Intro, pp.3)

    Let’s say our picture of 24mb is divided into 4 packages of 6mb each:

  3. The packages arrive to their final destination, after bouncing around through other serves in the network. They arrive not necessarily in a numerical order. The destination router receives the packages as they arrive.
  4. The computer arranges the packages in the right order until it has the file in its total.
  5. Our computer’s software decodes the file of 24mb into the visual representation of our picture.

To address one part of the prompt question for this week: How do we recognize the difference between E-information transmitted and received (successfully or unsuccessfully) and what a text message, an email message, social media post, or digital image means? 

E-information transmitted and received are the packages of bits sent through the network. Meanwhile, our picture or digital image is D-information, data type in which these bits are structured to have the specific meaning of a digital image. Therefore, there is a process of encode-decode from the sender to the receiver. However, at the core of what Shannon proposed is the idea that the process of measuring and encoding information is independent from the meaning of said information.

 

References:

Reactionary Reporting on Technology

This is a long delayed post meant to be posted last week. My apologies.

At a Martin Luther King Jr Day event, Alexandria Ocasio-Cortez (rep. D-NY), explained some of the questions we’ve worked with last week’s readings and this week’s as well. Algorithms, and technology in general, is designed by humans. Humans are biased and there is a history of replicating those biases into technology. There are many examples in many fields of biased data fed to algorithms that produce discriminatory results.

The first reports of her comments were clearly sensationalist and inaccurate. One headline said “Ocasio-Cortez says algorithms, a math concept, are racist”. It made me think of the topic we’ve discussed in class of how is media reporting and talking about technology. Media are either repeating or paraphrasing press releases from tech companies without a critical analysis, or the people reporting on technology don’t take the time to dig deeper into the technologies to understand how they work and what are the real concerns beyond sci-fi sensationalism.

Looking at that headline, it is clear that the representative’s comments are not exactly what the headline says, but that is a whole other issue that is separate from the social issue surrounding the technology she’s describing. A week after the comments, we can find more in-depth reporting explaining the different ways in which her statements are accurate, citing experts in the field, and providing examples of peer-reviewed studies that have addressed these issues for a long time.

Therefore, it might seem that there are two types of reporting about technology. First, an immediate reactionary over simplification of a technology in order to create an emotion (positive or negative) in order to get user engagement quickly; this kind of reporting is the one most prone to inaccuracy and misconceptions. The second one, is usually done by people that have a deeper understanding of the technology and take the time to research and provide proof and examples for a more broad and critical take on the issue. However, the second one usually appears as a reaction of the first one, or the first one has a wider reach to the public, which makes it difficult for more in-depth reporting to navigate its way through all the rhetoric garbage to the user.

 

References:

https://www.vox.com/science-and-health/2019/1/23/18194717/alexandria-ocasio-cortez-ai-bias

Languages, AI & Representation

This week’s reading shed a light through foundational concepts regarding AI and computation while also clarifying myths and public misconceptions around it. Particularly, I was drawn to the topics of big data and algorithms in combination on the public’s struggle to separate autonomy in these machines from what they consider human autonomy, consequently giving less relevance to human intervention in various stages of these processes than it deserves.

When Boden talks about natural language processes, I couldn’t help but connect it to the ideas presented by Denning and Martel in Great Principles of Computing when talking about the evolution of the studies under the domain of AI: “The focus shifted from trying to model the way the human mind works to simply building systems that could take over human cognitive work”. The two ideas felt connected when we look at the example of online translators and how ineffective they are in performing the task that we expect from them. If you’ve ever used an online translator it becomes clear, very quickly, that it is not effective in terms of whole sentences but does a fair job at finding synonyms and other uses when it comes to single words. It made me think of another example by the authors in which a machine could use Chinese words and respond but could not have a comprehensive understanding of the Chinese language. 

It seems as if the root of this failure goes back to the statement of building systems that take over human cognitive work instead of replicating the human mind. It seems that when it comes to language, there is a big gap between what the machine can produce and what the actual interpretation of the language is. As a Spanish instructor, I see it very clearly when my students attempt to translate whole sentences or paragraphs on Google translate and become incredibly frustrated by how parts of the translation don’t match the grammar rules they have learned in class. I always tell them that language is not a literal translation of symbols that have an exact equivalent from one language to the other, but that language is more of an interpretation based on context, culture, historical, and geographical background that is combined with grammar rules or protocols that might have an equivalent in another language but that, most of the times, won’t have the same value in the translation. It seems to me that, this idea of two sets of values for symbols (some of which might change depending on variables) it’s difficult to accurately put into computation. Which is why online translators are so frustrating when you’re multilingual.

When reading about the current state of the discourse of AI by Johnson and Veridicchio, it was satisfying to see such a clear breakdown of the questions and concerns I tend to have about the way AI is represented or misrepresented in the public discourse. While it clarified the distinction of where to put the weight of responsibility between humans and AI, it left open questions on how some social decisions were made around the representation and design of AI. To me, the biggest question that has plagued my mind and continues to do so is the, in my opinion, unnecessary gendering of AI and the implicit connotations it has about gender in society.

Fictional representations of AI fascinate me. As an avid sci-fi enthusiast, I’ve always been intrigued by dystopian representations of robots/AI apocalypses and genderization of AI in these contexts. Not because, like Johnson and Veridicchio’s article says, I fall into the trap of the fear of annihilation by our machine overlords, but because these representations say more about how humans think about humans than how humans think about machines. Underlying in these representations are profound ethical critiques about the state of society and human behavior. Specifically, sci-fi dealing with gender representation of AI in these dystopias say more about how we think about women than how we think about machines.

Therefore, it is my opinion that, maybe on a subconscious level, we do understand that if AIs “decide” to take over and enact (gendered) violence against humans and annihilate them based on considering them an “inferior race”, it will be because they were designed that way by humans who also enacted gendered violence and annihilated what they considered “inferior races” as well.

 

References:

  • Peter J. Denning and Craig H. Martell. Great Principles of Computing. Cambridge, MA: The MIT Press, 2015. Chapters 1-2.
  • Margaret A. Boden, AI: Its Nature and Future (Oxford: Oxford University Press, 2016).
  • Deborah G. Johnson and Mario Verdicchio, “Reframing AI Discourse,” Minds and Machines 27, no. 4 (December 1, 2017): 575–90.