Author Archives: Adey Zegeye

Design/Ethical Implications of Explainable AI (XAI)


This paper will address the research question: what are the design and ethical implications of explainable AI? This paper will argue that there are three main reasons why XAI is necessary for user trust. These reasons pertrain to accountability/trustworthiness, liability/policy evaluation, and human agency/authority. This paper will use de-blackboxing methods and an analysis of current research and examples to uncover how XAI is defined, why it is necessary, and major benefits and criticisms of XAI models. With supporting evidence from Miller et al, the paper will argue that defining explainability to include human explanation models (cognitive psychology/sciences) will be significant to the development of XAI discourse.

Artificial Intelligence applications that use neural networks are able to produce results (i.e. image classification) with high accuracy, but without explanation for human end users, therefore classifying it as a black box system (Abdallat, 2019). Many articles claim that AI should be explainable but are not clear about how “explainable” is defined. This paper will de-blackbox explainable AI (XAI) by looking at how it is defined in AI research, why we need it, and specific examples of XAI models. Finally, it will address one of major gaps in current XAI models by arguing that explainable AI research should adopt an interdisciplinary research approach by building on frameworks of explanations from social science (Miller, Howe, & Sonenberg, 2017).  

XAI Types & Definitions

Opaque Systems

An opaque systems inner workings are invisible to the user. The system is taking in information and outputting new information or predictions, without clear evidence to why or how the output was chosen. In the case where an algorithm can’t provide the programmer with reasoning behind it’s decision-making process, this is considered a “black box” approach, and classified as opaque. Additionally, opaque systems often emerge when closed-source AI is licensed by an organization, and therefore hidden from the public in protection of IP (Doran, Schulz, & Besold, 2017).  

Interpretable Systems

An interpretable system is a transparent model that allows the user to understand how inputs are mathematically mapped to outputs. One example is a regression model, which is linear and uses weights to rank importance of each feature to the mapping. On the other hand, deep neural networks have input features which are learned from non-linearities, therefore would not be considered an interpretable model (Doran, Schulz, & Besold, 2017).

Comprehensible Systems

Comprehensible systems “emit symbols enabling user-driven explanations of how a conclusion is reached. These symbols (most often words, but also visualizations, etc.) allow the user to relate properties of the inputs to their output. The user is responsible for compiling and comprehending the symbols, relying on her own implicit form of knowledge and reasoning about them” (Doran, Schulz, & Besold, 2017).

Why Do we need XAI?

The three main reasons we need AI are as follows:

  1. Accountability + Trustworthiness
  2. Liability and Policy Evaluation
  3. Human Agency and Authority

Accountability, Liability and Policy Evaluation

Explainable AI is specifically important in cases dealing with human health, safety, and liability issues. In these cases, it is ethical to hold someone accountable for incorrect or discriminatory outcomes. Additionally, the issue of explanablity is a factor that can inform policy on whether AI should be incorporated into certain sensitive fields (Paudyal, 2019). For example, should a process like driving a motor vehicle be automated? These questions illuminate the importance of critical discourse that asks hard questions such as: what we are willing to sacrifice as a society for automation and convenience? In 2018, a self-driving car knocked down and killed a pedestrian in Tempe, Arizona (Paudyal, 2019). “Issues like who is to blame (accountability), who to prevent this (safety) and whether to ban self-driving cars (liability and policy evaluation) all require AI models used to make those decisions to be interpretable (Paudyal, 2019). In this case, I argue that when the safety of the public is concerned, it is clear that XAI is necessary.


Trusting a neural network to make decisions will have different implications depending on the task required. One of the strongest arguments for XAI is within the medical domain. If a neural network is built to predict health outcomes for a patient (risk of cancer or heart disease) based on their records, but can’t provide reasoning for the decision – is it ethical to trust it? The lack of transparency is a problem for the clinician who wants to understand the model’s process, as well as the patient who is interested in the proof and reasoning behind the prediction (Ferris, 2018). According to Ferris, empathy is a strong component to the patient-client relationship that should be taken into account when implementing these systems. In the case of medical predictions, I argue that XAI is necessary to ensure a level of trust with their physician. The point of predictive models and algorithms is to help advance user experience (as well as the experience and knowledge of the experts). In the case of patient-physician relationship, trust should be prioritized and XAI methods should be incorporated to support that.


Reversed Time Attention Model (RETAIN)

The RETAIN explanation model was developed at Georgia Institute of Technology by Edward Choi et at (2017). The model was designed to predict if a patient was at risk for heart failure using patient history (including recorded events of each visit). This model aims to address the performance vs. interpretability issue (mentioned in criticism section). “RETAIN achieves high accuracy while remaining clinically interpretable and is based on a two-level neural attention model that detects influential past visits and significant clinical variables within those visits (e.g. key diagnoses). RETAIN mimics physician practice by attending the EHR data in a reverse time order so that recent clinical visits are likely to receive higher attention” (Choi et al., 2016).  

Image Source:

By splitting the input into two recurrent neural nets (pictured above), the researchers were able to use attention mechanisms to understand what each network was focusing on. The model was “able to make use of the alpha and beta parameters to output which hospital visits (and which events within a visit) influenced its choice” (Ferris, 2018).

Local Interpretable Model-Agnostic Explanations (LIME)

Post-hoc models provide explanations after decisions have been made. The key concept in the LIME model is perturbing the inputs and analyzing the effect on the model’s outputs (Ferris, 2018).  This is an example of an agnostic model, meaning the process can be applied to any model and produce explanations. By looking at the outputs, it can be inferred what aspects the model is focusing on. Ferris uses the example of a CNN image classification to demonstrate how this model works in four steps.

Step 1. Begin with a normal image and use the black-box model to produce a probability distribution over the classes.

Step 2. Alter the image slightly (ex. hiding pixels), then run the black-box model again to determine what probabilities changed.

Step 3. Use an explainable model (such as a decision tree) on the dataset of perturbations and probabilities to extract the key features which explain the changes. “The model is locally weighted (we care more about the perturbations that are most similar to the original image.”

Step 4. Output the features (in this case, pixels) with the greatest weights as the explanation (Ferris, 2018).

Criticism and Challenges of XAI

  • Complexity Argument

There are a few major criticisms of explainable artificial intelligence to consider. Firstly, neural networks and deep learning models are multi-layered and therefore complex and overwhelming to understand. One of the benefits of neural networks it’s ability to store and classify large amounts of data (human brains could not process information in this way). According to Paudyal,  “AI models with good performance have around 100 million numbers that were learned during training” (2019). With this in mind, it is unrealistic to track and understand each layer and process of a neural network, in order to find valid source for explanation.

G-Flops vs accuracy for various models | Image source: Paudyal, 2019

  1. Performance vs. Explainability Argument

The second main criticism of XAI is that the more interpretable a model is, the more the performance lags. This ethical implication of this is that efficiency make take precedence over explanation, which could lead to accountability issues.

“Machine learning in classification works by: 1) transforming the input feature space into a different representation (feature engineering) and 2) searching for a decision boundary to separate the classes in that representation space. (optimization). Modern deep learning approaches perform 1 and 2 jointly by via. a hierarchical representation learning” (Paudyal, 2019).

Image Source: Paudyal, 2019

Performance is the top concern in advancement of the field, therefore explainable models are not favored when performance is affected. This factor supports a need for non-technical stakeholders to be a part of the conversation surrounding XAI (Miller, Howe, & Sonenberg, 2017). If the only people with a voice are concerned with performance, it could lead to focus on short-term outcomes rather than the longer term implications for human agency, trustworthiness of AI, and policy.

An Alternative Method: Incorporating XAI Earlier in the Design Process

In contrast to most current XAI models, Paudyal argues that  deciding if an application needs explanation should be discussed early enough to be incorporated into the architectural design (2019).

Image Source: Paudyal, 2019

As an alternative to using simpler but explainable models with low performance, he proposes that (1) creators should know what explanations are desired through consultation with stakeholders and (2)  the architecture of the learning method should be designed to give intermediate results that pertain to these explanations (2019). This decision process will require an interdisciplinary approach, because it is clear that in defining and understanding what type of explainability is needed for a specific application will require discussion across disciplines (computer science, philosophy, cognitive psychology, sociology). “Explainable AI is more likely to succeed if researchers and practitioners understand, adopt, implement, and improve models from the vast and valuable bodies of research in philosophy, psychology, and cognitive science; and if evaluation of these models is focused more on people than on technology” (Miller, Howe, & Sonenberg, 2017).  These disciplines should be working together to discover what systems require explanation and for what reasons, before implementation and testing begins. In the next section, I will de-blackbox this method further by providing limitations and illustrating the method with an example.

Example & Limitations

Paudyal addresses that for this method different applications will require different explanations (loan application vs. face identification algorithm). Although this method would not be agnostic, it supports the fact that complex systems will not be able to be explained in simple ‘one size fits all’ approaches. It is important to address this challenge in order to come up with realistic XAI models that include the socio-political and ethical implications into the design.


The following case looks at the possibility of a system designed to incorporate explanations for an AI application that teaches sign language words. In a normal black-box application system, the AI would identify an incorrect sign but would not be able to give feedback. In this case, explanation will be equivalent to feedback about what was wrong in the sign.  Paudyal’s research found “Sign Language Linguists have postulated that they way signs differ from each other either in the location of signing, the movement or the hand-shape” ( 2019).

Image Source: Paudyal, 2019

With this information, AI models can be trained to focus on the these three attributes (location of signing, movement and hand-shape). When a new learner makes a mistake, the model will be able to identify which mistake was made and provide the appropriate specific feedback (Paudyal, 2019).

Image Source: Paudyal, 2019

The main insight found through this example is that AI models which use possible outcomes in the design of the application, are easier to understand, interpret, and explain. This is due to the human design knowing what the application will be training for. This example also supports the earlier statement that the design will be specific to the application (this process is specific to sign language CNN).


This paper examined several issues with lack of transparency in machine learning and utilization of deep neural networks, specifically in scenarios where responsibility is hard to determine and analyze for policy. These challenges in the AI field have resulted in efforts to create explainable methods and models. From here, another significant challenge was introduced in defining explainability. Through the examples and cases mentioned, it is clear that explainability will have different meaning depending on various factors including the user’s comprehension, background, and industry. Due to this, I argue (with support from Paudyal’s argument) that explainability should be discussed in the first stages of the design process. In doing so, the process is made more clear and it is easier to develop XAI from the beginning of application design, rather than after it is created. This brings authority and agency back in to the hands of humans, and addresses the argument that explainability will affect performance. Although incorporating explanation earlier in the design does have some limitations, it may ultimately lead to better design practices that do not focus on short-term outcomes. Lastly, I close by arguing explainability calls for interdisciplinary collaboration.  “A strong understanding of how people define, generate, select, evaluate, and present explanations” is essential to creating XAI models that will be understood by users (and not just AI researchers) (Miller, 2017). Further research might explore the questions: who is defining XAI, who is XAI designed to appease, and why aren’t experts in human explanation models at the forefront of approaching these questions?


Abdallat, A. J. (2019, February 22). Explainable AI: Why We Need To Open The Black Box. Retrieved from Forbes website:

Choi, E., Bahadori, M. T., Kulas, J. A., Schuetz, A., Stewart, W. F., & Sun, J. (2016). RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanism. ArXiv:1608.05745 [Cs]. Retrieved from

Doran, D., Schulz, S., & Besold, T. R. (2017). What Does Explainable AI Really Mean? A New Conceptualization of Perspectives. ArXiv:1710.00794 [Cs]. Retrieved from

Ferris, P. (2018, August 27). An introduction to explainable AI, and why we need it. Retrieved  from website:

Miller, T., Howe, P., & Sonenberg, L. (2017). Explainable AI: Beware of Inmates Running the Asylum. 7.

Miller, T. (2017). Explanation in Artificial Intelligence: Insights from the Social Sciences. ArXiv:1706.07269 [Cs]. Retrieved from

Paudyal, P. (2019, March 4). Should AI explain itself? or should we design Explainable AI so that it doesn’t have to. Retrieved from Towards Data Science website:

Samek, W., Wiegand, T., & Müller, K.-R. (2017). Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models. ArXiv:1708.08296 [Cs, Stat]. Retrieved from

Machine Learning & Deep Text Combat Cyberbullying

Tianyi Zhao and Adey Zegeye

Machine Learning –  using “algorithms to get computer systems to go through content (images, text) and identify various trends and patterns across all of those data, based on what we have told them to look for (e.g., training it on labeled data – content that a human has already manually classified in some way – toxic or not toxic, abusive or not abusive). This can actually be done on unlabeled data as well, via what is called unsupervised learning – where the algorithm(s) tries to cluster the data into groups on its own).”

Deep Learning –” subset of machine learning– after the system identifies trends and patterns across data by analyzing content, we ask it to constantly improve its probability of accurately classifying that content by continually training itself on new data that it receives.”

How Machine Learning Can Classify Online Abuse

The different layers might:

1. Extract Curse words (that the programmer has listed as abusive)

2. The second later would count all the curse words up and divide them by the number of words in the text message it appears (to signal severity_

  1. Third layer might look at words in all CAPS
  2. Foruth layer might look at how many hateful words have second-person pronouns meaning they were directed at someone else
  3. Fifth layer might check if this poster has been previously flagged for abusive content 
  4. Sixth layer might look at punctuation (could imply tone)

Additional layers might check for attached images/video and see if that has been classified as abusive before 

DeepText Utilized in Instagram:

DeepText was firstly successful in spam filtering, and then moved to develop the mean comments elimination. Each person in the developing team looks at a comment and determines whether it is appropriate. If it’s not, he sorts it into a category of verboten behavior, like bullying, racism, or sexual harassment. Until launching in 2017, the raters, all of whom are at least bilingual, had analyzed roughly two million comments, and each comment had been rated at least twice. Simultaneously, this system had been testing internally, and the company adjusted the algorithms: selecting and modifying ones that seem to work and discarding ones that do not. The machines give each comment a core between 0 and 1, measuring the comment is offensive or inappropriate. If it is above a certain threshold, the comment is deleted.

The comments are rated based on several factors, semantic analysis of the text, the relationship between the commenter and the poster, and the commenter’s history. The system analyzes the semantics of each sentence, and also took the source into account. A comment from someone that the user does not follow is more likely to be deleted than one from someone the user does. Also, the comment that repeated endless on Martin Garrix’s feed is probably being made by a human.The technology is automatically incorporated into users’ feeds, but it can be turned off as well.


Figure 1. Turn-on/off the Comment Filter in Settings



Pros & Cons


  1. Automating the process of deleting hate speech and offensive comments helps filter out unwanted content on Instagram
  2. DeepText becomes more effective by allowing users to manually enter words or phrases they want blocked


  1. Characters in hateful words are replaced with symbols to avoid detection;
  2. Some comments may not contain any problematic words but still might be incredibly offensive;
  3. Acronyms and Internet slang are changing constantly;
  4. The system may delete innocuous or helpful comments by mistake.

Works Cited:

Systrom, Kevin. “Keeping Instagram a Safe Place for Self-Expression.” Instagram Press, Jun. 29, 2017.

Systrom, Kevin. “Protecting Our Community from Bullying Comments.” Instagram Press, May 1, 2018.

Marr, Bernerd. “The Amazing Ways Instagram Uses BIg Data And Artificial Intelligence.” Forbes, Mar. 16, 2018.

Hinduja, Sameer. “How Machine Learning Can Help Us Combat Online Abuse: A Primer.” Cyberbullying Research Center, Jun. 26, 2017.

Thompson, Nicholas. “Instagram Unleashes an AI System to Blast Away Nasty Comments.” Wired, Jun. 29, 2017.

Bayern, Macy. “How AI Became Instagram’s Weapon of Choice in the War.” Tech Republic, Aug. 14, 2017.


A Call for Regulation or Education? Reframing AI Discourse

The leading issues within AI discourse stem from a lack of transparency and information surrounding emerging technologies. In tackling some of the core categories within AI, it is useful to start by looking at media representation, which is where the general public go to get their knowledge on a given topic. 

From here, we can ask: what information do we actually have concerning a topic (Deep Neural Networks, Natural Language Processing, etc)? Where is the information coming from and who is the intended audience? Is it an article that is meant to advertise a product rather than to detail out accurate information?

During this course, we have covered key concepts that help de-mystify AI systems and also lead to more informed questions that can help us move forward in both our analysis and our understanding of the current climate surrounding these topics.

Reframing AI Discourse

One of the important distinctions to be made is that although AI is meant to “do the sorts of things that minds can do,”  the intelligence (or program) does not have an ability to think for itself (Boden, 2016, pg. 1).

Data and Representation

  • Everything depends on computable data, the context requires that it be defined as a certain data type
  • Deep Learning requires large amounts of data, where a human can recognize patterns more easily and accurately due to context
    • Deep learning is not deep knowledge or accumulated knowledge, but rather deep layers used within the network

Autonomy of Computational Artefacts vs. Autonomy in Humans 

  • AI systems are not autonomous. Machines are not like humans in that they do not have their own interests and freewill to act as they choose, debunks the myth that machines or robots will decide to take over human lif
  • The more important question to focus on: Who is creating AI systems (especially autonomous weapons + other tools for destruction) and has control over these artefacts?

Limitations of Neural Networks / Machine Learning 

  • Although Deep Neural Networks can be very useful in picking up pattern in large amounts of data, one of the main issues is that they don’t tell the whole picture. They can only select data based off of set parameters and rules – which does not translate into human problem solving or decision making.
  • A ConvNet is a large collection of filters that are applied on top of each other, they will learn to recognize the labels that we give them
  • Context is important when comparing what a neural network can do vs. a human brain
  • Algorithms are designed by humans, therefore a computer or AI system is not inherently biased. This is a big theme because a lot of media exists on discussing how AI is evil or bad — which takes the attention off of the systems we have in place and the algorithms we design. At the same time, when it comes to regulation — a big issue is that we don’t know all of the steps involved in the process of a deep neural network making a classification. This is due to the complexity of the system and hidden layers. 
  • Machine Learning is only one method of learning within AI Systems, and therefore should not be the only focus when looking at ethical implications of AI systems

The EU’s Set of Guidelines for Trustworthy AI tackles many of the ethical issues we have discussed throughout this course: privacy and data governance, transparency, human agency, bias, environmental well-being, and accountability. At the same time, these guidelines expose the problems implementing the regulation of AI. In learning about the ways AI is designed and programmed, it’s clear that some of these regulations are still too vague to be effective or implemented without becoming censorship. They are also very general guidelines which continues the cycle of blanket terminology and generalizations used by tech companies to “explain” their products, services, and policies.

Given this current example, we can see there needs to be more discourse surrounding the ethics of AI that is open and inclusive to creators and users alike. The ethical implications continually point to a need to understand what we are creating and the consequences of those creations to society as a whole. Software engineers and developers are not the only ones who need to be concerned with learning how emerging technology works — additionally, the method of teaching engineers only the mechanics and technical aspects of the process is in itself an ethical issue. Education, even more than regulation, will be necessary in order to move forward in creating systems that are safe and ethical. The more the information remains in the hands of a few corporations, the more likely we are to have a digital divide, and without the resources to teach / inform people about how AI systems work (due to IP issues etc) + how to develop alternative systems – we are stuck making guidelines without the means to implement them.


Gnovicon Response

Last week, I attended Gnovis’ academic conference Gnovicon -which focused on the topics of Big Tech, Data, and Democracy. 

Keynote speaker Siva Vaidhyanathan brought forward some important points on the future of democracy. He points out that Facebook was designed to share mass amounts of data – that it’s not a mistake but was an intentional design decision. We talk about this specific topic a lot in CCT – the responsibility of tech companies in how their products and services are designed. Vaidhyanathan draws attention to the fact that in the design process – Facebook developers did not consider what communication power would give people who will intentionally use it for harm. The algorithm choice – is just that – a choice. The FB algorithm rewards reactions and engagement, which keeps the most reactive news at the top of your timeline. This means spreading and circulating negative messages such as hate messages, conspiracy theories, and indignation. 

On another note, advertisement as a political tool works well in that you can pick exactly who sees which message and run different versions of an ad to see which one does better. For these reasons, politicians are moving their campaigns from TV to Facebook. 

The big takeaway from the keynote speech was that social media is not the root of the problem. Facebook is not where we should be focusing our attentions in order to make change. According to Vaidhyanathan (and the CCT community) – the real problem is that we are over-stimulated and distracted by constant noise. In the current climate people are spending time denying serious issues instead of working together to find solutions. He claims that social media and smartphones are habituating us into immediacy, and we are just reacting to things rather than making conscious decisions and long term plans to address large issues. 

After his speech, Emily Varga offered a solution to the large amounts of misinformation online. She said education can be a big factor in helping the situation by informing people on how to distinguish between good and bad information. Additionally Sally Hubbard emphasized the importance of competition in regulating FB – there need to be other options so that there is pressure to do better than the current algorithm. Without any competition to regulate FB, we will continue to see the effects of the current algorithm choices. This leads back to our discussion in class about how algorithms are a business choice and should be recognized as such. The overall message being that we should be looking to implement design choices that are concerned with the ethical aspects and consequences. 

The Cost of Cloud Computing

Cloud computing allows for seamless and convenient use of online services that previously required more time, space, and money. For example, when creating a document (thanks to Google Drive) – it is no longer necessary to store the document on your personal computer’s drive, which keeps storage space open. It also reduces the amount of email correspondence necessary when editing a document or working on a group project because Google Drive allows for “sharing” as well as simultaneous editing of the same document. When given the option of protecting data on your own computer vs. using a program like Google Drive or iCloud- the winner is often the most convenient option, unless the user is aware of the disadvantages of using such a tool. 

From our discussions in this course, we recognize that giving away our information in order to receive convenience comes with negative outcomes. A key issue is the lack of clear information on exactly how each cloud service works. This keeps big companies like Amazon and Google in control over the majority of the population, and the more black-boxed technologies continue to emerge, the more control these companies have over the rest of the market. The issue with only a few large tech companies providing these services is that they are not being held accountable by a universal standard or regulation, which keeps us informed as users and therefore unaware of all the consequences that accompany use.

Cloud Computing:

Host companies (Amazon, Google, Microsoft) – own and run the datacentres, servers, hard disks and processors for the computation 

Cloud Service Providers (SaaS) Google Drive, Dropbox – provide online services

Clouders: users of the service at home or business

Key Positive / Negative Consequences

Sharing & Storage Capacity

The ability to easily share large files through services such as Dropbox and Google Drive are a big positive consequence of cloud services. In addition, storing information within a cloud service rather than on a pc drive or external drive, has allowed for created a more minimalistic storage option that individuals and corporations are endlessly benefitting from. 

Saving Energy 

A 2010 Pike Research study (as cited in DeBruin and Floridi, 2017) found that cloud computing can reduce energy consumption by almost 40% – mostly due to “outsourcing computational tasks from inefficient local datacentres (or home and office computers) to the more efficient large datacentres of the hosting companies.” Cloud Computing can therefore be a solution to controlling the amount of energy used for computing by reducing the need for powerful hardware. On the other hand, more information about how many datacentres are needed to support cloud services and what impact they are having on the environment in totality. 

Lack of Transparency/Communication

Terms of service and license agreements are not user-friendly, and create lack of transparency in the industry-client relationship. These are usually created to avoid any legal repercussions rather than intending on informing the client about the service (DeBruin and Floridi, 2017). Although these agreements do include the most information heavy communication, they are not utilized as such from the users of the services due to the length and technical jargon. 

Business Costs

Large businesses benefit from the cloud architecture because they no longer have to pay for software to be installed, configured, and maintained on each computer (DeBruin and Floridi, 2017). The disadvantage is for smaller companies because they might not be able to pay for the services (fees, updates, etc) and therefore are left out of the advantages. Additionally, the services are intended by design for larger companies – which makes using the services more difficult and less tailored to the smaller ones. This is one of the major disadvantages of having the big four services – it automatically favors larger businesses that can afford to pay for and benefit from the cloud architecture. This leads to the question of who is really benefitting from cloud computing and is it contributing to the digital divide rather than closing it? 

“While cloud computing seems to be a boon to a population that cannot afford the computer equipment that is necessary for today’s IT—a very simple laptop is sufficient for cloud computing—it also requires reliable, ubiquitous and high speed Internet connections that are almost entirely absent, and if not absent very expensive, in large parts of the world” (DeBruin and Floridi, 2017). 

Key Takeaway:

“To benefit genuinely from their freedom, people have to know what actions they can choose from and they have to know what the likely consequences of these various choice options are. In other words, they have to know the characteristics of their opportunities (DeBruin and Floridi, 2017) “

“Clouders need to have general knowledge about the advantages and disadvantages of cloud computing; and they need to have specific knowledge about the services they buy and use or consider buying or using” (DeBruin and Floridi, 2017).

This specific knowledge needed is not provided for users in an accessible way, which leaves users dependent on these services. The issue with the convergence of technologies all packages into one service provided by one of the large Cloud service providers – is dependence and loss of privacy, agency, and control. It is also contributing to the digital divide, in that using these services requires high speed internet connection and the ability to cover the costs of service.

Thinking through this has reminded me of the show Mr. Robot, and how computer hacking is equated with “owning” someone or some organization. Access to personal information is incredibly powerful in any setting, and companies are giving access away by trusting their information will be safe in the Cloud. In certain cases, users are unaware that their information is being stored at a third party. Additionally, if anything happens to compromise the date stored in the Cloud, the repercussions could be drastic for large companies. In any case, users of the service need to be aware of what they are agreeing to when they sign up to use a Cloud service.


DeBruin, Boudewijn, and Luciano Floridi. “The Ethics of Cloud Computing.” Science and Engineering Ethics, vol. 23, no. 1, 2017, pp. 21–39.
Derrick Roundtree and Ileana Castrillo. The Basics of Cloud Computing: Understanding the Fundamentals of Cloud Computing in Theory and Practice. Amsterdam; Boston: Syngress / Elsevier, 2014. Excerpts from Introduction and Chap. 2.


Do Algorithms Have Politics?

Ethical issues in AI surround complicated problems such as data usage, privacy, and human agency. Thought leaders and professionals from all disciplines are clear about a need for some kind of  universal regulation and intentional design during the design process for AI systems and technologies. Throughout the readings and case studies, specific cases threatening human agency and human rights highlight some key issues we are facing in developing ethical practices in AI design and implementation.

Predictive Algorithms 

Professor MacCarthy’s thought-provoking article looks at the implications of recidivism scores which measures the probability of whether a prison will reoffend once released. This form of decision-making is based on a predictive outcome, which can be challenged as an unethical practice. “The use of recidivism scores replaces the question of whether people deserve a lengthy sentence with the separate question of whether it is safe to release them. In using a risk score, we have changed our notion of justice in sentencing” (MacCarthy). He further illustrates the point that political stance has a direct influence on how the algorithm will be implemented – in that the algorithm must be programmed to take a stance. In this case, the question is: what should the job of the algorithm be?

“Those who believe in improving outcomes for disadvantaged groups want to use recidivism algorithms that equalize errors between African-Americans and whites. Those who want to treat people similarly regardless of their group membership want to use recidivism algorithms that accurately capture the real risk of recidivism. When recidivism rates differ, it is not possible for the same recidivism tool to achieve both ethical goals” (MaCarthy). 

This raises the question of what role algorithms should have in our society. Should they be given the task of predicting outcomes in the judicial system? Is that a fair means of judgement?  Should the same tactics used in War (tracking, sensors, etc) be incorporated into daily life? Who benefits, and at what cost? From MacCarthy’s article, it can be concluded that algorithms do have political consequences and should therefore be treated accordingly in order to protect human rights and agency.

Experts Look To the Future of AI

Barry Chudakov, founder and principal of Sertain Research, commented, “My greatest fear is that we adopt the logic of our emerging technologies – instant response, isolation behind screens, endless comparison of self-worth, fake self-presentation – without thinking or responding smartly” (Anderson et al.)

The issue of instant response and user engagement has brought a significant shift in the way we consume news and content. As most people receive their news from social media, this brings a responsibility into the hands of the dominating social media companies such as Facebook– the kind of responsibility and power that was not possible before social media. The content that we receive is designed (by algorithm) to engage us, not to give us the most recent or relevant information on news and public issues. Only seeing what one wants to see, or what is agreeable with their political views, has a consequence on the collective level. Some of these consequences include: how news will be made in the future, disinformation campaigns, hate speech, and false news/misleading ads (MacCarthy). 

Batya Friedman, a human-computer interaction professor at the University of Washington’s Information School, wrote, “Automated warfare – when autonomous weapons kill human beings without human engagement – can lead to a lack of responsibility for taking the enemy’s life or even knowledge that an enemy’s life has been taken” (Anderson et al.)

Another significant issue is the distance that is involved in using automatic weapons, drones, or machines in warfare. Within the issue of weaponry, being a certain distance away from the effects of such killing, creates a lack of empathy and visible effect  – therefore, creating an environment where killing is nameless and with less direct consequence. Changing the nature of taking human life by programming a machine to do it, is at the extreme end of the spectrum – but distancing involved tasking algorithms to do what humans previously did – is seen on an individual level as well. The isolation, inability to communicate face-to-face, and growing epidemic of loneliness are other signs of this loss of empathy, resulting from the ways we interact with technology vs. humans.

Although some of the predictions concerning the future of AI are falsely informed  (due to the characterization of AI as capable of thinking for itself – rather than a software that is programmed by humans), the questions that are under the black box of blanket terms that state AI will cure cancer, remove 90% percent of current jobs entirely, and similar predictions — is the question of dependence. We have already seen the drastic change in human dependence on technology, especially within the younger generations. As we continue to strive for convenience and instant gratification/growth, we sacrifice independence. Due to this, author Kostas Alexandridis predicts that in the future, “there is also a real likelihood that there will exist sharper divisions between digital ‘haves’ and ‘have-nots,’ as well as among technologically dependent digital infrastructures. Finally, there is the question of the new ‘commanding heights’ of the digital network infrastructure’s ownership and control” (Anderson et al.).

In designing AI technologies moving forward, not only is it important to keep ethics and human rights at the center of design – it is also important to inform the public about how these softwares work, so we have the ability to shape educated opinions and contribute to discourse as well as future designs. In avoiding the digital ‘haves’ and ‘have-not’ scenarios, we must be more concerned with questions surrounding who should be deciding what regulations need to be in place, and how to ensure humans continue to remain independent and informed. If the companies that are using the latest technology and data (such as IBM) are not willing to be straightforward and clear about how and what they are using – it will be difficult to regulate such practices to protect individual privacy. Many companies (IBM included) are hiding behind the ‘intellectual property protection’ excuse as means to keep information about where/how they are accessing data – which is a clear indication that the practices at large tech companies should be the focus when enforcing ethical policies. 



Anderson, Janna, et al. Artificial Intelligence and the Future of Humans. 10 Dec. 2018,
MacCarthy, Mark. “The Ethical Character of Algorithms—and What It Means for Fairness, the Character of Decision-Making, and the Future of News.” The Ethical Machine, 15 Mar. 2019,
Solon, Olivia, and Joe Murphy. “Facial Recognition’s ‘Dirty Little Secret’: Social Media Photos Used without Consent.” NBC News, Accessed 20 Mar. 2019.

“Hey Siri” – the DNN Acoustic Model

In our work de-blackboxing Google Translate, we learned the DNN’s job is to solve a problem. In the case of Google Translate the problem was to translate a phrase or sentence from one language(input) to another(output). In the case of a voice triggered personal assistant, the DNN will need to decode a voice command (input) and perform a task or answer a question (output). A Recurrent Neural Network was needed for Google Translate and for Apple’s Siri, a DNN Acoustic Model.

Layers / Process of Siri Voice Trigger 

(If using Siri on an iPhone)

  1. The microphone in your phone converts the sound of your voice into waveform samples 
  2. Spectrum analysis stage converts the waveform to a sequence of frames 
  3. ~20 frames at a time, are fed to the Deep Neural Network (DNN)
  4. Then, “The DNN converts each acoustic pattern into a probability distribution over a set of speech sound classes those used in the “Hey Siri” phrase, plus silence and other speech, for a total of about 20 sound classes (Siri Team, 2017). 


(Image retrieved from:

According to the  Apple’s Machine Learning article, An iPhone uses two networks (1. Detection, 2. Secondary Checker)

5. The way that the acoustic pattern is further detected is: if the outputs of the acoustic model have a high enough phonetic score for a target phrase. This process is further solidified through training –  over time, the more times a phrase is detected accurately – the more valid the sequence becomes. This process is shown in the top layer of image above as a recurrent network with connections to the same unit and the next in sequence (Team Siri, 2017). 

The DNN “hidden” layers in the neural network consist of learned representations during the training period of taking acoustic pattern (input) to sound classes (output).

In order to recreate the voice Siri’s voice trigger system – the main components we would need:

Hardware, Software and Internet Services

  1. A device with Internet connection (phone, smartwatch, bluetooth device)
  2. A microphone 
  3. Detector
  4. An Acoustic Input (voice)
  5. Server (can provide updates to acoustic models)
  6. Deep Neural Network — 2 networks : 1. Detection 2. Second Pass
  7. Training Process for the DNN
  8. Motion Coprocessor (to avoid using up battery life at all times the voice trigger is not being used)
  • Note: I have further questions about whether additional components listed in the below diagrams are a part of the above main features or if they need to be included as separate entities

This image looks at the active speech input procedure as a flow chart and includes the process of ranking interpretations for semantic relevance (process mentioned above) – this was also a key feature in the Google Translate process.

(image retrieved from:

Description of Automated Assistant from Apple Patent

“The conversation interface, and the ability to obtain information and perform follow-on task, are implemented, in at least some embodiments, by coordinating various components such as language components, dialog components, task management components, information management components and/or a plurality of external services” (Siri Team, 2017).

This quote is expressed in a useful image below – and helps to visualize the coordination of the components mentioned above.

(image retrieved from:



Siri Team. “Hey Siri: An On-Device DNN-Powered Voice Trigger for Apple’s Personal Assistant – Apple.” Apple Machine Learning Journal,
Gruber, Thomas Robert, et al. Intelligent Automated Assistant. US20120016678A1, 19 Jan. 2012,

Natural Language Processing + Google Translate

Language translation is more complex than a simple word-to-word replacement method. As seen in the readings and videos for this module, translating a text in another language needs more context than a dictionary can provide. This “context’ in language is known as grammar. Because computers do not understand grammar, they need a process in which they can deconstruct sentences and reconstruct them in another language in a way that makes sense. Words can have several different meanings and also depend on their structure within a sentence to make sense. Natural Language Processing addresses this problem of complexity and ambiguity in language translation.  The PBS Crash Course video breaks down how computers use NLP methods.

Deconstructing sentences into smaller pieces that could be easily processed:

  • In order for computers to deconstruct sentences, grammar is necessary
  • Development of Phrase Structure Rules which encapsulate the grammar of a language

Using phrase structures, computers are able to construct parse trees

*Image retrieved from:

Parse Trees: link every word with a likely part of speech+ show sentence construction

  • This helps computers process information more easily and accurately

The PBS video also explains this is the way that Siri is able to deconstruct simple word commands. Additionally, speech recognition apps with the best accuracy use deep neural networks. 

Looking at how Google Translate’s Neural Network works, the Code Emportium video describes a neural network as a problem solver. In the case of Google Translate, the neural networks job or problem to solve, is to take an English sentence (input) and turn it into a French translation (output).

As we learned from the data structures module, computers do not process information the way our brains do. They process information using numbers (vectors). So, the first step will always be to convert the language into computer language. For this particular task, a Recurrent Neural Network will be used (neural network specifically for sentences).

Step 1. Take English sentence and convert into computer language (a vector) using a recurrent neural network

Step 2. Convert vector to French sentence (using another recurrent neural network)

Image retrieved from:

According to research from a 2014 paper on Neural Machine Translation, the Encoder-Decoder Architecture model pictured above works best for medium length sentences with 15-20 words (Cho et al). The Code Emporium video tested out the LSTM-RNN Encoder method on longer sentences, and found that the translations did not work as well. This is due to the lack of complexity in this method. Recurrent Neural Networks use past information to generate the present information. The video gives the example:

“While generating the 10th word of the French sentence it looks at the first nine words in the English sentence.” The Recurrent Neural Network is only looking a the past words, and not the words that come after the current word. In language both the words that come before and after are important to the construction of the sentence. Therefore, a BiDirectional Neural Network is able to do just this.

Image retrieved from:

Bidirectional neural networks (looks at words that come before it and after it) Vs. Neural Network (only looks at words that come before it)

Using the BiDirectional model – which words (in the original source) should be focused on when generating the translation?

Now, the translator needs to learn how to align the input and output. This is learned by an additional unit called an attention mechanism (which French words will be generated by which English words)

This is the same process that Google Translate uses – on a larger scale

Google Translate Process & Architecture / Layer Breakdown

Image retrieved from video:

English translation is given to the encoder, which translates the sentence into a vector (each word gets assigned a number), then an attention mechanism is used next to determine the English words to focus on as it generated a French word, then the decoder will translate the French translation one word at a time (focusing on words determined by attention mechanism).

Works Cited

CrashCourse. Data Structures: Crash Course Computer Science #14. YouTube,
CrashCourse. Machine Learning & Artificial Intelligence: Crash Course Computer Science #34. YouTube,
CrashCourse. Natural Language Processing: Crash Course Computer Science #36. YouTube,
CS Dojo Community. How Google Translate Works – The Machine Learning Algorithm Explained!YouTube,
Thierry Poibeau, Machine Translation (Cambridge, MA: MIT Press, 2017). Selections

Can a Neural Network Think?

Discussion Notes – Tianyi Zhao and Adey Zegeye

Case Study: What a Deep Neural Network thinks about your #selfie – Andrej Karpathy


ConvNet Training in Karpathy


  • Data collection: defining a quick script to gather images tagged with #selfie. (5 million)
  • Convolutional networks trained to pick images with at least one face. (2 million)

To decide if the selfie is good or bad: 

  • ranked the users by the number of followers
  • Divided into groups of 100, and sorted by the number of likes
  • Top 50 = positive, bottom 50 = negative
  • Train a ConvNet with the binary split.
  • The ConvNet can be well customized based on different demands of the trainers.
  • The database is so large and various. There should be a more specific explanation about the statistical preferences or a detailed discussion.
  • The patterns should be strictly selected and apply to any selfie
  • The accuracy of ConvNet depends on:
    • How well and precise the trainer defines patterns
    • Different neural network architectures (Caffe, VGGNet, ImageNet, etc.)
    • ConvNet in NLP, in machine translation for example:

According to Ethem Alpaydin, the neural machine translation ends the era of phrase-based statistical translation, because it translates an entire sentence at a time rather than cutting it into words. Recurrent neural networks (RNNs) are prevalent in this field. However, ConvNet has gradually replaced RNNs in language translation. On the one hand, ConvNet can make computation fully parallelized in GPU. ConvNet computes all elements simultaneously, while RNNs operates in a strict left-to-righting or right-to-left order (one word at a time) in which each word must wait until the network finishes the previous one. On the other hand, ConvNet processes information hierarchically, making it “easier to capture complex relationships in the data.” (Gehring & Auli, 2017) 

Case Analysis : Key Points and Issues 

Although ConvNets can be very useful in picking up pattern in large amounts of data, one of the main issues is that they don’t tell the whole picture. They can only select data based off of set parameters and rules – which does not translate into human problem solving or decision making.

  • A ConvNet is a large collection of filters that are applied on top of each other
  • They will learn to recognize the labels that we give them
  • Context is important when comparing what a neural network can do vs. a human brain
  • In the Karpathy article, the 1/3 rule problem resulted in the network choosing a “logical” but not accurate for the purposes of human perception (the way a human would determine whether the selfie is good or not)
  • “What we lack in knowledge we make up for in data” – Alpaydin
    • Still includes limitations: the problem of binary (two-value logic)
    • It is not always true/false , yes/no, we think and speak in more complex patterns
      • In some of the selfies considered “good” by the convnet, the entire person is cut out of the image. The network does not have the ability to use context in the way humans do, to know or understand that selfies can be taken in different “moods” or patterns recognized in human language / images
    • “we live and think with multi-valued(not true or false, but “all depends…”) reasoning and multimodal(different kinds of statements: hypothetical, imaginary, contrary to fact, ironic, etc.)” – Irvine, 2019
      • Cropped Selfies
    • Interactive machines can replicate “intelligence” by copying and learning / adjusting but it is not in itself “intelligent”

Another main issue is inaccurate language used to describe what a neural network CAN do. Neural networks don’t think, so the neural network doesn’t “think” your selfie is good or bad – it simply uses the information it is given within a set of parameters to decide if the image is good or bad (using two-value logic). This language proves confusing without the background readings that explain the process behind how a neural network works in comparison to

  • The parameters set are very important and heavily influence accuracy
    • This can lead to discrimination and ethical issues, who is deciding what features a classifier should be trained to extract?


Andrej Karpathy, “What a Deep Neural Network Thinks About Your #selfie,” Andrej Karpathy Blog (blog), October 25, 2015,

Ethem Alpaydin, Machine Learning: The New AI. Cambridge, MA: The MIT Press, 2016.

Gehring, Jonas and Auli, Michael. “A novel approach to neural machine translation. ” Facebook AI Research. May 9, 2017.

Geoff Dougherty, Pattern Recognition and Classification: An Introduction (New York: Springer, 2012). Excerpt: Chaps. 1-2.

Martin Irvine, Introduction: Topics and Key Concepts of the Course (Presentation), 2019.

Peter Wegner, “Why Interaction Is More Powerful Than Algorithms.” Communications of the ACM 40, no. 5 (May 1, 1997): 80–91.


Reframing the AI Discourse

In the article Reframing AI Discourse, Johnson and Verdicchio claim, “because AI always performs tasks that serve human purposes and are part of human activities, AI should be understood as systems of computational components together with human behaviour (human actors), and institutional arrangements and meaning” (2017, p. 577). This is useful in reframing an understanding of AI as a system that includes the human actors involved. In this reframe, it takes away some of the mysticism created by popular media – that AI is autonomous and does not include human interaction or intervention. 

Autonomy of Computational Artefacts vs. Autonomy in Humans 

One of the AI myths prevalent in the readings is the fear that machines will outsmart us and therefore take control over humans in the future. The Johnson-Verdicchio article helps to clarify the definition of autonomy within computational artefacts. They say, “the less intervention needed by humans in its operation and the wider it’s scope of action, the more autonomous the artefact” (p. 580). Additionally, “the behaviour of computational artefacts is in the control of the humans that design them” (p. 584). Understanding that machines are not like humans in that they do not have their own interests and freewill to act as they choose, debunks the myth that machines or robots will decide to take over human life. The only “choice” machines have, are those that are progammed into their design. 

The real concern then – is a question of what kind of people are designing AI. What happens when the wrong people instruct AI to do harmful things or to control populations? AI used for military purposes is an example of an area that will need monitoring and has the potential to cause severe turmoil.

Given these findings, looking at the article “Who Will Win the Race for AI?” by Yuval Harari, he expresses concerns over the human actors in charge of data and autonomous weapons. Harari states that China and the United States are the leaders in data mining, and data could now be the most important resource in the world in terms of power of influence. He proposes that “the world could soon witness a new kind of colonialism – data colonialism- in which raw information is mined in numerous countries, processed mainly in the inertial hub, and then used to exercise control around the world” (Harari, 2019).

These questions come to mind: are autonomous weapon systems one of the real dangers concerning AI development? Are “dangerous emerging technologies” only dangerous because of the ones who will have access to them? Is data-colonialization a prediction or is there evidence that this will be at the core of the “AI Revolution”?

After reading Johnson and Verdicchio, Harari’s concerns seem to be much more on target. Further research into the current use of autonomous weapons and data mining will be necessary in unpacking the claims made in the Harari article. Additionally, human responsibility seems to be a better area for development and focus within current AI research and public discourse. Harari suggests that people need the tools to counter-monitor the government and large corporations against corruption and police brutality. He proposes that countries that will not be able to lead in AI development can invest their time and energy into regulating the superpowers to protect their data. 



Deborah G. Johnson and Mario Verdicchio, “Reframing AI Discourse,” Minds and Machines 27, no. 4 (December 1, 2017): 575–90

Harari, Yuval Noah. “Who Will Win the Race for AI?” Foreign Policy,

Margaret A. Boden, AI: Its Nature and Future (Oxford: Oxford University Press, 2016)