Ethics and socio-technical implications of AI or the broader technology we use has been a recent hot topic. However, although it is a “rising phenomenon”, I’d like to question even where and when we come across it. Documentaries such as Coded Bias and The Social Dilemma depict who are the few who are aware of the implications AI can have on our lives and livelihood as well as our rights as we have known them thus far. This course is an example of that; people in academia, people who work in the field such as coders, data scientists, etc., ‘tech activists’ and politicians specialized on privacy, tech and internet issues and legislature. But are the big companies concerned? Are the governments concerned? Who does this system benefit? 

Few weeks ago I was at the doctor’s and when asked what I study he turned around and said “Aren’t you scared?”. In that moment my first thought was what a bizarre question to get from a doctor. Scared of what? Robots taking over? Not necessarily. Machines taking over? Ehh, not necessarily still far away from that, plus humans still control what we create in terms of human vs robot. However, what is scary is the way in which the technologies we use today are embedded in human biases, racism, sexism, human rights issues and other societal implications. People assume that because we don’t see what is behind our technology, we don’t see the “blackbox” that governs how the technologies we use on a daily basis operate, who operate them and how. I think up until now it has been hard for people to understand why there are biases in algorithms or issues within the system. A lot of question that typically pop up are: Don’t more women work for tech companies? Don’t people of color work in the field nowadays? Don’t we have a more diverse workforce in the tech industry? Yes sure, is the simple answer (not as many as they should but let’s start with baby stems for the sake of this description). The problem is that that isn’t the root of the issue. I truly appreciated how well Coded Biases explains why and how these biases exist from the ground up. Simple starting with the Dartmouth conference in the summer of 1956 we see who were the creators of this initiative, of AI, of ML, white men. So the whole process, the whole system starts being fed with patterns that depict just that. Not necessarily in terms of what photos we fead the ‘machine’ for facial recognition, but concepts, theoretical and social representations were those depicted by a small pool of data. The data that you feed in the machine is what it will learn from so if most photos are of red flowers for example, the system has an easier time reading red typical flowers than lets say a white purple-dotted orchid. So the discord of AI began and picked up on that narrative alone. A machine doesn’t have a soul, doesn’t have its own thinking, it learns from what humans feed into it. It does what humans tell it to do. So who is scarier? Humans or the machine? 

Yes, it’s mostly based on a code. Yes there is more diversity in the tech industry and field of AI. But who writes the code? Or better yet, who had written all the code, today’s code is based off of. Last class, we talked about people in academia, such us ourselves, who question the technology we study, the technology we use and work on even though it may be our passion, it may be our interest. However, that is what CCT and other Interdisciplinary, ‘liberal art’ courses/programs are for. Studying what your interests are, but also question what that is. Don’t just look at it from one point. Nothing about technology is simple or lined up in one straight path. Maybe it’s the anthropologists in me but it’s hard to grasp that so many people in the overall field and not concerned about the implications of their actions. There is code so we hide behind it.  For the most part, computer scientists, software engineers, data scientists don’t come out of undergrad or their extremely software engineer concentrated master’s having discussed the big issues of the system, of the technologies we use today. Yes it is an amazing advancement of our days that has exponential improved our lives in more ways that we can probably think on the top of our head, but are the socio-political-economic repercussions and biases greater that what we have made them? 

We saw in the weeks of Cloud Computing and Big Data, the consequences or ideas of monopolizing an industry in which regular people and companies alike, rely on for the safekeeping and management of their documents, files and many more. The naiveness of people who assume that just because they don’t have their location settings on or they don’t use social media, that they also don’t have a technological footprint or a way for governments to monitor them. Unless you are 100% detached from everything technological, which honestly can be pretty hard these days especially because if you as an individual might be, something that you use such as a product or service might not be, one way or another data or information about you is out there. Big Data isn’t just random numbers and information collected from thousands of sources. Those numbers and figures didn’t magically appear out of nowhere. They are being fed into the system by our own usage of everything that we do. At the end of the day, we are the machines. Our societies are the technologies. They didn’t create themselves out of nowhere. And being skeptical and able to question what is really going on behind the scenes is what is going to help us conceptualize and overcome the many issues and implications that exist and are constantly happening, as depicted in Coded Bias. 

Resources 

Film Documentary, Coded Bias (Dir. Shalini Kantayya, 2020). Available on Netflix, and free to view on the PBS Independent Lens site

“Big Data”: ACM Ubiquity Symposium (2018).

Rob Kitchin, The Data Revolution: Big Data, Open Data, Data Infrastructures and Their Consequences. London; Thousand Oaks, CA: SAGE Publications, 2014. Excerpts.

Cathy O’Neil, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy (New York: Crown, 2016).

Boudewijn de Bruin and Luciano Floridi, “The Ethics of Cloud Computing,” Science and Engineering Ethics 23, no. 1 (February 1, 2017): 21–39.

Geoff Dougherty, Pattern Recognition and Classification: An Introduction (New York: Springer, 2012). Excerpt: Chaps. 1-2.

This weeks reading were definitely an interesting out look on what we mean by “big data” and re-defining not only the definitional but certain socio-economical theories and constitutes about it. For the most part, when people talk or generalize about “big data” it is this unfathomable, unquantifiable large amounts of information that we’re trying to categorize and clean-up for x and y reason for x and y company. Although this isn’t technically wrong, the authors and sources we looked at especially this week (but also can of course connect it to previous weeks and greatly falls right after last week’s cloud computing theme) give a different perspective on deeper issues and concepts that immediately surround and relate to the accumulation, understanding, meaning and use, purposeful or accidental, of the countless of data that is collected every second of every day. As Rob Kitchin puts it:

“Big Data creates a radical shift in how we think about research …. [It offers] a profound change at the levels of epistemology and ethics. Big Data reframes key questions about the constitution of knowledge, the processes of research, how we should engage with information, and the nature and the categorization of reality … Big Data stakes out new terrains of objects, methods of knowing, and definitions of social life. (boyd and Crawford, 2012)” (Kitchin, 2014, 1). 

In a more data science perspective, big data works to analyze, extract information, deal with large and complex amounts of data that cannot be deal with or processed through “every-day” software that we use on the go, as it also is sufficient enough for the fast pace lifestyles that most of us lead. Basically, accumulating unstructured data that needs to be filtered and categorize in order to serve a purpose. But what we are calling on here, is the realization of how much more complicated “big data” really is and the fact that in reality, “bit data” affect our lives way more than we actually realize and play a “prominent role in every aspect” of it (Huberman, 2017). The way we choose to live our lives nowadays, is a life that in a way is constantly interloped with technology (smart phones, emails, social media & networks, credit cards, smart home devices, cars, laptops, etc.) where knowingly or unknowingly we are constantly feeding back the system with so much information about us, what we do, where we are, what she buy, eat, drink, listen to, that in return we get a very personalized portfolio if you may, that matches our preferences, hobbies, lifestyles, etc. We get personalized ads, personalized feeds and more because of this mass accumulation of data that is taking place on a much larger scale than it did even 5-10 years ago since not only has the accumulation of different devices per individual increased but our lives on the internet have also developed on such an exponentially fast paced trajectory. Imagine how many people around the world are constantly “feeding” the cloud or companies with data and information that then has to be analyzed, categorized and set to its respective path to only be processed by companies and then be fed back to us in more disguised and discrete forms, one of those mainly being advertisements.

Johnson and Denning (2017) also emphasize this huge “big data revolution” as a result of “the expansion of the internet into billions of computing devices, and the digitization of almost everything. […] Digitization creates digital representations for many things once thought to be beyond the reach of computing technology”. And this exactly explains how much “big data” truly affect all aspects of life that not only have the ability to personalize ads but also indicate yet again how globalized this world has become because of the constant development of technology. For example, especially during this pandemic, we saw the importance of online education something that would have been never imagined or accepted years ago. The fact that so many children, people, educators, students, etc. all over the world are able to log onto to platforms from wherever they are for hours at a time while also being able to record, participate, interact and do so much more while getting an education can be attributed the the capacity of technology to support such activities and not only maintain them as they are happening live but also save them for future use. Even the cities we live in, accumulate countless of data over processes we most likely don’t even assume provide data yet it is truly hard to not be digitized nowadays, otherwise the difficulties and setbacks that can arise with being “disconnected” or not apart of the world. Transportation services in cities whether smart app rides Uber, Lyft, etc. who collect data, so does public transportation such as buses to monitor the amount of people who use them, to plan out routes, get informed on best possible routes, traffic accidents and more. CCTV and other security systems are constantly monitoring, recording and collecting information many times on the spot analyzing potential threats or issues. Of course, socio-political and economic issues are ultimately affected by the development and evolution of technology in all aspects of life. Examples of this are economic crisis that never just affect one entity or one country or one company but the whole system, wars and political disputes do not stay limited within borders or zones but expand into other circumstances and cross borders as people migrate, seek refuge, change status, etc. 

Resources

Bernardo A. Huberman, “Big Data and the Attention Economy” Ubiquity 2017, (December 2017): 2:1–2:7.

Jeffrey Johnson, Peter Denning, et al., “Big Data: Big Data or Big Brother? That Is the Question Now (Concluding Statement),” Ubiquity 2018, no. August (August 2018): 2:1–2:10.

Jeffrey Johnson, Peter Denning, et al., Big Data, Digitization, and Social Change (Opening Statement), Ubiquity 2017, (December 2017).

Rob Kitchin, “Big Data, New Epistemologies and Paradigm Shifts,” Big Data & Society 1, no. 1 (January 1, 2014): 1-12.

Rob Kitchin, The Data Revolution: Big Data, Open Data, Data Infrastructures and Their Consequences. London; Thousand Oaks, CA: SAGE Publications, 2014. Excerpts.

Wikipedia, “Big Data“

We have previously discussed the socio-ethical impact, biases of algorithms and AI yet focusing on the companies that control it all, takes a different turn on the overall implications that cloud computing can potentially have assuming that it converted into this “unified” architecture controlled by the “big four”; Google, IBM, Microsoft, Amazon. As de Bruin and Floridi (2017) explain, Generation X and Millennials seem to “care less about” where their private information is saved or who controls it but rather care more about the efficiency with which they can use the that information, for example, a photo and send it to friends, alter it, post it or upload it, share between devices, etc. I think overall there has been a larger impact of cloud computing in our daily lives exactly because it is so easy and efficient and allows us to have all our information, documents, material, etc. stored and located in one place and accessible from multiple locations. We definitely rely on the efficiency of it all, more than what we actually question the overall concept of what is happening to our material when we do upload/save them on “the cloud”. However, there is quite a difference when the responsibility is split up among more and smaller sized companies and therefore you also get more services and products to choose from that could possibly represent your needs or preferences better. 

AWS defines cloud computing as: “the on demand delivery of IT resources via the internet” where you can access any tech services on an as-needed basis, you can use it for data backup, disaster recovery, email services, sharing virtual desktops, big data analytics, customer-facing web applications. It can also be used for personalized treatment of patents, fraud detection for finance companies, provide online games for millions of people/players around the world and more. Basically, your private or work information is somewhere on “the cloud” and even though it might seem private on your end, as the user who could similarly be accessing let’s say a notebook or a vault, I doubt most of us read the fine prints, where more details of who has ownership and what can happen to the material and data is disclosed and explained. “Cloud computing suits the interests and values of those who adopt a deflated view of the value of ownership and an inflated view of freedom” (de Bruin & Floridi, 2017, 22). In Cloud Computing (2016), Ruparelia mentions the three things that most us look for in these services and are usually what make us choose the company/product/service we will go with whether for personal use or business purposes. Integrity and reputation, you want to be able to rely on the product, the company and the service. Which means most of us will only trust companies who not only we know because they are famous but because many others also use. It is more likely that we will trust a brand we have heard of and know can be well supported. Another important factor that goes into choosing your preferred cloud provider would be the benefits you get from cloud computing such as efficiency and promptness. Having the ability to access anything you want from anywhere in the world at any time while knowing that it is in “good hands”, safe and reliable but also in a sense never disappoints. Forgetting your hard-drive or usb stick for example can lead to major issues especially if you desperately need your data in that moment, but with cloud computing that is not something to worry about since you know whatever is on there will always be on there. Finally, pricing of each product or service plays a crucial role in the selection of a cloud computing company that would supposedly match its pricing to what it has to offer in terms of space, accessibility, security, usage, organization and more. 

Moving cloud computing into a “unified architecture” provided only by Amazon Web Services, Apple, Microsoft and IBM would be imaging a different style of data storage, accessibility, manipulation and distribution of content. Ruparelia’s (2016) given characteristics of what we look for in our cloud computing services definitely become more conscience and give you less choices to choose from. Of course this also implies the power of everything we refer to as “the cloud” to be kept among the four companies giving them more flexibility to control the data and information that is uploaded on them? What happens to those fine prints on the terms and services agreements? Where does the date ownership go or rather who does the data really belong to? What about pricing? Would it become more expensive since choices are more limited and we won’t really have more options or would they actually be more beneficial and “get-what-you-payed-for” situation? Does having all of the words cloud split among only four companies and “kept in one place”  make it riskier and more exposed to outside threats? Or does it increase security since the “big four” could potentially work together to provide a “united front” again outside threat or even cultivate healthy competition amongst them therefore creating stronger systems and walls. Floridi and de Bruin (2015) discuss the power of “interfluency” when it comes to ethically effective communication among companies but also between companies and their customers. If the big four hosting companies have the same information or share a large part of it, they should therefore  by able to “provide and seek information about relevant issues such as consumer privacy, reliability of services, data mining and data ownership” (22). The two authors also discuss  the possibility of a stricter government regulation and overall involvement in cloud computing and what that could mean in terms of restriction of the use of cloud computing, regulating what we can and cannot upload, share, distribute, etc. and what its future will look like. 

Resources

• AWS, What is Cloud Computing?
• Boudewijn de Bruin and Luciano Floridi, “The Ethics of Cloud Computing,” Science and Engineering Ethics 23, no. 1 (February 1, 2017): 21–39.
• Cloud Computing Services Models – IaaS PaaS SaaS Explained (EcoCourse)
• Derrick Roundtree and Ileana Castrillo. The Basics of Cloud Computing: Understanding the Fundamentals of Cloud Computing in Theory and Practice. Amsterdam; Boston: Syngress / Elsevier, 2014.
• Nayan B. Ruparelia. (2016) Cloud Computing. Cambridge, MA: MIT Press. 

Over the years, in one way or another we have all seen the existence and implementation of Artificial Intelligence take over so many aspects of our lives. For the most part, we don’t even see or realize that there is some sort/form of AI being implemented and used for a specific thing, products, circumstance. However, when we do notice we should also be recognizing the countesses instances where the biases and prejudices that existence within AI, are very much so present. I think because it is something not tangible or so obsolete, we assume that as an “electronic” being there is no association between AI and the societal issues that exist in our societies today. But the AI hasn’t coded itself. It hasn’t necessarily created itself out of nowhere. And so as us, the humans, “manufacture”, code and establish these presences in our lives, unfortunately or fortunately (since we know what is wrong and work towards fixing it) with it any bias of prejudice that is embedded in our human minds and lives will get encoded with it. Apart from those, relying so heavily on produced intelligence can have its own ethical implications and issues. 

With data collection being a huge part of our electronic and digital presence, most times we’re not even aware of that taking place. We’re not always really sure or aware of how and when data is being collected and what is being done with it. But if there is one thing I have realized is that data is constantly collected. Some of our readings had this as two separate ethical issues that govern AI but I feel like they are pretty similar and interchangeable and that is the autonomy and power that we rely on our AIs. The automation that goes behind AI causes “individuals [to] experience a loss of control over their lives” (Anderson & Rainie, 2018). “When we adopt AI and its smart agency, we willingly cede some of our decision-making power to
technological artefacts (Floridi & Cowel, 2019, 7). Partially, this is because the deblack-boxing of AI is still very much so in the box. However, there is a question to be posed here; will we truly ever learn what is behind the AI that we use on a daily basis? Will these companies/products ever truly reveal how they work, what they really do with all this information and data collection? Honestly, probably not since that would make them weaker to competitors. Unless, more people start realizing, noticing and want some change in terms of the control and power they have towards their use of this type of technology. As Nemitz, also explains, large (tech) corporation have taken over every aspect our of lives whether or not we realize it or sign up for it. “The pervasiveness of these corporations is thus a reality not only in technical terms, but also
with regard to resource attribution and societal matters” (Nemitz, 2018, 3). All these companies and brands, have basically collected in “their hands” countless information and data that with it they basically are able to control so many aspects of the human life especially in terms of technological, economic and political power that has been given to them through this digital power. Since nowadays, we rely so heavily on technological and the use of a digitized framework, most aspects of human life are also controlled by technology. So in a way whoever is more “ahead of the game” in the field, is the one who also has the power, the information, the data. Everyone else has pretty much lost their ability to pick and choose when, how, where they share information. It is one way or the other. If you want to have any sort of digital presence, talk on the phone, use your credit car, pay for something, look up something, everything you do is pretty much tracked down and collected, formed into a bigger overall ‘picture’. 

Another ethical issue/implication of AI, is of course the idea that all this information and data can be used for the destruction and with malicious intent towards others. Apart from ” autonomous military applications and the use of weaponized information” (Anderson & Rainie, 2018) we can also speak on the collection of information aimed towards capturing people such as facial recognition. The problem here is who is using this technology? and for what reasons? Of course, we also have to consider yet again the biases that go into this type of “vigilantism”. Racists views and opinions definitely influence who this type of technology can be geared at and who are going to be the people mostly being targeted by it. Floridi et al, also explain this in terms of how “developers may train predictive policing software on policing data that contains deeply ingrained prejudices. When discrimination affects arrest rates, it becomes embedded
in prosecution data. Such biases may cause discriminatory decisions (e.g., warnings or arrests) that feed back into the increasingly biased datasets, thereby completing a vicious cycle” (Floridi et al., 2019, 1788). 

How do we apply laws/regulations/safety measures for something so widely used? 

We have seen how hard it has been to manage data privacy uses and laws from one country to another, how can something so universal become so specific when it comes to protecting people? 

References 

Janna Anderson, Lee Rainie, and Alex Luchsinger, “Artificial Intelligence and the Future of Humans,” Pew Research Center: Internet and Technology, December 10, 2018.

Karen Hao, “In 2020, Let’s Stop AI Ethics-Washing and Actually Do Something,” MIT Technology Review, December 27, 2019.

Karen Hao, “Establishing an AI Code of Ethics Will Be Harder Than People Think,” MIT Technology Review, October 21, 2018. 

Karen Hao, “This Is How AI Bias Really Happens — and Why It’s so Hard to Fix,” MIT Technology Review, February 4, 2019.

Luciano Floridi and Josh Cowls, “A Unified Framework of Five Principles for AI in Society,” Harvard Data Science Review 1, no. 1

Luciano Floridi, Josh Cowls, et al., “How to Design AI for Social Good: Seven Essential Factors,” Science and Engineering Ethics, 26/3, 2020: 1771-1796.

Paul Nemitz, “Constitutional Democracy and Technology in the Age of Artificial Intelligence,” Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 376, no. 2133 (November 28, 2018): 20180089

This was yet again an exciting week for me as we focused more on IPAs and that is what I have been mostly focusing on since coming to CCT* as I wanted to continue what I was learning and working on during my undergrad years studying and analyzing IPAs! From studying IPAs and Alexa coming out during my undergrad times, I can’t say that there wasn’t an uneasiness of some sort surrounding this topic. Having a device in the same room as you that can constantly hear you or record you and of course keeps track of info and data, can sound pretty scary especially as a new advancement. So I tended to stay away from having my own device (not including Siri in this – because I did use Siri beforehand) yet still found the concept extremely intriguing. For me, it is the closest thing that we have to a human-like-robot as part of our daily lives. When I moved to DC a family friend gave me a Google Home as a house warming gift so we could share our photos with each other as it displays them on the screen. Since it was a gift I kept it and gave it a try and have been using it since then which is why I decided to focus on Google Assistant for this post. 

Google Assistant comes in a mobile and smart home (Google Home) versions and was initially debuted in 2016. The assistant interacts through human voice and dialogue and provides results or executes commands based on the users verbal demands. 

*There is a research paper(s) that goes along with this, please feel free to reach out if you’re interested! 

If not all of us, most of have definitely used Google translate at some point in our lives. I don’t doubt that some instances have been successful but I bet most have unfortunately not. The reason behind this is because natural human language is such a complex system of labyrinths that sometimes can only truly take place in the human mind as we can understand and interpret context, meaning and overall situational understanding that comes hand in hand with language. This is something that computers and machines have not been able to perfectly achieve yet as assigning cultural, interpretations, etc. is extremely challenging for a machine and if we think about it even for humans as often times not being a native and have grown up in the country of the language, you miss a lot of cultural nuances, interpretations, signs, etc. As Thierry Poibeau (2017) explains; “Natural language processing is difficult because, by default, computers do not have any knowledge of what a language is. […] There is a dynamic co-construction of interpretation in the brain that is absolutely natural and unconscious” (Poibeau, 2017, 23-26). However, this challenge is a highlight and key point in today’s technology and leads the path for further advancements as nothing can be done without language. So how does Google Translate work and why is not as reliable often times (aka why does the translation never make perfect sense in the final language)? 

Natural Language Processing and Machine Learning play a crucial role in how Google translate, IPAs and many more computers and systems are able to read, interpret, understand and emulate a sentence whether phonetically or in the context of translation in order to fit natural language and real human dialogue standards. NLP is “concerned with the interactions between computers and human language. How to program computers to process and analyze large amounts of natural language data.” (Wikipedia, 2021) It is needed and used to understand the context and contextual nuances to which I was referring above. Computational linguistics also play a role in this “decoding” (pun intended) process and are “concerned with the computational modeling of natural language, the study of appropriate computational approaches to linguistic questions” (Wikipedia, 2021). One of the fundamental problems in NLP was sentence deconstruction and the ability for a computer to break down the sentence into bite-size pieces in order for it to be processed more easily. Of course, it also has to follow phrase structure rules in order to encapsulate the grammar and meaning of a language. As we have seen from this weeks and previous weeks readings: parse tress, neural networks, natural language understanding (how we get meaning out of combination of letters) and natural language generation (how to generate language from knowledge, distributional semantics (make the machine/computer guess words that have similar meaning but seeing which words appear in the same sentence a lot), count vectors (the number of times a word appears in the same place or sentence as other common words), etc. all build up a system where all this data is pulled and “stored in a web of semantic information where entities are linked to one another in meaningful relationships and informational sentences” (CrachCourse #7, 2019; CrashCourse #36, 2017). 

Google Translate uses Machine Translation (constituted by all of the above and more): “the idea is to give the most accurate translation of everyday texts. […] The ultimate goal is to obtain quality of translation equivalent to that of a human being.” (Poibeau, 2017, 14-21). Google Translate, relies on NN as they look at thousands and thousands of examples to give the best solution/result. By using the Encoder-Decoder model, MT “builds internal representions and generates predictions. The encoder (a model that can read in the input aka the sentence) stands for “what we should think and remember about what we just read” and the decoder “decides what we want to say or do” (CrachCourse #7, 2019). The NN converts the words into a form, numbers, vectors and matrices that the computer understands. Recurrent Neural Networks, have a loop that allows them to “reuse single hidden layer, which gets updated as the model reads on at a time and by training the model on which to predict next, the model waits for the encoder RNN and the decoder prediction layer. The RNN are long short term memory RNNs (LSTM-RNNs) that can deal with longer sentences (instead of just words) much better (CS Dojo Community, 2019). By doing this consistently, if the computer notices that the two words mean something similar, the model makes their vectors more similar and therefore can “predict” the word that will follow the next time it is asked to do so (CrachCourse #7, 2019). The E-D model, takes the words/sentences the RNN turns them into Vector (sequence to vector) and the decoder takes the vectors and the RNN turns it into the words of the other language (vector to sequence) (CS Dojo Community, 2019) However, this fails to address the complexity that comes with contextual meaning and understanding and can be limiting when we are dealing with longer sentences that have more than 15-20 words. Just because words have similar meanings doesn’t mean that they can necessarily be interchangeable in all contexts. (See blog post)

The solution that Google Translate has “come up with” is by replacing RNN with BidirectionalRNN which uses an attention mechanism between the encoder and the decoder and helps the computer know which words to focus on while generating the words for another language (CS Dojo Community, 2019). During the translation process, a – lets say- English sentence is “fed into” the encoder, “translated” (again, pun intended) into a vector which is then taken my the attention mechanism that correlates which -lets say- Greek word will be generated by which English words. The decoder will then generate the result of the translation (in Greek) by focusing on one word at a time as the words have been determined by the attention mechanism. In this specific case, Google translate actually uses 8 LSTM because “deeper networks help better model complex problems” and are “more capable of understanding the semantics of language and grammar” (CS Dojo Community, 2019). 

What does this data look like? Is it saved as words or as vectors? Are knowledge graphs shared across any type of machine/computer/software i.e. does google translate share its data collection with others? 

References 

Blog post from Prof. Irvine’s class 711: Computing and the Meaning of Code (Ironically enough I though of the same title for both of these, i.e. “take 2” on this one’s title) 

Crash Course Computer Science: Natural Language Processing (PBS). 

Crash Course AI, 7: Natural Language Processing

Thierry Poibeau, Machine Translation (Cambridge, MA: MIT Press, 2017).

How Google Translate Works: The Machine Learning Algorithm Explained (Code Emporium).

Wikipedia: Machine Translation,

Wikipedia: Computational Linguistics 

Wikipedia: Natural Language Processing 

This was such an interesting topic to further dive into not only because it perfectly explains what I’d describe as one of today’s multi-used yet still “black-boxed” phenomenon of pattern recognition especially as it is applied to computer vision and images. Karpathy’s article doesn’t only highlight and break down the functions and uses of Convolutional Neural Networks but he has managed to depict through his findings how something so computerized can still be very much so human in terms of the societal biases it brings into play. 

“In machine learning, the aim is to fit a model to the data”, explains Alpaydin (Alpaydig, 2016, 58). Computer don’t just know what to do. Someone, a human, has to feed them with  directions and instructions in order for them to actually do something. The computer will follow whatever set of instructions is made for it by the human and execute the commands it has to. This means, that this human that has all of their opinions, biases, beliefs, experience, etc. is to encode into a non-human thing, the ability to execute commands based on human characteristics, capabilities, ways of knowing and understandings. Karpathy’s “experiment” shows exactly how there is biases in algorithms, especially his own, a topic I really looked into during my undergrad (one of Dr. Sample’s very looked into topic) and through my research on uses of ML and NLP in IPAs focusing on speech, language recognition and more. 

Karapthy explains how “a ConvNet is a large collection of filters that are applied on top of each other”. In these convolutional neural networks, “an aritifical neuron, which is the building block of a neural networks takes a series of inputs and multiplies each by a specified weight/number/characteristic and then sums those values all together (CrashCourse, #35, 2017)  To break it down, artificial neural networks, have artificial neurons that basically rake numbers in and spit more numbers out (CrashCourse, #34, 2017). You have the input layer, the hidden layers and the output layer. In the hidden layers, is pretty much where it all happens. It is where the computer sums the weighted inputs, the biases are applied and the activation function is also applied as this is computed for all the neurons in each layer (CrashCourse, #34, 2017). The deeper the neural net, the “weaker” or “stronger” the AI is. The NN can learn to find their own useful kernels/inputs and learn from those. The same was the ConvNets use stored information, banks of these neurons to process image data. 

As you run through them, convolutions happen over and over again as they run small filters and slide them over the image spatially to dig through the different layers of an image in order to find different features. This operation is repeated over and over again “detecting more and more complex visual patters until the last set of filters is computing the probability of entire visual classes in the image” (Karpathy, 2015). This is the part where we have come in and told the AI how to use these filters, when, where, what do we want out of the, etc. We train the ConvNets to know what to keep and what to emit by telling it what is in a way, good or bad, pretty or ugly, etc. Practice makes perfect, is a great saying to apply here as these neural networks will learn through re-inforced learning and by trial and error. The more data points you have the more information you can collect, which means that the more data you have the less uncertainty you also have about the classification, layering and choices made. However, since not all data points are equal and can’t be measure appropriately, the ML model can identify where the highest uncertainty is and ask the human to label the example, learning from those. Through active learning, the model is constantly synthesizing new inputs creating layer after layer until it reaches the wanted result and outcome (Dougherty, 2013, 3). 

For face recognition, the input layer is the image captures which is stored as pixels, defined by a color and stored as a combination of three additive primary colors, RGB, as we saw in our previous lessons as well. (Alpaydin, 2016, 65; CrashCourse #35). With biometrics we then get the ability to recognize and authenticate people by using their characteristics both behavioral and physiological. Of course, this also helps with training computers to recognize mood and emotions and not just one’s identity which trains them to learn, pick up and adapt to a human’s or their user’s mood and feelings.  

During the classification process, during the segmentation and labelling part, the image is separated into regions that are used for each particular task. The foreground which entails the objects of interest and the background which is everything else that will be disregarded. Labelling the objects then comes into play which obviously makes it easier for future use to immediately categorize or extract whatever needed from an image but ironically, we can even say that labelling in many cases, is foreshadowing the biases that can be found in algorithms. The following feature extraction is when characteristic properties of the objects come into play and distinguishes them/places them in a different category from objects they either share similarities or differences with and so forth… (Dougherty, 2013, 4-13). Further playing a role and testifying to how biases are created even in tech exactly because it is basically a reflection of societal biases, issues and human systems of classification. 

I couldn’t stop thinking about how much Karpathy’s experiment reminded me of how if a few years ago (and by few I mean even 2-3 years ago) if you Googled “beautiful girls” for a few scrolls the only photos with be those of generic (pun-intended – “generated”) white women because the algorithms identified as beautiful (honestly, not much has changed now either). A computer doesn’t know what is “pretty”, “ugly”, “good”, “evil”. Humans have inputed and labelled recognizable patters and standards of beauty further bringing to the surfaces, the racisms and biases that are very much so present in our world but also the underrepresentation of minority groups and BIPOC in tech. Even in Karpathy’s results, one can see the obvious majority of who are in these selfies. 

Best 100 out of 50,000 selfies, as judged by the Convolutional Neural Network. https://karpathy.github.io/2015/10/25/selfie/The cimputer doesn;t 

Based on his explanations of what was categorized as a good and bad image, I’d definitely would like to ask him what and how those distinctions where made. Also, how is a selfie of Ellie Goulding (famous singer) there if he supposedly through out and separated photos with either too many or not enough likes compared to others and people with too many or not enough followers as others? 

Based on his worst selfies, one of the criteria is “low lighting”, however, is it just the low lighting that is categorized as bas or is dark skin also included in that? “Darker photos (which usually include much more noise as well) are ranked very low”.  This also speaks to the issue of Snapchat or instagram filters and their inability to pinpoint and find features on people with darker skin in order to apply the filter on them. 

P.S. Check back in the future for a more updated list on cool articles and readings about biases in algorithms! Need to do some digging and go through my saved material and notes from previous years! 

References

Ethem Alpaydin, Machine Learning: The New AI. Cambridge, MA: The MIT Press, 2016.

Crash Course Computer Science, no. 34: Machine Learning & Artificial Intelligence

Crash Course Computer Science, no. 35: Computer Vision

Crash Course AI, no. 5: Training an AI to Read Your Handwriting

Geoff Dougherty, Pattern Recognition and Classification: An Introduction (New York: Springer, 2012). Excerpt: Chaps. 1-2.

Andrej Karpathy, “What a Deep Neural Network Thinks About Your #selfie,” Andrej Karpathy Blog (blog), October 25, 2015, https://karpathy.github.io/2015/10/25/selfie/.

Before diving into the more complex attributes of exactly how it is that you are able to read these letters, words and phrases that I am typing out on my computer right now – aka these “data types, we can briefly describe what we commonly understand by “data”, at least in the context of computing and coding. Professor Irvine defines; “‘Data’, in our contexts of computing and information, is always something with humanly imposed structure, that is, an interpretable unit of some kind understood as an instance of a general type. […] (T)o be interpretable, [data] is something that can be named, classified, sorted, and be given logical predicates or labels (attributes, qualities, properties, and relations)” (Irvine, 2021, 1). As we briefly touched upon in our last class as well, a “token” can stand for something else, something that can be represented as something, something that is immediately related and connected to something else. Data can be a token or tokens. “Data is inseparable from the concept of representation” (Irvine, 2021, 2). Data alone would not stand for anything if it didn’t actually represent something. Focusing on this context of computing and information, representation means a “computable structure” and is “defined byte sequences capable of being assigned to digital memory and interpreted by whatever software layer or process corresponds to the type of representation — text character(s), types of numbers, binary arrays, etc”. And simply put, we can also say that this is why “data” is considered to be of a more ‘higher esteem’ than “information”. I Imagine information as being the biggest mass of general, undefined, ‘unsupervised’, facts, clues, concepts, etc, and no matter what it is it can just exist, it can ‘tell’ us something, it can let us know of something but it doesn’t have the purposefully structure nature and meaningful existential representation that “data” consists of. 

A part of this data, are the data types we all know as texts, images, sounds, etc. If I send the words “Hello! How are you?” from my iPhone to someone with a Samsung, they will receive letter by letter, symbol by symbol, the same thing. If I copy and paste the same phrase from WordPress, to my notes, to a personal message on Facebook, to someone else on Whatsapp, to a chat room on Twitch, etc., the same exact message will appear once again. The reason for this is Unicode. Unicode is the international standard, an “information technology standar (Wikipedia, 2021), that has been creates and formatted in order for all computing devices and software applications to interpret the same representation throughout the world. “Unicode is thus the universal internal language of computing” (Irvine 2021, 5). It is the data for representing written characters aka strings, “of any language by specifying a code range for a language family and a standard bytecode definition for each character in the language” (Irvine, 2021, 3). The reason why we are able to read text on any device, emails, messages, etc, is because of Unicode. “Unicode is what is inside the black boxes of our devices with pixel-based screens and software layers designed for text characters and emoji” (Irvine, 2021, 5). Joe Becker, in August of 1988 in his draft proposal for this character encoding system explains why even the name matches as it is “intended to suggest a unique, unified, universal encoding” (Wikipedia, 2021). 

Some fun/interesting facts about Unicode (Wikipedia 2021; Wisdom, 2021):

• total of 143,859 characters 
• 143,696 graphic characters 
• 163 format characters 
• 154 modern and historic scripts, symbols, emojis 
• current version: 13.0  
• Unicode cannot run out of space. If it were linear, we would run out in 2140 AD! 

The reason why Unicode works as a standard for everyone is because these data standards cannot align with a specific system, software, platform, etc., but needs to be “independent of any software context designed to use them. They can work with any software or device because they reference bytecode units, which are independent data. “What we see on our screens is a  software ‘projection’ of the bytecode interpreted in whatever selected font style for ‘rendering’ the pixel pattern on screens” (Irvine, 2021, 4). How it comes all together is with the aid of The Unicode Standard which uses code charts for the visual representation, encoding methods, standard character encodings, reference data files, character properties and rules, etc., “provides a unique number for every character, not matter what platform, device, application or language” (Unicode Technical Site, 2021). Which if you think about it, it is pretty cool that we were all able to agree on something (of course, without getting into the complications, issues, biases, etc. that come with adopting Unicode), but in a cliche way – technology did bring us (almost) all together! For text processing, unicode translates to a unique code point, a number, for each character so it represents the character in a more general computing format where the visual representation of it, i.e. font, shape, size, etc., is taken care of by different software, unicode provides the meaning, the what it is (Unicode Technical Site, 2021).

Unicode used different types of character encodings, the Unicode Transformation Format (UTF), “an algorithm mapping from every Unicode code point […] to a unique byte sequence (the ISO/IEC 10646 standard uses the term “UCS transformation format)” (Unicode Technical Site, 2021). Most commonly used are UTF-8, UTF-16, UTF-32. UTF-8 is the byte-oriented encoding form, it is the dominant encoding used on the World Wide Web and even the first 128 characters are ASCII (American Standard Code for Information Interchange) which means that they also are under UTF-8 (Unicode Technical Site, 2021; Wikipedia; 2021). 

Crash Course Computer Science 

Since emojis have to be bytecode definitions to be interpreted as software context, All emojis must have Unicode byte definitions in order to work for all devices, software, graphic rendering. Updates with new emojis or some emojies will not be consistent from one device to the next or from one system to the next, i.e. sometimes the typical red emoji heart ❤️ (note: the irony that this heart emoji looks different on my iOS system than on WordPress) would show up as a smaller dark black heart or maybe a box with a “?” would appear in that emojis place if you hadn’t updated the version, etc., Is this due to non-updated bytecode definitions? Or is it because the software/system didn’t use/follow the ISO/IEC standards? Is this the reason why each company/program/software has their own “look”/style for each emoji because that is how it transforms/translates into from the Unicode CLDR data? Does the same apply for unreadable fonts as mentioned in the readings with the problem that arises with Unicode? 

I’d like to further look into the connection between Unicode and software stack design? How do they connect to each other and how does one symbol go through the “journey” of Unicode to the process of adopting whatever font, size, color it is given. 

References 

Irvine, Martin. (2021). “Introduction to Data Concepts and Database Systems.”

Crash Course Computer Science 

John D. Kelleher and Brendan Tierney, Data Science (Cambridge, Massachusetts: The MIT Press, 2018). 

Unicode Wikipedia 

The Unicode Standard 

Emoji – Unicode 

Unicode is Awesome – Wisdom 

A re-occurring theme/discussion point through out our conversations and readings, has been the concept behind de-black-boxing tech and the reason behind that is probably due to the fact that we struggle to truly grasp what is behind these machines and how do we really end up getting the “information” that we do in the formats that we do. How does an image get so easily captured and transferred from reality to the initial shutter of the specific camera that took the photo, to a digitalized form that can be re-used for multiple purposes, over an infinite time period in a gazillion different ways. You can send it, post it, delete it, put a filter on it, put a bunch of filters on it and photoshop it. No matter what you’re doing, you have managed to take something that exists in the physical real world and converted it to its digitized representation and depiction aka form. 

“E-information theory supports our
whole symbolic-cognitive system by providing the techniques for using electrical
energy quantities in time for physical substrates of all human sign and symbol
structures (data representations)” (Irvine, 2020, 4). We are basically trying to imprint our human and cultural understandings, meanings and values into an electronic format where automated calculations and interpretations can take place in order to take the information that has been fed into the machine or system and “spit out” new representations and meanings that yet have an overall meaning used for communication and interpretation. Physical electronics and machines such as our computers or smartphones have been created in order to present and hold, in a way “carry” these data representations that have been translated from human symbol meanings to bits and bytes, aka data. (“Data: Binary structure for units that can be assigned a first level of meaning” (Irvine, 2020, 3). Information theory is considered to be what we call “designer electronics” as it represents, processes and transmits designed binary electronics (what we have fed into the computer and converted it into the “language” it understands) as an “engineering solution to a semiotic problem” that of “imposing human logical and values on electronic and physical media for creating meaningful patterns” (Irvine, 2020, 2). 

Indicating a state of existence, the signal-code-transmission model does not replace the meaning of whatever the representation is, stands for. “Meanings are not properties of electrical structure” (Irvine, 2020, 9). We can understand and conceptualize the message that we have received through the transmission because of the pre-supposed meaning or notion we have attributed to it. The same way a color blind personal will view the color blue on his screen differently than a person who isn’t colorblind. However, the system itself does not make distinctions between the two blues, those are made solemnly by the person at the receiving end who conceptualize and see the color differently. The transmission just gets the job done of “carrying” the “message”/transmission from one designer electronic to the next. As Shannon also explains in his theories and findings, that “what a relay passes onward from
one circuit to the next is not really electricity but rather a fact: the fact of
whether the circuit is open or closed” (Gleick, 2011, 18). 

Whatever is transmitted is what will be shown. The system/computer does not have actual concepts or true understanding of what is happening, said, painted, shown, depicted, stated, etc. on the receiving end of a signal. The system/computer have zero pre-supposed cultural notions or any sort of conceptual belief about the E-information, that is up to the user, the real human. Leading to why the information theory model works for everything digital and electronic but “not extending to models for meanings, uses, and purposes of our sign and symbol systems” since that is ultimately left to the task of the human user. In reality, information isn’t something tangible, we can’t see it and most times we cannot touch it, especially if we are talking about E-information (Irvine, 2020). As long as the system runs and works properly what we see are “the effects of information shaped and “communicable” for internal processing in the systems” (Irvine, 2020, 13) that take place in seconds as the order come from binary switches and other processes that do not contain any further meaning other than on/off, 1/0, etc. The meaning-making comes in a later stage that takes place in our human brains that can unfortunately or fortunately, not work as fast as a computer’s processor and systems and therefore need time to process, conceptualize and interpret meaning found in the data using our own human symbol systems and cultural notions. 

Sources

Gleick, J. (2011). The Information: A History, a Theory, a Flood. Bantheon Books: NY

Irvine, M. (2021). Introducing Information Theory: The Context of Electrical Signals Engineering and Digital Encoding.

Martell, C & Denning, P (2015). Great Principles of Computing. The MIT Press: London

A big part of “deblackboxing” the mystery behind computer is realizing that there isn’t something necessarily to de-blackbox. The gap between true understanding of computing and what goes on behind ‘closed screens’ is the complex arbitrary notions that we ourselves have given to something we have actually created. The truth behind it, is that it is not as unfamiliar as we think. If we think about it, all these designs, systems, software didn’t create themselves. Someone had to build them based on human knowledge, needs, desires, etc. In reality, they are only a reflection of our human day-to-day functions that we encoded to make our lives easier and faster, or at least that is the goal without including all the ethical, security, privacy, etc. issues that have rises over the years. “The action of computing comes from people, not principles” (Denning & Martel, 2015, 19). However, breaking down and highlighting these subparts of computing and systems in order to understand the information process and algorithms that guide them towards executing specific commands and demands. We use design structures and principles of computing to transform information, discover it, classify it, store it and communicate it, these “structures are not just descriptive, they are generative” (Denning & Martel, 2015, 15). The countless masses of information whether physical, digital or even conceptional have been overwhelmingly growing through out the years and scientists, coders, etc. have needed to find different and more sufficient ways to manage such matters but also “build systems that could take over human cognitive work” (Denning & Martel, 2015, 27) and as Morse had suggested; to “construct a system of signs which intelligence could be instantaneously transmitted” (Irvine, 2020, video). 

Digging into what are these main concepts helps us realize that in reality computing and this black box isn’t so dark and mysterious after all. A simple duo of numbers, 1 and 0 have managed to create such a vast system of knowledge, storage and processing of information that have ultimately changed life as we know it forever. For example, just as human memory is crucial in conducting really any type of daily matter no matter how important or unimportant it can be. Similarly, computer, digitized and software memory is a crucial design principle for the functionality and existence of computes as we know them today and “the most sophisticated mechanism” (Denning & Martel, 2015, 23). However, in order to keep that memory and all of its functionalities safe, the concept of security came to play a major role in the computer’s system design principles as life slowly started taking a turn “online”, we had to find ways to secure privacy, individuality, etc. the same way we did in real life, online. Starting with time-sharing systems in the 1960s, information protection, ways to control access to confidential and private information, ways to file systems hierarchically to provide user customization and more, policies for computer operators ((Denning & Martel, 2015, 23), needed to be created in order for people to share the same familiarity and feeling of safety that they do in real life, virtually. 

Because of the aforementioned, two number usage, the “Y2K” problem arose highlighting the importance of danger in information vulnerability that can be found due to network sharing, the World Wide Web and more, database records, passwords, personal information, etc. can be easily accessed and uncovered if they want to be (Denning & Martel, 2015, 23-25). Machine Learning and Artificial Intelligence have made it possible to create for security purposes factors of authentication and identification. Biometrics, for example is the “recognition or authentication of people using their physiological and/or behavioral characteristics”, these can include “the face, […], fingerprints, iris, and palm [as well as]. dynamics of signature, voice, gait and keystrokes” (Alpaydin, 2016, 66). Even under these circumstances where technology has developed to such an extent where we can literally unlock our phones with our faces or walk through stores and office spaces while purchasing things and tracking location is rooms through facial recognition, to unlocking high risk information and privacy matters with your eyeball or finger-print, we can extensively comment and discuss the social and ethical issues that arise from such capabilities, showing exactly the idea that in reality all of these “ultimate-crypto-computer-sciency-too-hard-for-anyone-else-to-understand” myths, are truly just a reflection of very human selves onto something technological that we have created so extensively to the point were even our human biases, debates, prejudices, etc., have been unconsciously (or consciously) applied on/in to them. 

Denning & Martell, 2015 p.27-28 

“Automated cognition systems need not work the same way the human mind works; they do not even need to mimic a human solving a problem. […] The methods used in these programs were highly effective but did not resemble human thought or brain processes. Moreover, the methods were specialized to the single purpose and did not generalize.” 

So why do we alienated ourselves and are so concerned/scared about the development of tech, AI, computers, etc. when they can basically never be as intelligent and as advanced as the human cognitive brain and mind? 

References 

Alpaydin, E. (2016). Machine learning: The new AI. MIT Press.

Denning, P. J., & Martell, C. H. (2015). Great principles of computing. The MIT Press.

Irvine, M. (2020). Introduction to Computer System Design. 

It is interesting to think about the conceptualization of Artificial Intelligence. Most of the readings, discuss the dystopian accountabilities that are attributed to AI since it was brought to the forefront of technology. However, people have always had this hidden or apparent fear of the deepening threat of technological innovation and the ultimate and detrimental effects that it could have on society and our world. When talking about cyborgs and robots people expected a type of technology that was cold, distinctive and far away. Yes a cyborg and/or a robot might re-install a lot of human attributes but at least it doesn’t necessarily resemble the human species. If anything, if a cyborg-robotic attack were to take place, the human race always had a dichotomized “us” and “them” side in order to defeat them. What AI reveals though, is that no one really imagined the form technology would soon take on. Maybe the familiarity with Siri’s voice and attitude, or Google’s Assistant ability to always know what you’re into, the fact that your phone always provides you with results and ads you have discussed with a group of friends, is what creates this bizarre feeling of acquaintance with this type of technology. Yet its complexity also creates the fear and misunderstanding that comes along with it. 

The independence found in this type of technologies, creates this “discourse and false assumptions around AI and computer systems” (Irvine 2021). Ironically, media culture has only deepened the misinformation issue with AI and enhanced this sense of a threatening dystopia (Boden, 2016; Wooldridge, 2020). In reality if more people were to truly understand AI and de-blackbox it, the desensitization towards AI would become obsolete. The success of AI as we know it today can only be attributed to the cumulative expansion and adaptation of various aspects of computation and computing systems that have taken place through out the years.  (Irvine, 2021; CrashCourse, 2019; Boden, 2016; Wooldridge, 2020). From the very early human symbol systems to complicated automated computing calculations, AI’s history is way closer to home and “humanity” than people often think; “Everything we do in computing and AI is part of a long continuum of extending the human symbolic-cognitive capacity for symbolic systems (from language to abstract algebra and digitally encoding multimedia), and designing technical media systems for and with those systems.” (Irvine, 2021, 9). Concepts and patterns that were created even thousands of years ago to improve and facilitated human life and development through its every stage are still being used and the reason behind why we are able to have the technology that is available for us today. 

I really enjoyed going through these readings as they were the perfect connection and delved into my focus on daily uses of AI. I found a lot of similarities in concepts and facts that were mentioned and that I had previously used for other class and research from previous semesters such as my own research for 505 and for Computing and the Meaning of Code (711). 

AI has the capacity to touch most aspects of our lives whether that is with its applications i.e. where can we find AI: everywhere! Its capabilities to adapt to a vast aspect of things from self driven cars to IPAs to sat-nav systems, AI’s “homologous design” (Boden, 2016) can be represented through a myriad of different ways, forms and even languages. It combines the “spirit” of humanistic psychology, philosophy and neuroscience with that of technology, binary, computational and symbol systems that together work towards enhancing and providing solutions for the real human world and our lives. 

Some more questions/comments: 

overselling AI? 

Marvin Minsky 

when overrating AI are we overrating our capacity? the computers/systems? or capacity of our binary symbol and other symbol systems? 

Where does the misinformation problem with AI really originate from? 

Sources:

• Boden, Margaret. 2016. AI-Its Nature and Future. Great Britain: Oxford University Press. https://drive.google.com/file/d/1P40hHqgDjysytzQfIE7ZXOaiG0Z8F2HR/view?usp=drive_open&usp=embed_facebook.
• CrashCourse. 2019. What Is Artificial Intelligence? Crash Course AI #1. https://www.youtube.com/watch?v=a0_lo_GDcFw&list=PL8dPuuaLjXtO65LeD2p4_Sb5XQ51par_b&index=2.
• Woodbridge, Michael. 2020. A Brief History of Artificial Intelligence. 1st ed. New York: FlatIron Books. https://drive.google.com/file/d/1zSrh08tm9WbYtERSNxEWvItnKdJ5qmz_/view?usp=sharing&usp=embed_facebook.
• Irvine, Martin. CCTP-607: Leading Ideas in Technology: AI to the Cloud. Introductory Essay: Part 1