Author Archives: Danae Theocharaki

De-blackboxing “the Cloud” and the Principle of Scalability

Abstract

            The abstractness of the term “the cloud” has left many unknowns in a technology that has been rapidly evolving and present in most computational and technological advancements that we use on a regular basis. The nature and characteristics of the cloud create a mystery behind the systems and infrastructure both computational and physical that accompany cloud computing. By de-blackboxing and navigating through the main features, characteristics and concepts of cloud computing, an emphasis is placed on the understanding that the vast production of data can also lead to the overuse of data centers and physical concepts that ultimately have an impact on the environment.

Introduction

Figure 1. via GIPHY

Cloud computing has been an expanding phenomenon and been put to great use over the past decade. From personal use, to businesses, educational institutions, governmental institutions and even health care establishments, rely on the efficiencies, safety and operability of “the cloud” for day-to-day functions and operations. The effectiveness and performance of the cloud slowly became adopted by anyone with a smart device as big tech and software companies not only use cloud computing technology in their products but are also the ones who create it, develop it and hold major decisions over it. With the rapid evolution of technology, more and more data is being constantly transferred, saved, uploaded, downloaded and more, in such large amounts that only powerful “high-performance computing” systems such as cloud computing, can “handle the big data problems we have today in reasonable time” (Alpaydin,2017, p. 176). The concept of being able to access your data from a non-specific location, without having to use a specific device, or carry a floppy disk or USB-stick, was not fathomable a few decades ago. The idea of an “invisible” cloud where everything and anything can be manipulated, stored and re-distributed made peoples’ fast-paced lives even more accommodating. Of course, it’s not just personal use that comes into play, but also businesses, companies and large corporations do not have to invest in thousands of computes, maintenance and support staff nor their own data servers and space, since someone else can provide that service to them (De Bruin & Floridi, 2017; Bojanova et al., 2013). An intangible, invisible “cloud”. Or is it? To what extent is it as abstract as most people think it is? De-black boxing cloud computing or “the cloud”, is critical towards understanding its implications both virtually and in the real, physical world. This piece further investigates how and to what extent does cloud computing use and consumption, affect the physical implications and infrastructures in terms of their environmental impact.

What is “The Cloud”?

One of the biggest cloud computing management companies Amazon’s Amazon Web Services (AWS) defines cloud computing as “the on-demand delivery of IT resources via the Internet” that provides access to any tech services on a on an as-needed basis (AWS, 2019). Among the plethora of things that cloud computing can be and is used for some are: “data backup, disaster recovery, email services, sharing virtual desktops, big data analytics, customer-facing web applications. It can also be used for personalized treatment of patents, fraud detection for finance companies, provide online games for millions of people/players around the world and more” (Theocharaki, 2021).

The National Institute of Standards and Technology also known as NIST, define cloud computing as:

 Cloud computing is a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction. This cloud model promotes availability and is composed of five essential characteristics, three service models, and four deployment models. (Ruparelia, 2016, p. 4)

Although, as a technology it is still a new and constantly evolving “phenomenon” and because of the black-box mystery that is attached to it, one can’t say that there is an exact definition but more of an overall concept of what cloud computing is and what it does. Overall, the cloud is “actually a services or group of services” (Roundtree et al., 2014, p. 1), where a collection of technologies or a set of common technologies work together, so that the data and computation attributes are handled in large and remote off-site data centers. According to the NIST, cloud computing can be distinguished by three main components; key cloud characteristics, cloud deployment models and cloud service models (Alpaydin, 2017; Roundtree et al., 2014). Behind this “modern buzzphrase” of cloud computing, “hides a rich tradition of information sharing and distributed computing” (Denning & Martell, 2015) whose vast unknown of what took place behind the border of the box gave it its famous name; “the Cloud”.

History of The Cloud

Figure 2. Project MAC’s, IBM 7094

In the 1950s and 1960s, big companies such as IBM had already figured out a business model for cloud computing with the use of “rented computation time on large mainframe computers” and researches such as John McCarthy, who was a leading Artificial Intelligence computer scientist at Stanford, investigated the “ideas of computation as a utility function (De Bruin & Floridi, 2017, p. 24). In the mid-1960s the Massachusetts Institute of Technology (MIT), built the Project MAC – an acronym for “multiple-access computer” or “man and computer”, which conceptualized the “idea of building systems that could share computing power among many users” (Denning & Martell, 2015, 28). Project MAC lead to the invention of Multics an early operating system that allowed memory, disk and CPU to be distributed over among many people with the incentive of sharing the cost responsibility and therefore lowering the price of individual payment.

Figure 3. The H6180 Multics at MIT

The supply of the computing power would be used as a utility, a commodity that anyone could use.  Towards the end of the decade, ARPANET (The Advanced Research Projects Agency Network) followed the essence of utility; resource sharing and wide accessibility and as long as you were connected to the network you could connect with any host and therefore service(s). This soon evolved in what we now know as TCP/IP protocols, which official set and standardized in 1983 by APRANET. TCP/IP protocols allowed for message exchange without having to know someone’s actual location but just IP addresses, it was based on open standards that could be used in open-source software (Denning & Martell, 2015; Irvine 2021; Nelson, 2009). After adopting the Domain Naming System (DNS) a year later, host names now had their personalized numeric IP addresses (xxx.xxx.x.xx) creating even more flexibility between communications and location of internet matter (Denning & Martell, 2015).

By the 1990s when the World Wide Web was taking over, just as Cloud Computing started gaining more fame in the early 2000s, the de-blackboxing of such types of computing and the knowledge behind their functionalities, paved the way of how they were to be understood by the general public. The presence of the WWW, created further transparency and ‘manipulation’ of information objects across networks and the Internet especially after the appearance and creation of Uniform Resource Locators (URLs) and the Digital Object Identifier (DOI) system, that gave unique identifiers and ‘names’ to ‘things’ on the Internet creating unique digital objects (Nelson, 2009; Denning & Martell, 2015).

The client-server architecture that is used by most web services in the cloud even today, can be attributed to MIT’s Multics, which developed the idea of sharing resources from a mainframe system for multiple users, Xerox Palo Alto Research Center’s system “Alto”, a network of independent graphic workstations that were all connected together on an Ethernet, and another MIT creation the ‘X-Window’ client-server system, that basically granted pre-established client-server communication protocols, allowing new service providers to user their own hardware and user interfaces without the extra hassle of designing new protocols (Denning & Martell, 2015, 30).

Figure 4. Xerox PARC’s Alto system

 With the creation of more and different forms and products in tech, such as PCs, tablets, smart phones, email services etc. cloud computing gained huge interest as it managed to adapt and support these ‘expansions’. In 2006, Google’s then CEO – Eric Schmidt, popularized the term to what most people now refer to as “The Cloud” and has become a part of pretty much anything we do that is related to technology in one way or another (DeBruin & Floridi, 2017, p.23-24).

Architecture & Functionality

Almost everything we do or use in terms of our day-to-day technology is in one way or another a part or a process of cloud computing. From our email services, to video streaming services such as Netflix or YouTube, to smart phone speech recognition to sharing your files on Google Drive, uploading them on Dropbox, sending photos, doing online school on Zoom, or working with ten other people on the same project at the same time on a specific platform, and so much more relies on “the cloud” for our daily functioning that has now become almost something we take for granted. The perplexity of the systems and processes that go on into what makes “the cloud” and the fact that it encompasses such an interconnected vastness of groups of services, frameworks, paths, etc. makes it all that more complicated to detangle and understand. Exactly because of how broad the definition or concept of “the cloud” can be, doesn’t necessarily mean that everything that is on the internet or Web-based application/product, make it a cloud application.

The five main characteristics that a service/software/system/platform needs to have in order to be considered a part of cloud computing are: on-demand self-service, broad network access, resource pooling, rapid elasticity and measured service (Ruparelia, 2016; Rountree, 2014).On-demand self-service is the idea that users/consumers have the ability to access the service on their own without the presence of a “middleman”. Broad Network Access refers to the accessibility of cloud services that only require a network connection in order to connect to the services/products/applications. The better the connection (i.e. LAN – Local Area Network connections) or a good Internet connection, the better and more efficient the services will be and should also support access from any type of device (i.e. smartphones, tablets, computers, etc.).Resource Pooling benefits the provider’s side as it implies that since the customers/users will not always have the need to simultaneously use all the resource available to them at the same time, the ones that are not being put to use can benefit and be used by another customer/user. This in a sense saves resources and allows providers to service more customers than they would’ve if all resources were constantly having ‘to work’ for one user, even though they were not being used. 

Rapid Elasticity entails the ability of the cloud, to grow to the user’s demand and satisfaction. Through automation and orchestration, when the resources have been used to their full capacity, the system will automatically seek to gain more capacity expansion. On the customer’s end this looks like this unimaginable space in the cloud, but in reality, for the providers this means that the more space is wanted the more physical resources need to be implemented such as computer resources, hard disks, servers, etc. However, the key to this is that the resources in demand is that in order for the providers to “save on consumption costs” such as power, electricity, cooling systems and more, “even though the resources are available, they are not used until needed”. Similarly, once “the usage has subsided, the capacity shrinks as needed to ensure that resources are not wasted” (Roundtree, 2014). Measured services are the fifth characteristics that a service/software/system/platform in order to be considered cloud computing. Measured services means having the ability for cloud service/providers to measure usage such as the time i.e. for how long has someone been using the service, the amount of data used i.e. how much space is it taking up, etc. This also is what determines the rates and prices of plans. If you have ever gotten notifications about running out of cloud storage on an Apple device or needing to update your cloud payment options/plan on your Google Drive, and you have payed money that has ‘magically’ increased your cloud space in “the cloud”, then it is this ‘phenomenon’, one could even say ‘luxury’, of measured services and rapid elasticity (Rountree, 2014).

Cloud Service Models

As previously mentioned, the vastness of services that can be offered from cloud computing are called “cloud service models” and are more broadly categorized into the types/kinds of services that they offer based on their target audience, responsibilities and tasks, costs, etc. The three basic service models are Infrastructure as a Service, Platform as a Service and Software as a Service.

Infrastructure as a Service also referred to as IaaS, is the service that provides “basic infrastructure services to customers” and the hardware infrastructure so both physical and virtual machines, i.e. networking, the servers, storage, plants, etc. on a utility basis. Example of this can also include; IT systems monitoring, backup and recovery, platform and web hosting, etc. (Rountree, 2014, p. 7; Ruparelia, 2016, p. 21 & 131). Some “real life” examples and applications of Dropbox with file synchronization, printing with Google Print, hosting on Amazon EC2 or HP Cloud or storage on Amazon Cloud Drive, Apple’s iCloud, etc. (Ruparelia, 2016). Platform as a Service or PaaS, “provides an operating system, development platform, and/or a database platform”. This allows and creates the ideal environment for installing and running software, developing applications by eliminating the need for a company – the client, to have to build their own infrastructure in order to develop those aps. Other “real life” examples and uses include development with languages such as C, C++ and Java, database services for business intelligence and data-warehousing. Software as a Service or SaaS, provide “application and data services” by supplying hosted applications without the need of installing and downloading them, paying extra for them or giving up space for them on your hard drive or storage disk/drive. From the application skeleton itself to all the data that comes with it, SaaS means that the cloud service/provider is responsible for maintain and keeping all platforms and infrastructure needed for the services to take place. SaaS is “the original cloud service model […] and remains the most popular” as it “offers the largest number of provider options” (Rountree, 2014, 7). It also entails use cases such as billing and invoicing, asset and content management, image rendering, email and instant messaging and more. Applications of SaaS include email services such as Gmail, Hotmail, Outlook, etc., collaborative suites such as Google Docs and Microsoft Office 365, content management such as Box and SpringCM and customer relationship management with Salesforce. (Ruparelia, 2016).

Figure 5. Cloud Service Models diagram by David Choo

In Cloud Computing (2016), Ruparelia and a few other identify and discuss the presence of further/more specific service offerings in terms of their abstraction levels. Information as a Service (INaaS) and Business Process as a Service (BPaaS) are two of those. Information as a Service (INaaS) is responsible for providing business information that is relevant to the specific customer/client whether on an individual, business or corporate level. This may include market information, price information, stock price information, information validation, business processes and tasks, health information from patients, real-time flight information, etc. (Ruparelia, 2016, p. 177; Mosbah et al., 2013, p. 3). Business Processes as a Service (BPaaS) aids in business process outsourcing by carrying out business functions that rely on large amount of service and data that facilitate in a business’s functioning. This can include ecommerce, payroll and printing, tax computation, auditing, health pre-screening, ticketing and billing, etc. Google’s AdSense and IBM’s Blue are examples of these. (Ruparelia, 2016, p. 193; Mosbah et al., 2013, p. 3).

 

Cloud Deployment Models

 

With the wide variety of cloud computing options and services each individual, business, organization, corporation, etc. differs in what they need to use cloud services for. In order to support the environment in which personal or business use is needed or wanted, a certain kind of cloud environment must be implemented by having different service models. The four deployment models of the cloud are public, private, community and hybrid.

The public cloud service model is the most commonly thought of as all of its services, systems and operations take place in a housed external service provider. The infrastructure of the cloud is owned by the cloud service organizations who are responsible for administering and managing the provided service and can apply this across abstraction levels and available via the Internet. Some example of the public cloud model are Google Docs, Microsoft Office 365 and Amazon Cloud Player. (Ruparelia, 2016; Mosbah et al., 2013; Rountree, 2014).The private cloud service model all the services, systems and resources are provided and located by the individual’s company’s, organization’s or person’s private cloud with zero access to the public. Private clouds can be accesses through a local (LAN), wide area network (WAN) or through a private virtual network, VPN and is managed, operated and maintained by the individual(s) in question. (Ruparelia, 2016; Mosbah et al., 2013; Rountree, 2014).The community cloud service model is a semi-public cloud or a “broader version of a private cloud” (Ruparelia, 2016, 32) and is shared among members of a group, organization, etc. that have some sort of shared goals, missions, concerns, etc. This is specific to groups/organizations that perhaps for security and safety measures/reasons do not want to use the public cloud and theresponsibility of maintenance is shared among the members/users who have access to it. Examples of its use include a publishing cloud, a health industry cloud or a banking regulation cloud. (Ruparelia, 2016; Mosbah et al., 2013; Rountree, 2014). Finally, the last cloud service model is the hybrid 

Figure 6. Representation of cloud variety by Ruparelia et al.

cloud. This entails a combination of two or more of the aforementioned cloud models that are not mixed but linked together to work more efficiently and to achieve their specific goals/operations and allow data and application portability. A hybrid cloud can consist of public and private clouds and the mixing and matching allows its users/customers more flexibility and choices in what they do and how they use their cloud services. (Ruparelia, 2016; Mosbah et al., 2013; Rountree, 2014).

Figure 7. A great depiction of The Relationship between Services, Deployment Models, and Providers by Mosbah et al.

 

Data Centers, Principle of Scalability and Cloud Computing Emissions

With the ambiguity that accompanies what “the cloud” really is, this concept that after all it might really just be a cloud, an invisible mass of data, information, systems and software comes a lot of misunderstanding about its functions, operations and of course consequences. However, in order for the computational and electronic aspect of cloud computing to take place there needs to be some sort of physical support that accompanies the cloud products and services, in general the overall system. With the mass production, circulation, consumptions, manipulation, etc. of data in unquantifiable amounts, technological challenges can come into play. Scaling out is a main concern of cloud computing that is getting more and more attention and being further addressed not only by people in the tech or science field but also those in the natural and environmental scientists and even pop-culture. The environment of cloud infrastructure, entails and relies on commodity equipment which means that in order to “add capacity, you need to scale out instead of scaling up” (Rountree, 2014, 16). Scaling out can lead to extra pressure and burden for datacenter and facilities that host the cloud’s infrastructure and “increased environment-related costs in resources such as power and cooling” (Rountree, 2014, 16) amongst a variety of other things.

Data centers are physical locations/sites/areas/ spaces, the true “home” of cloud computing” where all the countless of servers and processors are housed. Data centers are spread out in all different areas and cities, remote or otherwise, in the U.S. and all over the world. The various data centers can communicate and collaborate with each other through a network through which “tasks are automatically distributed and migrated from one to the other” (Alpaydin, 2017, 152).

“As cloud computing adoption increases, the energy consumption of the network and of the computing resources that underpin the cloud is growing and causing the emission of enormous quantities of CO2”, explains Gattulli et al., in their research on cloud computing emissions (Gattulli et al., 2014).  In the past decade alone, “data center energy usage has decoupled from growth in IT workloads” with public cloud vendors, also being among the biggest (tech) companies in the world, deploying large amounts of new cloud systems and networks leaving an environmental impact that is often times harder to asses because of the nature of this technology, than it is to calculate other sort of emissions (Mytton, 2020). “Although the large cloud vendors are amongst the largest purchasers of renewable electricity, customers do not have access to the data they need to complete emissions assessments under the Greenhouse Gas Protocol” leading the way for scientist and researchers such as Gattulli and Mytton, to find new ways and methods to control IT emissions and lessen the environmental impact that our overreliance on the efficiency of this technology has on our planet. Over the past 5 or so years, the Information and Communication Technology’s carbon emissions alone have amounted to 1.4% – 2% of total global greenhouse gas emissions, “approximately 730 million tones CO2 equivalent (Mt CO2-eq)” (Ericsson, 2021; Gattulli et al., 2014). Data centers that are used for public internet alone consumed 110TWh in 2015, almost 1% of the world’s electricity consumption (Ericsson, 2021). Often, we do not think of all the daily services and products we use that ultimately rely on the cloud for their functions, such as video streaming platforms, gaming, overall uses of AI and Machine Learning, cryptocurrencies, etc. In 2017 for example, Justin Bieber’s song “Despacito”, “consumed as much electricity as Chad, Guinea‑Bissau, Somalia, Sierra Leone and the Central African Republic put together in a single year” through streams and downloads (five billion) and Bitcoin mining “accounted for 0.2 percent of global electricity usage in mid-2018” (Ericsson, 2021).

Figure 8. Representation of the Carbon footprint of ICT and data traffic development by Ericsson

Figure 9. Distribution of ICT’s carbon footprint in 2015 by Ericsson

Conclusion

The technological evolutions of the past decades have led to the amazing invention of cloud computing. The “explosive growth of data and the need for this data to be securely stored yet accessible anywhere, anytime” lead to a higher demand and even need of cloud computing (Bojanova et al., 2013).  Of course, this has created a circle of constant data and data services being constantly re-born and re-distributed in the broad network and cloud. The mystery behind what cloud computing and “the cloud” is, doesn’t necessarily help with understanding and conceptualizing the physical and material aspect of this technology. Therefore, this further instigates the hidden implications that come along with disregarding the fact that cloud computing isn’t so much in “the cloud” but on physical location on earth that keep getting larger and more with the exponential increase of cloud computing services demand. As it happens, data centers that hold and are the backbone of cloud computing, as well as all the other external ‘expenditures’ such as electricity, maintenance, etc. have much heavier implications on the environment than we assume from a conceptually intangible technological advancement. Recent research and environmental analysis, support the idea that low-carbon cloud-computing solutions, renewable energy sources, as well as gaining access to data about cloud computing emissions and power usage effectiveness can increase awareness and understanding of what is going on behind the scenes of this technology that we truly hold so dear to us (Mytton, 2020; Gattulli et al., 2013; Ericsson, 2021).

 

Bibliography 

Alpaydin, Ethem. (2016). Machine Learning: The New AI. Cambridge, MA: The MIT Press.

Amazon’s Amazon Web Services 

Bojanova, I., Zhang, J., and Voas, J. (2013).  “Cloud Computing,” in IT Professional, vol. 15, no. 2, pp. 12-14, doi: 10.1109/MITP.2013.26.

De Bruin, Boudewijn and Floridi, Luciano. (2017). The Ethics of Cloud ComputingScience and Engineering Ethics vol. 23, no. 1 (February 1, 2017): 21–39.

Denning, Peter J.  and Martell, Craig H.. (2015). Great Principles of Computing. Cambridge, MA: The MIT Press. 

Ericsson. (2021). ICT and the Climate. Ericson. https://www.ericsson.com/4907a4/assets/local/reports-papers/consumerlab/reports/2020/ericsson-true-or-false-report-screen.pdf

Gattulli, M., Tornatore, M., Fiandra, R., and Pattavina, A. (2014). “Low-Emissions Routing for Cloud Computing in IP-over-WDM Networks with Data Centers,” in IEEE Journal on Selected Areas in Communications, vol. 32, no. 1, pp. 28-38, doi: 10.1109/JSAC.2014.140104.

Irvine, M. (2021) What is Cloud Computing? AI/ML Applications Now Part of Cloud Services. Class notes: https://irvine.georgetown.domains/607/

Mosbah, Mohamed Magdy, Soliman, Hany and El-Nasr Mohamad Abou. (2013). Current Services in Cloud Computing: A Survey. International Journal of Computer Science, Engineering and Information Technology (IJCSEIT), Vol.3,No.5 DOI : 10.5121/ijcseit.2013.3501

Mytton, D. (2020). Assessing the suitability of the greenhouse gas protocol for calculation of emissions from public cloud computing workloads. Journal of Cloud Computing, 9(1) doi:http://dx.doi.org.proxy.library.georgetown.edu/10.1186/s13677-020-00185-8

Nelson, M. (2009). Building an Open Cloud. Science, 324(5935) from http://www.jstor.org/stable/20536490

Ruparelia, Nayan B. (2016). Cloud Computing. Cambridge, MA: MIT Press, 2016. 

Roundtree, Derrick and Castrillo, Illeana.(2014)The Basics of Cloud Computing: Understanding the Fundamentals of Cloud Computing in Theory and Practice. Amsterdam; Boston: Syngress / Elsevier.

 

Theocharaki, D. (2021). Cloud Monopoly. Class notes: https://blogs.commons.georgetown.edu/cctp-607-spring2021/2021/04/08/cloud-monopoly/

 

Figure 1: GIF from Giphy 

Figure 2: Photo of Project MAC’s, IBM 7094 from Multicians

Figure 3: Photo of H6180 Multics at MIT from http://gunkies.org/wiki/Multics 

Figure 4: Photo of Xerox PARC’s Alto system from Wired article “The 1970s Conference That Predicted the Future of Work” by Leslie Berlin 

Figure 5: Photo of Cloud Service Models diagram by David Choo

Figure 6: Screenshot from Ruparelia, Nayan B. (2016). Cloud Computing. Cambridge, MA: MIT Press, 2016. 

Figure 7: The Relationship between Services, Deployment Models, and Providers by Mosbah, Mohamed Magdy, Soliman, Hany and El-Nasr Mohamad Abou. (2013). Current Services in Cloud Computing: A Survey. International Journal of Computer Science, Engineering and Information Technology (IJCSEIT), Vol.3,No.5 DOI : 10.5121/ijcseit.2013.3501

Figure 8: Representation of the Carbon footprint of ICT and data traffic development from Ericsson. (2021). ICT and the Climate. Ericson. https://www.ericsson.com/4907a4/assets/local/reports-papers/consumerlab/reports/2020/ericsson-true-or-false-report-screen.pdf

Figure 9: Distribution of ICT’s carbon footprint in 2015 from Ericson. https://www.ericsson.com/4907a4/assets/local/reports-papers/consumerlab/reports/2020/ericsson-true-or-false-report-screen.pdf

 

 
 

 

 

 

Aren’t you scared?

Ethics and socio-technical implications of AI or the broader technology we use has been a recent hot topic. However, although it is a “rising phenomenon”, I’d like to question even where and when we come across it. Documentaries such as Coded Bias and The Social Dilemma depict who are the few who are aware of the implications AI can have on our lives and livelihood as well as our rights as we have known them thus far. This course is an example of that; people in academia, people who work in the field such as coders, data scientists, etc., ‘tech activists’ and politicians specialized on privacy, tech and internet issues and legislature. But are the big companies concerned? Are the governments concerned? Who does this system benefit? 

Few weeks ago I was at the doctor’s and when asked what I study he turned around and said “Aren’t you scared?”. In that moment my first thought was what a bizarre question to get from a doctor. Scared of what? Robots taking over? Not necessarily. Machines taking over? Ehh, not necessarily still far away from that, plus humans still control what we create in terms of human vs robot. However, what is scary is the way in which the technologies we use today are embedded in human biases, racism, sexism, human rights issues and other societal implications. People assume that because we don’t see what is behind our technology, we don’t see the “blackbox” that governs how the technologies we use on a daily basis operate, who operate them and how. I think up until now it has been hard for people to understand why there are biases in algorithms or issues within the system. A lot of question that typically pop up are: Don’t more women work for tech companies? Don’t people of color work in the field nowadays? Don’t we have a more diverse workforce in the tech industry? Yes sure, is the simple answer (not as many as they should but let’s start with baby stems for the sake of this description). The problem is that that isn’t the root of the issue. I truly appreciated how well Coded Biases explains why and how these biases exist from the ground up. Simple starting with the Dartmouth conference in the summer of 1956 we see who were the creators of this initiative, of AI, of ML, white men. So the whole process, the whole system starts being fed with patterns that depict just that. Not necessarily in terms of what photos we fead the ‘machine’ for facial recognition, but concepts, theoretical and social representations were those depicted by a small pool of data. The data that you feed in the machine is what it will learn from so if most photos are of red flowers for example, the system has an easier time reading red typical flowers than lets say a white purple-dotted orchid. So the discord of AI began and picked up on that narrative alone. A machine doesn’t have a soul, doesn’t have its own thinking, it learns from what humans feed into it. It does what humans tell it to do. So who is scarier? Humans or the machine? 

Yes, it’s mostly based on a code. Yes there is more diversity in the tech industry and field of AI. But who writes the code? Or better yet, who had written all the code, today’s code is based off of. Last class, we talked about people in academia, such us ourselves, who question the technology we study, the technology we use and work on even though it may be our passion, it may be our interest. However, that is what CCT and other Interdisciplinary, ‘liberal art’ courses/programs are for. Studying what your interests are, but also question what that is. Don’t just look at it from one point. Nothing about technology is simple or lined up in one straight path. Maybe it’s the anthropologists in me but it’s hard to grasp that so many people in the overall field and not concerned about the implications of their actions. There is code so we hide behind it.  For the most part, computer scientists, software engineers, data scientists don’t come out of undergrad or their extremely software engineer concentrated master’s having discussed the big issues of the system, of the technologies we use today. Yes it is an amazing advancement of our days that has exponential improved our lives in more ways that we can probably think on the top of our head, but are the socio-political-economic repercussions and biases greater that what we have made them? 

We saw in the weeks of Cloud Computing and Big Data, the consequences or ideas of monopolizing an industry in which regular people and companies alike, rely on for the safekeeping and management of their documents, files and many more. The naiveness of people who assume that just because they don’t have their location settings on or they don’t use social media, that they also don’t have a technological footprint or a way for governments to monitor them. Unless you are 100% detached from everything technological, which honestly can be pretty hard these days especially because if you as an individual might be, something that you use such as a product or service might not be, one way or another data or information about you is out there. Big Data isn’t just random numbers and information collected from thousands of sources. Those numbers and figures didn’t magically appear out of nowhere. They are being fed into the system by our own usage of everything that we do. At the end of the day, we are the machines. Our societies are the technologies. They didn’t create themselves out of nowhere. And being skeptical and able to question what is really going on behind the scenes is what is going to help us conceptualize and overcome the many issues and implications that exist and are constantly happening, as depicted in Coded Bias. 

 

Resources 

Film Documentary, Coded Bias (Dir. Shalini Kantayya, 2020). Available on Netflix, and free to view on the PBS Independent Lens site

“Big Data”: ACM Ubiquity Symposium (2018).

Rob Kitchin, The Data Revolution: Big Data, Open Data, Data Infrastructures and Their Consequences. London; Thousand Oaks, CA: SAGE Publications, 2014. Excerpts.

Cathy O’Neil, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy (New York: Crown, 2016).

Boudewijn de Bruin and Luciano Floridi, “The Ethics of Cloud Computing,” Science and Engineering Ethics 23, no. 1 (February 1, 2017): 21–39.

Geoff Dougherty, Pattern Recognition and Classification: An Introduction (New York: Springer, 2012). Excerpt: Chaps. 1-2.

Big Data; myth or reality?

This weeks reading were definitely an interesting out look on what we mean by “big data” and re-defining not only the definitional but certain socio-economical theories and constitutes about it. For the most part, when people talk or generalize about “big data” it is this unfathomable, unquantifiable large amounts of information that we’re trying to categorize and clean-up for x and y reason for x and y company. Although this isn’t technically wrong, the authors and sources we looked at especially this week (but also can of course connect it to previous weeks and greatly falls right after last week’s cloud computing theme) give a different perspective on deeper issues and concepts that immediately surround and relate to the accumulation, understanding, meaning and use, purposeful or accidental, of the countless of data that is collected every second of every day. As Rob Kitchin puts it:

“Big Data creates a radical shift in how we think about research …. [It offers] a profound change at the levels of epistemology and ethics. Big Data reframes key questions about the constitution of knowledge, the processes of research, how we should engage with information, and the nature and the categorization of reality … Big Data stakes out new terrains of objects, methods of knowing, and definitions of social life. (boyd and Crawford, 2012)” (Kitchin, 2014, 1). 

In a more data science perspective, big data works to analyze, extract information, deal with large and complex amounts of data that cannot be deal with or processed through “every-day” software that we use on the go, as it also is sufficient enough for the fast pace lifestyles that most of us lead. Basically, accumulating unstructured data that needs to be filtered and categorize in order to serve a purpose. But what we are calling on here, is the realization of how much more complicated “big data” really is and the fact that in reality, “bit data” affect our lives way more than we actually realize and play a “prominent role in every aspect” of it (Huberman, 2017). The way we choose to live our lives nowadays, is a life that in a way is constantly interloped with technology (smart phones, emails, social media & networks, credit cards, smart home devices, cars, laptops, etc.) where knowingly or unknowingly we are constantly feeding back the system with so much information about us, what we do, where we are, what she buy, eat, drink, listen to, that in return we get a very personalized portfolio if you may, that matches our preferences, hobbies, lifestyles, etc. We get personalized ads, personalized feeds and more because of this mass accumulation of data that is taking place on a much larger scale than it did even 5-10 years ago since not only has the accumulation of different devices per individual increased but our lives on the internet have also developed on such an exponentially fast paced trajectory. Imagine how many people around the world are constantly “feeding” the cloud or companies with data and information that then has to be analyzed, categorized and set to its respective path to only be processed by companies and then be fed back to us in more disguised and discrete forms, one of those mainly being advertisements.

Johnson and Denning (2017) also emphasize this huge “big data revolution” as a result of “the expansion of the internet into billions of computing devices, and the digitization of almost everything. […] Digitization creates digital representations for many things once thought to be beyond the reach of computing technology”. And this exactly explains how much “big data” truly affect all aspects of life that not only have the ability to personalize ads but also indicate yet again how globalized this world has become because of the constant development of technology. For example, especially during this pandemic, we saw the importance of online education something that would have been never imagined or accepted years ago. The fact that so many children, people, educators, students, etc. all over the world are able to log onto to platforms from wherever they are for hours at a time while also being able to record, participate, interact and do so much more while getting an education can be attributed the the capacity of technology to support such activities and not only maintain them as they are happening live but also save them for future use. Even the cities we live in, accumulate countless of data over processes we most likely don’t even assume provide data yet it is truly hard to not be digitized nowadays, otherwise the difficulties and setbacks that can arise with being “disconnected” or not apart of the world. Transportation services in cities whether smart app rides Uber, Lyft, etc. who collect data, so does public transportation such as buses to monitor the amount of people who use them, to plan out routes, get informed on best possible routes, traffic accidents and more. CCTV and other security systems are constantly monitoring, recording and collecting information many times on the spot analyzing potential threats or issues. Of course, socio-political and economic issues are ultimately affected by the development and evolution of technology in all aspects of life. Examples of this are economic crisis that never just affect one entity or one country or one company but the whole system, wars and political disputes do not stay limited within borders or zones but expand into other circumstances and cross borders as people migrate, seek refuge, change status, etc. 

 

 

Resources

Bernardo A. Huberman, “Big Data and the Attention Economy” Ubiquity 2017, (December 2017): 2:1–2:7.

Jeffrey Johnson, Peter Denning, et al., “Big Data: Big Data or Big Brother? That Is the Question Now (Concluding Statement),” Ubiquity 2018, no. August (August 2018): 2:1–2:10.

Jeffrey Johnson, Peter Denning, et al., Big Data, Digitization, and Social Change (Opening Statement)Ubiquity 2017, (December 2017).

Rob Kitchin, “Big Data, New Epistemologies and Paradigm Shifts,” Big Data & Society 1, no. 1 (January 1, 2014): 1-12.

Rob Kitchin, The Data Revolution: Big Data, Open Data, Data Infrastructures and Their ConsequencesLondon; Thousand Oaks, CA: SAGE Publications, 2014. Excerpts.

Wikipedia, “Big Data

Cloud Monopoly

We have previously discussed the socio-ethical impact, biases of algorithms and AI yet focusing on the companies that control it all, takes a different turn on the overall implications that cloud computing can potentially have assuming that it converted into this “unified” architecture controlled by the “big four”; Google, IBM, Microsoft, Amazon. As de Bruin and Floridi (2017) explain, Generation X and Millennials seem to “care less about” where their private information is saved or who controls it but rather care more about the efficiency with which they can use the that information, for example, a photo and send it to friends, alter it, post it or upload it, share between devices, etc. I think overall there has been a larger impact of cloud computing in our daily lives exactly because it is so easy and efficient and allows us to have all our information, documents, material, etc. stored and located in one place and accessible from multiple locations. We definitely rely on the efficiency of it all, more than what we actually question the overall concept of what is happening to our material when we do upload/save them on “the cloud”. However, there is quite a difference when the responsibility is split up among more and smaller sized companies and therefore you also get more services and products to choose from that could possibly represent your needs or preferences better. 

AWS defines cloud computing as: “the on demand delivery of IT resources via the internet” where you can access any tech services on an as-needed basis, you can use it for data backup, disaster recovery, email services, sharing virtual desktops, big data analytics, customer-facing web applications. It can also be used for personalized treatment of patents, fraud detection for finance companies, provide online games for millions of people/players around the world and more. Basically, your private or work information is somewhere on “the cloud” and even though it might seem private on your end, as the user who could similarly be accessing let’s say a notebook or a vault, I doubt most of us read the fine prints, where more details of who has ownership and what can happen to the material and data is disclosed and explained. “Cloud computing suits the interests and values of those who adopt a deflated view of the value of ownership and an inflated view of freedom” (de Bruin & Floridi, 2017, 22). In Cloud Computing (2016), Ruparelia mentions the three things that most us look for in these services and are usually what make us choose the company/product/service we will go with whether for personal use or business purposes. Integrity and reputation, you want to be able to rely on the product, the company and the service. Which means most of us will only trust companies who not only we know because they are famous but because many others also use. It is more likely that we will trust a brand we have heard of and know can be well supported. Another important factor that goes into choosing your preferred cloud provider would be the benefits you get from cloud computing such as efficiency and promptness. Having the ability to access anything you want from anywhere in the world at any time while knowing that it is in “good hands”, safe and reliable but also in a sense never disappoints. Forgetting your hard-drive or usb stick for example can lead to major issues especially if you desperately need your data in that moment, but with cloud computing that is not something to worry about since you know whatever is on there will always be on there. Finally, pricing of each product or service plays a crucial role in the selection of a cloud computing company that would supposedly match its pricing to what it has to offer in terms of space, accessibility, security, usage, organization and more. 

Moving cloud computing into a “unified architecture” provided only by Amazon Web Services, Apple, Microsoft and IBM would be imaging a different style of data storage, accessibility, manipulation and distribution of content. Ruparelia’s (2016) given characteristics of what we look for in our cloud computing services definitely become more conscience and give you less choices to choose from. Of course this also implies the power of everything we refer to as “the cloud” to be kept among the four companies giving them more flexibility to control the data and information that is uploaded on them? What happens to those fine prints on the terms and services agreements? Where does the date ownership go or rather who does the data really belong to? What about pricing? Would it become more expensive since choices are more limited and we won’t really have more options or would they actually be more beneficial and “get-what-you-payed-for” situation? Does having all of the words cloud split among only four companies and “kept in one place”  make it riskier and more exposed to outside threats? Or does it increase security since the “big four” could potentially work together to provide a “united front” again outside threat or even cultivate healthy competition amongst them therefore creating stronger systems and walls. Floridi and de Bruin (2015) discuss the power of “interfluency” when it comes to ethically effective communication among companies but also between companies and their customers. If the big four hosting companies have the same information or share a large part of it, they should therefore  by able to “provide and seek information about relevant issues such as consumer privacy, reliability of services, data mining and data ownership” (22). The two authors also discuss  the possibility of a stricter government regulation and overall involvement in cloud computing and what that could mean in terms of restriction of the use of cloud computing, regulating what we can and cannot upload, share, distribute, etc. and what its future will look like. 

 

Resources

Who has the power?

Over the years, in one way or another we have all seen the existence and implementation of Artificial Intelligence take over so many aspects of our lives. For the most part, we don’t even see or realize that there is some sort/form of AI being implemented and used for a specific thing, products, circumstance. However, when we do notice we should also be recognizing the countesses instances where the biases and prejudices that existence within AI, are very much so present. I think because it is something not tangible or so obsolete, we assume that as an “electronic” being there is no association between AI and the societal issues that exist in our societies today. But the AI hasn’t coded itself. It hasn’t necessarily created itself out of nowhere. And so as us, the humans, “manufacture”, code and establish these presences in our lives, unfortunately or fortunately (since we know what is wrong and work towards fixing it) with it any bias of prejudice that is embedded in our human minds and lives will get encoded with it. Apart from those, relying so heavily on produced intelligence can have its own ethical implications and issues. 

With data collection being a huge part of our electronic and digital presence, most times we’re not even aware of that taking place. We’re not always really sure or aware of how and when data is being collected and what is being done with it. But if there is one thing I have realized is that data is constantly collected. Some of our readings had this as two separate ethical issues that govern AI but I feel like they are pretty similar and interchangeable and that is the autonomy and power that we rely on our AIs. The automation that goes behind AI causes “individuals [to] experience a loss of control over their lives” (Anderson & Rainie, 2018). “When we adopt AI and its smart agency, we willingly cede some of our decision-making power to
technological artefacts (Floridi & Cowel, 2019, 7). Partially, this is because the deblack-boxing of AI is still very much so in the box. However, there is a question to be posed here; will we truly ever learn what is behind the AI that we use on a daily basis? Will these companies/products ever truly reveal how they work, what they really do with all this information and data collection? Honestly, probably not since that would make them weaker to competitors. Unless, more people start realizing, noticing and want some change in terms of the control and power they have towards their use of this type of technology. As Nemitz, also explains, large (tech) corporation have taken over every aspect our of lives whether or not we realize it or sign up for it. “The pervasiveness of these corporations is thus a reality not only in technical terms, but also
with regard to resource attribution and societal matters” (Nemitz, 2018, 3). All these companies and brands, have basically collected in “their hands” countless information and data that with it they basically are able to control so many aspects of the human life especially in terms of technological, economic and political power that has been given to them through this digital power. Since nowadays, we rely so heavily on technological and the use of a digitized framework, most aspects of human life are also controlled by technology. So in a way whoever is more “ahead of the game” in the field, is the one who also has the power, the information, the data. Everyone else has pretty much lost their ability to pick and choose when, how, where they share information. It is one way or the other. If you want to have any sort of digital presence, talk on the phone, use your credit car, pay for something, look up something, everything you do is pretty much tracked down and collected, formed into a bigger overall ‘picture’. 

Another ethical issue/implication of AI, is of course the idea that all this information and data can be used for the destruction and with malicious intent towards others. Apart from ” autonomous military applications and the use of weaponized information” (Anderson & Rainie, 2018) we can also speak on the collection of information aimed towards capturing people such as facial recognition. The problem here is who is using this technology? and for what reasons? Of course, we also have to consider yet again the biases that go into this type of “vigilantism”. Racists views and opinions definitely influence who this type of technology can be geared at and who are going to be the people mostly being targeted by it. Floridi et al, also explain this in terms of how “developers may train predictive policing software on policing data that contains deeply ingrained prejudices. When discrimination affects arrest rates, it becomes embedded
in prosecution data. Such biases may cause discriminatory decisions (e.g., warnings or arrests) that feed back into the increasingly biased datasets, thereby completing a vicious cycle” (Floridi et al., 2019, 1788). 

 

How do we apply laws/regulations/safety measures for something so widely used? 

We have seen how hard it has been to manage data privacy uses and laws from one country to another, how can something so universal become so specific when it comes to protecting people? 

References 

Janna Anderson, Lee Rainie, and Alex Luchsinger, “Artificial Intelligence and the Future of Humans,” Pew Research Center: Internet and Technology, December 10, 2018.

Karen Hao, “In 2020, Let’s Stop AI Ethics-Washing and Actually Do Something,” MIT Technology Review, December 27, 2019.

Karen Hao, “Establishing an AI Code of Ethics Will Be Harder Than People Think,” MIT Technology Review, October 21, 2018. 

Karen Hao, “This Is How AI Bias Really Happens — and Why It’s so Hard to Fix,” MIT Technology Review, February 4, 2019.

Luciano Floridi and Josh Cowls, “A Unified Framework of Five Principles for AI in Society,” Harvard Data Science Review 1, no. 1

Luciano Floridi, Josh Cowls, et al., “How to Design AI for Social Good: Seven Essential Factors,” Science and Engineering Ethics, 26/3, 2020: 1771-1796.

Paul Nemitz, “Constitutional Democracy and Technology in the Age of Artificial Intelligence,” Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 376, no. 2133 (November 28, 2018): 20180089

Ok Google

This was yet again an exciting week for me as we focused more on IPAs and that is what I have been mostly focusing on since coming to CCT* as I wanted to continue what I was learning and working on during my undergrad years studying and analyzing IPAs! From studying IPAs and Alexa coming out during my undergrad times, I can’t say that there wasn’t an uneasiness of some sort surrounding this topic. Having a device in the same room as you that can constantly hear you or record you and of course keeps track of info and data, can sound pretty scary especially as a new advancement. So I tended to stay away from having my own device (not including Siri in this – because I did use Siri beforehand) yet still found the concept extremely intriguing. For me, it is the closest thing that we have to a human-like-robot as part of our daily lives. When I moved to DC a family friend gave me a Google Home as a house warming gift so we could share our photos with each other as it displays them on the screen. Since it was a gift I kept it and gave it a try and have been using it since then which is why I decided to focus on Google Assistant for this post. 

Google Assistant comes in a mobile and smart home (Google Home) versions and was initially debuted in 2016. The assistant interacts through human voice and dialogue and provides results or executes commands based on the users verbal demands. 

*There is a research paper(s) that goes along with this, please feel free to reach out if you’re interested! 

Lost in Translation – take 2

If not all of us, most of have definitely used Google translate at some point in our lives. I don’t doubt that some instances have been successful but I bet most have unfortunately not. The reason behind this is because natural human language is such a complex system of labyrinths that sometimes can only truly take place in the human mind as we can understand and interpret context, meaning and overall situational understanding that comes hand in hand with language. This is something that computers and machines have not been able to perfectly achieve yet as assigning cultural, interpretations, etc. is extremely challenging for a machine and if we think about it even for humans as often times not being a native and have grown up in the country of the language, you miss a lot of cultural nuances, interpretations, signs, etc. As Thierry Poibeau (2017) explains; “Natural language processing is difficult because, by default, computers do not have any knowledge of what a language is. […] There is a dynamic co-construction of interpretation in the brain that is absolutely natural and unconscious” (Poibeau, 2017, 23-26). However, this challenge is a highlight and key point in today’s technology and leads the path for further advancements as nothing can be done without language. So how does Google Translate work and why is not as reliable often times (aka why does the translation never make perfect sense in the final language)? 

Natural Language Processing and Machine Learning play a crucial role in how Google translate, IPAs and many more computers and systems are able to read, interpret, understand and emulate a sentence whether phonetically or in the context of translation in order to fit natural language and real human dialogue standards. NLP is “concerned with the interactions between computers and human language. How to program computers to process and analyze large amounts of natural language data.” (Wikipedia, 2021) It is needed and used to understand the context and contextual nuances to which I was referring above. Computational linguistics also play a role in this “decoding” (pun intended) process and are “concerned with the computational modeling of natural language, the study of appropriate computational approaches to linguistic questions” (Wikipedia, 2021). One of the fundamental problems in NLP was sentence deconstruction and the ability for a computer to break down the sentence into bite-size pieces in order for it to be processed more easily. Of course, it also has to follow phrase structure rules in order to encapsulate the grammar and meaning of a language. As we have seen from this weeks and previous weeks readings: parse tress, neural networks, natural language understanding (how we get meaning out of combination of letters) and natural language generation (how to generate language from knowledge, distributional semantics (make the machine/computer guess words that have similar meaning but seeing which words appear in the same sentence a lot), count vectors (the number of times a word appears in the same place or sentence as other common words), etc. all build up a system where all this data is pulled and “stored in a web of semantic information where entities are linked to one another in meaningful relationships and informational sentences” (CrachCourse #7, 2019; CrashCourse #36, 2017). 

Google Translate uses Machine Translation (constituted by all of the above and more): “the idea is to give the most accurate translation of everyday texts. […] The ultimate goal is to obtain quality of translation equivalent to that of a human being.” (Poibeau, 2017, 14-21). Google Translate, relies on NN as they look at thousands and thousands of examples to give the best solution/result. By using the Encoder-Decoder model, MT “builds internal representions and generates predictions. The encoder (a model that can read in the input aka the sentence) stands for “what we should think and remember about what we just read” and the decoder “decides what we want to say or do” (CrachCourse #7, 2019). The NN converts the words into a form, numbers, vectors and matrices that the computer understands. Recurrent Neural Networks, have a loop that allows them to “reuse single hidden layer, which gets updated as the model reads on at a time and by training the model on which to predict next, the model waits for the encoder RNN and the decoder prediction layer. The RNN are long short term memory RNNs (LSTM-RNNs) that can deal with longer sentences (instead of just words) much better (CS Dojo Community, 2019). By doing this consistently, if the computer notices that the two words mean something similar, the model makes their vectors more similar and therefore can “predict” the word that will follow the next time it is asked to do so (CrachCourse #7, 2019). The E-D model, takes the words/sentences the RNN turns them into Vector (sequence to vector) and the decoder takes the vectors and the RNN turns it into the words of the other language (vector to sequence) (CS Dojo Community, 2019) However, this fails to address the complexity that comes with contextual meaning and understanding and can be limiting when we are dealing with longer sentences that have more than 15-20 words. Just because words have similar meanings doesn’t mean that they can necessarily be interchangeable in all contexts. (See blog post)

The solution that Google Translate has “come up with” is by replacing RNN with BidirectionalRNN which uses an attention mechanism between the encoder and the decoder and helps the computer know which words to focus on while generating the words for another language (CS Dojo Community, 2019). During the translation process, a – lets say- English sentence is “fed into” the encoder, “translated” (again, pun intended) into a vector which is then taken my the attention mechanism that correlates which -lets say- Greek word will be generated by which English words. The decoder will then generate the result of the translation (in Greek) by focusing on one word at a time as the words have been determined by the attention mechanism. In this specific case, Google translate actually uses 8 LSTM because “deeper networks help better model complex problems” and are “more capable of understanding the semantics of language and grammar” (CS Dojo Community, 2019). 

What does this data look like? Is it saved as words or as vectors? Are knowledge graphs shared across any type of machine/computer/software i.e. does google translate share its data collection with others? 

References 

Blog post from Prof. Irvine’s class 711: Computing and the Meaning of Code (Ironically enough I though of the same title for both of these, i.e. “take 2” on this one’s title) 

Crash Course Computer Science: Natural Language Processing (PBS). 

Crash Course AI, 7: Natural Language Processing

Thierry Poibeau, Machine Translation (Cambridge, MA: MIT Press, 2017).

How Google Translate Works: The Machine Learning Algorithm Explained (Code Emporium).

Wikipedia: Machine Translation,

Wikipedia: Computational Linguistics 

Wikipedia: Natural Language Processing 

Biases in AI

This was such an interesting topic to further dive into not only because it perfectly explains what I’d describe as one of today’s multi-used yet still “black-boxed” phenomenon of pattern recognition especially as it is applied to computer vision and images. Karpathy’s article doesn’t only highlight and break down the functions and uses of Convolutional Neural Networks but he has managed to depict through his findings how something so computerized can still be very much so human in terms of the societal biases it brings into play. 

“In machine learning, the aim is to fit a model to the data”, explains Alpaydin (Alpaydig, 2016, 58). Computer don’t just know what to do. Someone, a human, has to feed them with  directions and instructions in order for them to actually do something. The computer will follow whatever set of instructions is made for it by the human and execute the commands it has to. This means, that this human that has all of their opinions, biases, beliefs, experience, etc. is to encode into a non-human thing, the ability to execute commands based on human characteristics, capabilities, ways of knowing and understandings. Karpathy’s “experiment” shows exactly how there is biases in algorithms, especially his own, a topic I really looked into during my undergrad (one of Dr. Sample’s very looked into topic) and through my research on uses of ML and NLP in IPAs focusing on speech, language recognition and more. 

Karapthy explains how “a ConvNet is a large collection of filters that are applied on top of each other”. In these convolutional neural networks, “an aritifical neuron, which is the building block of a neural networks takes a series of inputs and multiplies each by a specified weight/number/characteristic and then sums those values all together (CrashCourse, #35, 2017)  To break it down, artificial neural networks, have artificial neurons that basically rake numbers in and spit more numbers out (CrashCourse, #34, 2017). You have the input layer, the hidden layers and the output layer. In the hidden layers, is pretty much where it all happens. It is where the computer sums the weighted inputs, the biases are applied and the activation function is also applied as this is computed for all the neurons in each layer (CrashCourse, #34, 2017). The deeper the neural net, the “weaker” or “stronger” the AI is. The NN can learn to find their own useful kernels/inputs and learn from those. The same was the ConvNets use stored information, banks of these neurons to process image data. 

As you run through them, convolutions happen over and over again as they run small filters and slide them over the image spatially to dig through the different layers of an image in order to find different features. This operation is repeated over and over again “detecting more and more complex visual patters until the last set of filters is computing the probability of entire visual classes in the image” (Karpathy, 2015). This is the part where we have come in and told the AI how to use these filters, when, where, what do we want out of the, etc. We train the ConvNets to know what to keep and what to emit by telling it what is in a way, good or bad, pretty or ugly, etc. Practice makes perfect, is a great saying to apply here as these neural networks will learn through re-inforced learning and by trial and error. The more data points you have the more information you can collect, which means that the more data you have the less uncertainty you also have about the classification, layering and choices made. However, since not all data points are equal and can’t be measure appropriately, the ML model can identify where the highest uncertainty is and ask the human to label the example, learning from those. Through active learning, the model is constantly synthesizing new inputs creating layer after layer until it reaches the wanted result and outcome (Dougherty, 2013, 3). 

For face recognition, the input layer is the image captures which is stored as pixels, defined by a color and stored as a combination of three additive primary colors, RGB, as we saw in our previous lessons as well. (Alpaydin, 2016, 65; CrashCourse #35). With biometrics we then get the ability to recognize and authenticate people by using their characteristics both behavioral and physiological. Of course, this also helps with training computers to recognize mood and emotions and not just one’s identity which trains them to learn, pick up and adapt to a human’s or their user’s mood and feelings.  

During the classification process, during the segmentation and labelling part, the image is separated into regions that are used for each particular task. The foreground which entails the objects of interest and the background which is everything else that will be disregarded. Labelling the objects then comes into play which obviously makes it easier for future use to immediately categorize or extract whatever needed from an image but ironically, we can even say that labelling in many cases, is foreshadowing the biases that can be found in algorithms. The following feature extraction is when characteristic properties of the objects come into play and distinguishes them/places them in a different category from objects they either share similarities or differences with and so forth… (Dougherty, 2013, 4-13). Further playing a role and testifying to how biases are created even in tech exactly because it is basically a reflection of societal biases, issues and human systems of classification. 

I couldn’t stop thinking about how much Karpathy’s experiment reminded me of how if a few years ago (and by few I mean even 2-3 years ago) if you Googled “beautiful girls” for a few scrolls the only photos with be those of generic (pun-intended – “generated”) white women because the algorithms identified as beautiful (honestly, not much has changed now either). A computer doesn’t know what is “pretty”, “ugly”, “good”, “evil”. Humans have inputed and labelled recognizable patters and standards of beauty further bringing to the surfaces, the racisms and biases that are very much so present in our world but also the underrepresentation of minority groups and BIPOC in tech. Even in Karpathy’s results, one can see the obvious majority of who are in these selfies. 

 

Based on his explanations of what was categorized as a good and bad image, I’d definitely would like to ask him what and how those distinctions where made. Also, how is a selfie of Ellie Goulding (famous singer) there if he supposedly through out and separated photos with either too many or not enough likes compared to others and people with too many or not enough followers as others? 

Based on his worst selfies, one of the criteria is “low lighting”, however, is it just the low lighting that is categorized as bas or is dark skin also included in that? “Darker photos (which usually include much more noise as well) are ranked very low”.  This also speaks to the issue of Snapchat or instagram filters and their inability to pinpoint and find features on people with darker skin in order to apply the filter on them. 

 

P.S. Check back in the future for a more updated list on cool articles and readings about biases in algorithms! Need to do some digging and go through my saved material and notes from previous years! 

References

Ethem Alpaydin, Machine Learning: The New AI. Cambridge, MA: The MIT Press, 2016.

Crash Course Computer Science, no. 34: Machine Learning & Artificial Intelligence

Crash Course Computer Science, no. 35: Computer Vision

Crash Course AI, no. 5: Training an AI to Read Your Handwriting

Geoff Dougherty, Pattern Recognition and Classification: An Introduction (New York: Springer, 2012). Excerpt: Chaps. 1-2.

Andrej Karpathy, “What a Deep Neural Network Thinks About Your #selfie,” Andrej Karpathy Blog (blog), October 25, 2015, https://karpathy.github.io/2015/10/25/selfie/.

Unicode: A innovation creating commonalities between the black boxes data encoding

Before diving into the more complex attributes of exactly how it is that you are able to read these letters, words and phrases that I am typing out on my computer right now – aka these “data types, we can briefly describe what we commonly understand by “data”, at least in the context of computing and coding. Professor Irvine defines; “‘Data’, in our contexts of computing and information, is always something with humanly imposed structure, that is, an interpretable unit of some kind understood as an instance of a general type. […] (T)o be interpretable, [data] is something that can be named, classified, sorted, and be given logical predicates or labels (attributes, qualities, properties, and relations)” (Irvine, 2021, 1). As we briefly touched upon in our last class as well, a “token” can stand for something else, something that can be represented as something, something that is immediately related and connected to something else. Data can be a token or tokens. “Data is inseparable from the concept of representation” (Irvine, 2021, 2). Data alone would not stand for anything if it didn’t actually represent something. Focusing on this context of computing and information, representation means a “computable structure” and is “defined byte sequences capable of being assigned to digital memory and interpreted by whatever software layer or process corresponds to the type of representation — text character(s), types of numbers, binary arrays, etc”. And simply put, we can also say that this is why “data” is considered to be of a more ‘higher esteem’ than “information”. I Imagine information as being the biggest mass of general, undefined, ‘unsupervised’, facts, clues, concepts, etc, and no matter what it is it can just exist, it can ‘tell’ us something, it can let us know of something but it doesn’t have the purposefully structure nature and meaningful existential representation that “data” consists of. 

A part of this data, are the data types we all know as texts, images, sounds, etc. If I send the words “Hello! How are you?” from my iPhone to someone with a Samsung, they will receive letter by letter, symbol by symbol, the same thing. If I copy and paste the same phrase from WordPress, to my notes, to a personal message on Facebook, to someone else on Whatsapp, to a chat room on Twitch, etc., the same exact message will appear once again. The reason for this is Unicode. Unicode is the international standard, an “information technology standar (Wikipedia, 2021), that has been creates and formatted in order for all computing devices and software applications to interpret the same representation throughout the world. “Unicode is thus the universal internal language of computing” (Irvine 2021, 5). It is the data for representing written characters aka strings, “of any language by specifying a code range for a language family and a standard bytecode definition for each character in the language” (Irvine, 2021, 3). The reason why we are able to read text on any device, emails, messages, etc, is because of Unicode. “Unicode is what is inside the black boxes of our devices with pixel-based screens and software layers designed for text characters and emoji” (Irvine, 2021, 5). Joe Becker, in August of 1988 in his draft proposal for this character encoding system explains why even the name matches as it is “intended to suggest a unique, unified, universal encoding” (Wikipedia, 2021). 

Some fun/interesting facts about Unicode (Wikipedia 2021; Wisdom, 2021):

  • total of 143,859 characters 
  • 143,696 graphic characters 
  • 163 format characters 
  • 154 modern and historic scripts, symbols, emojis 
  • current version: 13.0  
  • Unicode cannot run out of space. If it were linear, we would run out in 2140 AD! 

The reason why Unicode works as a standard for everyone is because these data standards cannot align with a specific system, software, platform, etc., but needs to be “independent of any software context designed to use them. They can work with any software or device because they reference bytecode units, which are independent data. “What we see on our screens is a  software ‘projection’ of the bytecode interpreted in whatever selected font style for ‘rendering’ the pixel pattern on screens” (Irvine, 2021, 4). How it comes all together is with the aid of The Unicode Standard which uses code charts for the visual representation, encoding methods, standard character encodings, reference data files, character properties and rules, etc., “provides a unique number for every character, not matter what platform, device, application or language” (Unicode Technical Site, 2021). Which if you think about it, it is pretty cool that we were all able to agree on something (of course, without getting into the complications, issues, biases, etc. that come with adopting Unicode), but in a cliche way – technology did bring us (almost) all together! For text processing, unicode translates to a unique code point, a number, for each character so it represents the character in a more general computing format where the visual representation of it, i.e. font, shape, size, etc., is taken care of by different software, unicode provides the meaning, the what it is (Unicode Technical Site, 2021).

Unicode used different types of character encodings, the Unicode Transformation Format (UTF), “an algorithm mapping from every Unicode code point […] to a unique byte sequence (the ISO/IEC 10646 standard uses the term “UCS transformation format)” (Unicode Technical Site, 2021). Most commonly used are UTF-8, UTF-16, UTF-32. UTF-8 is the byte-oriented encoding form, it is the dominant encoding used on the World Wide Web and even the first 128 characters are ASCII (American Standard Code for Information Interchange) which means that they also are under UTF-8 (Unicode Technical Site, 2021; Wikipedia; 2021). 

 

Crash Course Computer Science 

Since emojis have to be bytecode definitions to be interpreted as software context, All emojis must have Unicode byte definitions in order to work for all devices, software, graphic rendering. Updates with new emojis or some emojies will not be consistent from one device to the next or from one system to the next, i.e. sometimes the typical red emoji heart ❤️ (note: the irony that this heart emoji looks different on my iOS system than on WordPress) would show up as a smaller dark black heart or maybe a box with a “?” would appear in that emojis place if you hadn’t updated the version, etc., Is this due to non-updated bytecode definitions? Or is it because the software/system didn’t use/follow the ISO/IEC standards? Is this the reason why each company/program/software has their own “look”/style for each emoji because that is how it transforms/translates into from the Unicode CLDR data? Does the same apply for unreadable fonts as mentioned in the readings with the problem that arises with Unicode? 

I’d like to further look into the connection between Unicode and software stack design? How do they connect to each other and how does one symbol go through the “journey” of Unicode to the process of adopting whatever font, size, color it is given. 

 

References 

Irvine, Martin. (2021). “Introduction to Data Concepts and Database Systems.”

Crash Course Computer Science 

John D. Kelleher and Brendan Tierney, Data Science (Cambridge, Massachusetts: The MIT Press, 2018). 

Unicode Wikipedia 

The Unicode Standard 

Emoji – Unicode 

Unicode is Awesome – Wisdom 

 

What “meaning” to we find when we “meaning” signal-code transmission and information theory models

A re-occurring theme/discussion point through out our conversations and readings, has been the concept behind de-black-boxing tech and the reason behind that is probably due to the fact that we struggle to truly grasp what is behind these machines and how do we really end up getting the “information” that we do in the formats that we do. How does an image get so easily captured and transferred from reality to the initial shutter of the specific camera that took the photo, to a digitalized form that can be re-used for multiple purposes, over an infinite time period in a gazillion different ways. You can send it, post it, delete it, put a filter on it, put a bunch of filters on it and photoshop it. No matter what you’re doing, you have managed to take something that exists in the physical real world and converted it to its digitized representation and depiction aka form. 

“E-information theory supports our
whole symbolic-cognitive system by providing the techniques for using electrical
energy quantities in time for physical substrates of all human sign and symbol
structures (data representations)” (Irvine, 2020, 4). We are basically trying to imprint our human and cultural understandings, meanings and values into an electronic format where automated calculations and interpretations can take place in order to take the information that has been fed into the machine or system and “spit out” new representations and meanings that yet have an overall meaning used for communication and interpretation. Physical electronics and machines such as our computers or smartphones have been created in order to present and hold, in a way “carry” these data representations that have been translated from human symbol meanings to bits and bytes, aka data. (“Data: Binary structure for units that can be assigned a first level of meaning” (Irvine, 2020, 3). Information theory is considered to be what we call “designer electronics” as it represents, processes and transmits designed binary electronics (what we have fed into the computer and converted it into the “language” it understands) as an “engineering solution to a semiotic problem” that of “imposing human logical and values on electronic and physical media for creating meaningful patterns” (Irvine, 2020, 2). 

Indicating a state of existence, the signal-code-transmission model does not replace the meaning of whatever the representation is, stands for. “Meanings are not properties of electrical structure” (Irvine, 2020, 9). We can understand and conceptualize the message that we have received through the transmission because of the pre-supposed meaning or notion we have attributed to it. The same way a color blind personal will view the color blue on his screen differently than a person who isn’t colorblind. However, the system itself does not make distinctions between the two blues, those are made solemnly by the person at the receiving end who conceptualize and see the color differently. The transmission just gets the job done of “carrying” the “message”/transmission from one designer electronic to the next. As Shannon also explains in his theories and findings, that “what a relay passes onward from
one circuit to the next is not really electricity but rather a fact: the fact of
whether the circuit is open or closed” (Gleick, 2011, 18). 

Whatever is transmitted is what will be shown. The system/computer does not have actual concepts or true understanding of what is happening, said, painted, shown, depicted, stated, etc. on the receiving end of a signal. The system/computer have zero pre-supposed cultural notions or any sort of conceptual belief about the E-information, that is up to the user, the real human. Leading to why the information theory model works for everything digital and electronic but “not extending to models for meanings, uses, and purposes of our sign and symbol systems” since that is ultimately left to the task of the human user. In reality, information isn’t something tangible, we can’t see it and most times we cannot touch it, especially if we are talking about E-information (Irvine, 2020). As long as the system runs and works properly what we see are “the effects of information shaped and “communicable” for internal processing in the systems” (Irvine, 2020, 13) that take place in seconds as the order come from binary switches and other processes that do not contain any further meaning other than on/off, 1/0, etc. The meaning-making comes in a later stage that takes place in our human brains that can unfortunately or fortunately, not work as fast as a computer’s processor and systems and therefore need time to process, conceptualize and interpret meaning found in the data using our own human symbol systems and cultural notions. 

 

Sources

Gleick, J. (2011). The Information: A History, a Theory, a Flood. Bantheon Books: NY

Irvine, M. (2021). Introducing Information Theory: The Context of Electrical Signals Engineering and Digital Encoding.

Martell, C & Denning, P (2015). Great Principles of Computing. The MIT Press: London

Behind our familiarity with AI

A big part of “deblackboxing” the mystery behind computer is realizing that there isn’t something necessarily to de-blackbox. The gap between true understanding of computing and what goes on behind ‘closed screens’ is the complex arbitrary notions that we ourselves have given to something we have actually created. The truth behind it, is that it is not as unfamiliar as we think. If we think about it, all these designs, systems, software didn’t create themselves. Someone had to build them based on human knowledge, needs, desires, etc. In reality, they are only a reflection of our human day-to-day functions that we encoded to make our lives easier and faster, or at least that is the goal without including all the ethical, security, privacy, etc. issues that have rises over the years. “The action of computing comes from people, not principles” (Denning & Martel, 2015, 19). However, breaking down and highlighting these subparts of computing and systems in order to understand the information process and algorithms that guide them towards executing specific commands and demands. We use design structures and principles of computing to transform information, discover it, classify it, store it and communicate it, these “structures are not just descriptive, they are generative” (Denning & Martel, 2015, 15). The countless masses of information whether physical, digital or even conceptional have been overwhelmingly growing through out the years and scientists, coders, etc. have needed to find different and more sufficient ways to manage such matters but also “build systems that could take over human cognitive work” (Denning & Martel, 2015, 27) and as Morse had suggested; to “construct a system of signs which intelligence could be instantaneously transmitted” (Irvine, 2020, video)

Digging into what are these main concepts helps us realize that in reality computing and this black box isn’t so dark and mysterious after all. A simple duo of numbers, 1 and 0 have managed to create such a vast system of knowledge, storage and processing of information that have ultimately changed life as we know it forever. For example, just as human memory is crucial in conducting really any type of daily matter no matter how important or unimportant it can be. Similarly, computer, digitized and software memory is a crucial design principle for the functionality and existence of computes as we know them today and “the most sophisticated mechanism” (Denning & Martel, 2015, 23). However, in order to keep that memory and all of its functionalities safe, the concept of security came to play a major role in the computer’s system design principles as life slowly started taking a turn “online”, we had to find ways to secure privacy, individuality, etc. the same way we did in real life, online. Starting with time-sharing systems in the 1960s, information protection, ways to control access to confidential and private information, ways to file systems hierarchically to provide user customization and more, policies for computer operators ((Denning & Martel, 2015, 23), needed to be created in order for people to share the same familiarity and feeling of safety that they do in real life, virtually. 

 

Because of the aforementioned, two number usage, the “Y2K” problem arose highlighting the importance of danger in information vulnerability that can be found due to network sharing, the World Wide Web and more, database records, passwords, personal information, etc. can be easily accessed and uncovered if they want to be (Denning & Martel, 2015, 23-25). Machine Learning and Artificial Intelligence have made it possible to create for security purposes factors of authentication and identification. Biometrics, for example is the “recognition or authentication of people using their physiological and/or behavioral characteristics”, these can include “the face, […], fingerprints, iris, and palm [as well as]. dynamics of signature, voice, gait and keystrokes” (Alpaydin, 2016, 66). Even under these circumstances where technology has developed to such an extent where we can literally unlock our phones with our faces or walk through stores and office spaces while purchasing things and tracking location is rooms through facial recognition, to unlocking high risk information and privacy matters with your eyeball or finger-print, we can extensively comment and discuss the social and ethical issues that arise from such capabilities, showing exactly the idea that in reality all of these “ultimate-crypto-computer-sciency-too-hard-for-anyone-else-to-understand” myths, are truly just a reflection of very human selves onto something technological that we have created so extensively to the point were even our human biases, debates, prejudices, etc., have been unconsciously (or consciously) applied on/in to them. 

Denning & Martell, 2015 p.27-28 

“Automated cognition systems need not work the same way the human mind works; they do not even need to mimic a human solving a problem. […] The methods used in these programs were highly effective but did not resemble human thought or brain processes. Moreover, the methods were specialized to the single purpose and did not generalize.” 

So why do we alienated ourselves and are so concerned/scared about the development of tech, AI, computers, etc. when they can basically never be as intelligent and as advanced as the human cognitive brain and mind? 

 

References 

Alpaydin, E. (2016). Machine learning: The new AI. MIT Press.

Denning, P. J., & Martell, C. H. (2015). Great principles of computing. The MIT Press.

Irvine, M. (2020). Introduction to Computer System Design. 

 

Our World with AI

It is interesting to think about the conceptualization of Artificial Intelligence. Most of the readings, discuss the dystopian accountabilities that are attributed to AI since it was brought to the forefront of technology. However, people have always had this hidden or apparent fear of the deepening threat of technological innovation and the ultimate and detrimental effects that it could have on society and our world. When talking about cyborgs and robots people expected a type of technology that was cold, distinctive and far away. Yes a cyborg and/or a robot might re-install a lot of human attributes but at least it doesn’t necessarily resemble the human species. If anything, if a cyborg-robotic attack were to take place, the human race always had a dichotomized “us” and “them” side in order to defeat them. What AI reveals though, is that no one really imagined the form technology would soon take on. Maybe the familiarity with Siri’s voice and attitude, or Google’s Assistant ability to always know what you’re into, the fact that your phone always provides you with results and ads you have discussed with a group of friends, is what creates this bizarre feeling of acquaintance with this type of technology. Yet its complexity also creates the fear and misunderstanding that comes along with it. 

The independence found in this type of technologies, creates this “discourse and false assumptions around AI and computer systems” (Irvine 2021). Ironically, media culture has only deepened the misinformation issue with AI and enhanced this sense of a threatening dystopia (Boden, 2016; Wooldridge, 2020). In reality if more people were to truly understand AI and de-blackbox it, the desensitization towards AI would become obsolete. The success of AI as we know it today can only be attributed to the cumulative expansion and adaptation of various aspects of computation and computing systems that have taken place through out the years.  (Irvine, 2021; CrashCourse, 2019; Boden, 2016; Wooldridge, 2020). From the very early human symbol systems to complicated automated computing calculations, AI’s history is way closer to home and “humanity” than people often think; “Everything we do in computing and AI is part of a long continuum of extending the human symbolic-cognitive capacity for symbolic systems (from language to abstract algebra and digitally encoding multimedia), and designing technical media systems for and with those systems.” (Irvine, 2021, 9). Concepts and patterns that were created even thousands of years ago to improve and facilitated human life and development through its every stage are still being used and the reason behind why we are able to have the technology that is available for us today. 

I really enjoyed going through these readings as they were the perfect connection and delved into my focus on daily uses of AI. I found a lot of similarities in concepts and facts that were mentioned and that I had previously used for other class and research from previous semesters such as my own research for 505 and for Computing and the Meaning of Code (711). 

AI has the capacity to touch most aspects of our lives whether that is with its applications i.e. where can we find AI: everywhere! Its capabilities to adapt to a vast aspect of things from self driven cars to IPAs to sat-nav systems, AI’s “homologous design” (Boden, 2016) can be represented through a myriad of different ways, forms and even languages. It combines the “spirit” of humanistic psychology, philosophy and neuroscience with that of technology, binary, computational and symbol systems that together work towards enhancing and providing solutions for the real human world and our lives. 

Some more questions/comments: 

overselling AI? 

Marvin Minsky 

when overrating AI are we overrating our capacity? the computers/systems? or capacity of our binary symbol and other symbol systems? 

Where does the misinformation problem with AI really originate from? 

 

Sources: