Image source: The Intercept 2020, MIT Technology Review Volume 123 Number 3 [2, 6].

Maputi E.Botlhole, Spring 2020

Abstract:

The COVID-19 outbreak coerced communities across the globe to enter into unprecedented territory, which compelled multiple stakeholders to devise new social contracts in response to the health crisis. The authority of state actors became more pronounced in the instituting of lockdowns, enforcing of social distancing guidelines and the re-directing of resources to healthcare workers at the frontlines of the pandemic. Thus far, communities across the globe have adhered to terms of new social contracts in recognition of social benefit over individual benefit. However, the move towards digitization in an effort to assess the role of technology to help limit the spread of COVID-19 has reinvigorated public discourse on the need to weigh public health benefits against the loss of privacy. This is because countries across the world have repurposed mass surveillance technologies such as location tracking and facial recognition to conduct rapid contact tracing. Internet sovereigns such as Google and Apple also announced an unprecedented partnership to help public health agencies worldwide leverage smartphones to contain the COVID-19 pandemic. This paper will 1) de-blackbox digital location tracking for COVID-19 contact tracing through GPS and Bluetooth enabled devices 2) highlight the case studies of South Korea and Israel to illustrate that the properties of these mass surveillance technologies is not a problem; rather the challenge is with blatant disregard for privacy 3) discuss the Apple and Google joint effort as a model for privacy-preserving contact tracing 4) argue for stronger data protection laws to limit the possibility of function creep and surreptitious use of health surveillance data and technology post COVID-19.

Introduction:

COVID-19 like SARS and Ebola is a zoonotic disease that is believed to have originated from a wet market in Wuhan City located in Hubei province, China [6]. The first case or “patient zero” of COVID-19 is unknown because reports indicate that in mid-November of 2019, whistleblowers such as the late Dr.Li Wenliang were silenced by local police, and accused of “spreading rumors” after disseminating information about a SARS-like virus via WeChat [7]. The suppression of information that could’ve been useful in the containment of the virus; coupled by the targeting of healthcare workers through instant messaging platforms such as WeChat created an antagonistic stance on the role of surveillance technologies in this global health crisis. The idea that the technical underpinnings of surveillance technologies that were used to actively censor doctors, could be repurposed for case identification and contact tracing during COVID-19 continues to be a sensitive and controversial issue. However, the argument is that the properties of these mass surveillance technologies is not a problem; rather the challenge is with antagonistic information control, blatant disregard for privacy and undermining of civil liberties.

Image source: Citizen’s Lab Report About How Censorship Works on YY and WeChat [8].

The creators of WeChat have since adapted censorship methods such as optical character recognition for content moderation to debunk misinformation and to counter the spread of disinformation on COVID-19. On the server-side, the surveillance method of optical character recognition works by extracting text from an image and censoring certain blacklisted text [8]. In addition to censorship, contact tracing is a methodology that has been credited with helping to limit the spread of other zoonotic diseases such as Ebola. In the public health sphere, contact tracing has been done manually by deployed healthcare workers. According to the World Health Organization, contact tracing is “close contact with someone who is infected with a virus, such as the Ebola virus, are at higher risk of becoming infected themselves, and of potentially further infecting others [10].” The WHO breaks down the monitoring process into 3 basic steps, verbatim:

1. 1. Contact identification: Once someone is confirmed as infected with a virus, contacts are identified by asking about the person’s activities and the activities and roles of the people around them since onset of illness. Contacts can be anyone who has been in contact with an infected person: family members, work colleagues, friends, or health care providers [10].
   2. Contact listing: All persons considered to have contact with the infected person should be listed as contacts. Efforts should be made to identify every listed contact and to inform them of their contact status, what it means, the actions that will follow, and the importance of receiving early care if they develop symptoms. Contacts should also be provided with information about prevention of the disease. In some cases, quarantine or isolation is required for high risk contacts, either at home, or in hospital[10].
   3. Contact follow-up: Regular follow-up should be conducted with all contacts to monitor for symptoms and test for signs of infection[10].

Giff source: TIME, Coronavirus Brief [11].

Public healthcare workers have been conducting contact tracing for years with a robust workforce needed to break disease transmission chains; nowadays the capabilities of surveillance technologies offer an opportunity to bolster this practice. China as ground zero of COVID-19 utilized the government’s evolving algorithmic surveillance systems for contact tracing as a response to the health crisis [12]. The Chinese government introduced a “Close Contact Detector” mobile application that uses big data gleaned from the database systems of public health authorities, the Ministry of Transport, China Railway, China’s aviation authority and mobile data to notify users if they had been in close proximity to people who had tested positive for COVID-19 [13]. The users of the “Close Contact Detector” app were required to register with a phone number and ID number in order to see if they had worked, lived or traveled with a person confirmed to have tested positive for COVID-19 within the past 14 days of contact [7]. The “Close Contact Detector” allowed a central database analyzed by an artificial intelligence algorithm to collect data on user movement and coronavirus diagnosis. The user interface then displayed a green, amber or red code to relax or enforce restrictions on movement [10].The smartphone application had a plug-in to WeChat and other mobile applications hence China was able to garner engagement metrics of 130 million recurrent users within the first month of launching the mobile application [10]. The government was also in a position to leverage the large volumes of collected data because China was already a surveillance state prior to the global health crisis [12]. 

Video source: China Central Television News Agency (CCTV – YouTube Channel). 

Critics and activists continue to question China’s “successful” deployment of mass surveillance technologies to limit the spread of COVID-19, due to the belief that the repurposing of the mass surveillance technologies is also an ongoing attempt to rewire people’s sensibilities about data privacy, location tracking and aggressive collection of personal data [1]. Mass surveillance technologies present a possibility that was not available during prior pandemics, and China is one example of how a digital location tracking ecosystem enabled by smartphones, big data and cloud computing could curb the spread of COVID-19. The role of cross-sector partnerships is also important as evidenced by a case in Mexico, wherein a peer-to-peer platform such as Uber was able to share rider data with government authorities to trace the route of an infected tourist [14]. Furthermore, Uber and the government authorities notified 240 other users who had taken rides with the same driver [14]. The mapping of people’s movements through their digital footprints for purposes of contact tracing can be done in a responsible and privacy preserving manner;  the upcoming sections of this paper will elaborate more on this.

As COVID-19 spreads across the globe, more and more governments are repurposing mass surveillance methods and technologies to focus on public health outcomes. To date, the world has 3.2 million confirmed cases of COVID-19 and mass surveillance technologies have been adopted and repurposed by at least 30 governments  [15, 16}. Smartphone location data is the most popular for supporting contact tracing and enforcement of individual quarantines [15]. The properties of these mass surveillance technologies are not a problem and are much needed tools to assist public health workers who are at the front-lines of the pandemic.

Image source: OneZero 2020 [15].

De-blackbox: Smartphone Location Data

Various options exist for tri-sector actors to develop location tracking applications as part of contact tracing and/or quarantine enforcement in the fight against COVID-19. These are surveillance technologies that could be applied as part of data-driven containment strategies that make use of smartphones. Potential technologies include GPS, Bluetooth, cellular location tracking and QR codes – this section of the paper will de-blackbox GPS and Bluetooth technology, and also highlight the role of Big data. The capabilities of the aforementioned technologies have their own advantages and disadvantages when it comes to the practicalities of contact tracing, quarantine enforcement and privacy concerns involved in smartphone location data [17. For instance, the technical requirements of a contact-tracing app that uses cellular location data or GPS would need to operate at close range in order to accurately determine whether the user’s smartphone has come into close enough contact with the smartphone of an infected person. GPS would only be able to provide accuracy down to the nearest 10 to 20 meters and would be less effective inside buildings [2].

Cellular location data is known to be even less precise and a QR code-based system would be useful for collecting a user’s data record on places visited. The QR code-based system would be less effective for determining whether or not someone has been in close contact with an infected individual nor for determining adherence to social distancing [17]. The most promising solution is to repurpose Bluetooth technology on a smartphone to operate on a system of individual anonymous codes for contact tracing [17]. For instance, if someone were to test positive for COVID-19 their anonymous code would be sent to other relevant users and those who had been in close proximity to them could be warned [17]. 

Image source: A schematic of app-based COVID-19 contact tracing [10].

The design behind the schematic of app-based COVID-19 contact tracing is based on smartphone functionality and an algorithm from epidemiological principles [10]. The app would keep a temporary record on closeness of events between individuals and send out an instant alert to self-isolate if a recent close contact tested positive for COVID-19 [10]. The algorithmic functionality of the app would be manually overridden in cases where public health workers had additional information to intervene in a particular case. The main push for such an application would be to replace a week’s work of manual contact tracing with instant signal transmission, reduce delays between positive COVID-19 case confirmation and notification of contacts, preserve the anonymity of infected individuals and refine the app to be more informative on high-risk areas to avoid [10]. These repurposed surveillance technologies are promising, however, they operate within a digital tracking ecosystem that excludes individuals without smartphones or bluetooth supported feature phones. Individuals that are often at the bottom of the pyramid, but perhaps the adaptation of surveillance technology to fight COVID-19 would free-up public health resources necessary to address the needs of individuals in under-resourced communities. 

GPS AND SMARTPHONE LOCATION TRACKING: 

“In places like India with smartphones, there’s an app now for women if they’re in a violent situation, they can press one button. They’ve given their cell-phone number to five trusted friends, and right away their GPS location goes out: “Here I am.” Melinda Gates, Interview with Jessica Grose (2016).

Identifying a person’s current, physical location by obtaining GPS data from their smartphones or other GPS-enabled devices is known as geotracking. GPS is an abbreviation for Global Positioning System: a radio navigation system that has become an indispensable part of modern life and found in cars, machinery and smartphones. GPS is owned by the United States government and overseen by the country’s Air Force [20]. Smartphones have a GPS receiver chip that uses radio waves from at least four satellites to provide location and time information to any software that needs to use it [20]. There are 28 satellites in orbit that are dedicated to geolocation and a phone’s GPS receiver uses data from signals to triangulate a person’s location and time [20]. The satellites have an internal atomic clock that sends time-coded signals on a specific frequency, and the fourth signal in triangulation is used to determine altitude hence a smartphone receives geolocation data on a map with three signals [20]. The smartphone GPS receiver chip requires unobstructed view of multiple satellites in order to gather location and time data from the strongest satellite signals [20].

Assisted Global Positioning System (AGS) adds cellular location data to assist geolocation from your phone [20]. The phone carrier “pings” cell towers and triangulates location based on the strength of the GPS enabled phone and tower [20]. AGS is useful when individuals want location whilst on the move and when obstructions such as tall buildings block GPS receiver chips from getting location and time data [20]. Countries such as Norway are already experimenting with GPS data for contact tracing in response to the COVID-19 outbreak [10]. GPS capabilities have limitations in terms of difficulty to anonymize, accuracy that is dependent on distance, power requirements due to satellites that run on rechargeable batteries and whether or not there’s an obstructed view [20]. However, from the quote by Melinda Gates, it is evident that GPS technology can be repurposed not only to ensure the safety and security of women in India; the technology can also be repurposed for contact tracing necessary to combat the spread of COVID-19. 

Image source: IUNERA 2020 [19].

BLUETOOTH AND SMARTPHONE LOCATION TRACKING:  

Image source: Medium 2019 [21].

Bluetooth technology can be found in homes, cars and smartphones – it has evolved to be an important part of the Internet-of-Things ecosystem as a low energy connectivity option. Bluetooth is a short-range wireless communications technology which was developed by the Swedish telecommunications company Ericsson [22]. The technology replaced cable connection and enabled electronic devices that are equipped with Bluetooth connectivity, such  smartphones, to use radio waves to transmit information between devices and over a short distance [23]. Smartphones are built with a radio frequency transceiver that is already installed. The chip has to be turned on by a user in order to emit Bluetooth radio waves to allow for communication between devices over a short-range network called a piconet [23]. The piconet is a network of Bluetooth connected devices that use a frequency-hopping pattern for transmission or reception of packets between Bluetooth enabled devices  [22]. Piconet are automatically established hence Bluetooth enabled devices can “remember” each other if a pairing had been previously occurred. 

The design principle behind Bluetooth wireless communication is known as “inquiry” and “inquiry scan,” which means that when the Bluetooth is turned on, the scanning device listens to discover known frequencies for devices that are actively inquiring [23]. After the inquiry is received, Bluetooth pairing occurs when the scanning device sends a response with information needed by the inquiring device [23]. This is the Bluetooth technology that has the potential to enable anonymized location tracking for contact tracing in the fight to limit the spread of  COVID-19.

Image source: IUNERA 2020 [24]

The schematic above is of a mobile application concept for a COVID-19 privacy compliant Bluetooth app. The schematic shows how direct Bluetooth data exchange would be strengthened by blockchain technology to allow for a high grade of privacy and to avoid potential corruption to ensure transparency and no single party control [24]. The short-range Bluetooth communications still has its limitations: battery drain may become an issue and devices generally needs to stay within 10 feet to work properly [23]. However, when compared to other options, most countries such as Germany, Singapore, Norway and Indonesia seem to be coming to the conclusion that Bluetooth is their best bet [15].

The concept of Bluetooth low energy and voluntary contact-tracing network has also been praised by advocates of the General Data Protection Regulation due to the characteristic of a smartphone application that would operate on a system of individual anonymous codes [25]. If someone were to test positive for COVID-19 their anonymous code would be sent to other relevant users, so that those who had been in close proximity to them could be warned [17]. No GPS location data or personal information would be recorded and users would voluntarily opt-in to use the application.The smartphones users who would opt-in and choose to install the COVID-19 app would generate personal keys via Bluetooth. The personal keys would be long strings of letters and numbers unique to each smartphone [17]. The long strings would in turn generate randomized temporary contact numbers and use low-energy Bluetooth to communicate those temporary numbers with each other when nearby [26]. Signal strength could provide a measure of how close the users’ phones are to assess the risk of COVID-19 transmission when people crossed paths [27].

Image source: BBC Technology 2020 [27]. 

The repurposing of mass surveillance technology for contact tracing in the form of GPS or Bluetooth enhanced digital tracking would also provide valuable data to further support disease modeling efforts and predictions about the flow of the COVID-19 pandemic [28]. In Taiwan, big data played a crucial role in limiting the spread of COVID-19. The country’s National Health Command Center integrated its national health insurance database with its immigration and customs database to create big data for analytics [29]. This allowed the government to use all available data to conduct case identification and real-time alerts based on travel history during clinical visits [29]. The success in Taiwan indicates that big data and outbreak analysis form part of smart prevention capabilities necessary for an integrated coronavirus control strategy. All the data that would be collected from digital tracking could help healthcare systems to create models that predict hotspots and inform decisions on the best preparation and response to the global health crisis. 

Mini Case Studies

Image source: The Intercept 2020 , OneZero 2020 [14,15]. 

SOUTH KOREA: DATA PROTECTION LAWS AND DISASTER RESPONSE

South Korea is a country with expansive data protection laws that are covered under the Personal Information Protect Act (PIPA) [35]. The citizens of the country have the right to be forgotten and can pursue legal recourse if an entity were to collect any information that could be used to identify specific persons. South Korea also developed an information and communications infrastructure needed to fight disaster from geopolitical conflict with its fraught neighbor North Korea. The ICT infrastructure consisted of mass surveillance technologies that could be repurposed to respond to the COVID-19 outbreak. The government and health administration authorities developed mobile applications that tracked the movements of citizens, and the applications had adoption rates of 90% [35].  The Korean Center for Disease Control and Prevention (KCDC) also installed drive-through tests, additional CCT camera’s for facial recognition, body sterilizers to spray through commercial areas and venues for thermal scanners to test citizens [15]. South Korea’s response was largely informed by the country’s playbook on response to anti-terrorism drills that involved mass surveillance and distribution of medicines in large scale to the public [35]. The country was able to successfully implement mass surveillance technologies without public outcry because the existing data protection laws such as PIPA quelled suspicion of function creep by the government. South Koreans also have legal recourse if their personal data were to be used surreptitiously or pivoted from health outcomes to commercial outcomes post COVID-19.

ISRAEL: UNPRECEDENTED USE OF SECRETLY GATHERED DATA 

Israel authorized the country’s Internal Security Agency: Shin Bet, to collect cellphone metadata to combat the COVID-19 outbreak [36]. The data would be used to retrace the movements of people who had been infected with COVID-19, and to notify those who would need to be placed under quarantine. This was met with opposition because civil society groups in the country hadn’t been informed of a legislative framework that allowed the security agency to amass data, and fears intensified about the targeting of Palestinians. It is reported that Shin Bet has been collecting metadata since 2002 under Israel’s Telecommunications Law and Israeli Security Agency Law [36]. Both laws allowed the agency to gain access to the facilities and databases of cellular networks and for the agency to request for data transfers [36]. In addition to data collection through transfers, Palestinians that work in Israel have been required to download a mobile application that enables Shin Bet to access their cellphones, geolocation information, etc. [37]. Currently, data protection laws for Palestinians do not exist and no safeguards have been installed to determine how Shin Bet uses the data, whether or not the data is deleted, who else has access to the data collected and under what conditions. Furthermore, Palestinians have no choice but to download the digital tracking application because it is tied to their ability to renew their work-permits in Israel due to the closure of the office of Coordination of Government Activities in the Territories – the agency that would issue and renew work permits under normal circumstances [36]. The repurposing of mass surveillance technologies for contact tracing in Israel has been viewed with scrutiny because it is an effort led by a security agency rather than a health agency, and there is no legal recourse for Israelis and Palestinians who might want to ensure that their data is not used for other purposes.

Privacy-Preserving Contact Tracing 

Apple and Google announced a joint effort “to help public health agencies worldwide leverage smartphones to contain the COVID-19 pandemic [30]. This is an unprecedented partnership that would enable the tech giants to combine their assets and to have access to 3.5 billion iOS and Android devices from across the globe [5]. The use of Bluetooth Low Energy technology and an option for users to opt-in has been touted as “Privacy-Preserving Contact Tracing” for COVID-19 and this is a graphic illustration of how the method would work [31] :

Image source: New Atlas 2020 [31]. 

The Apple and Google initiative will work by harnessing short-range Bluetooth signals and follow a privacy-preserving protocol called PACT for private automated contact tracing [32]. PACT is a technical standard/specification developed by the Massachusetts Institute of Technology and recommended by the United States Center for Disease Control and Prevention [33]. PACT is open, interoperable and can be deployed to any smartphone and will enable Apple and Google to balance efficacy while maintaining the privacy of users that opt-in [30]. The Apple and Google system will be rolled out in two phases:

1. 1. Deployment of an application programming interface (API) released to allow smartphone application developers to start designing contact tracing apps. At this phase, users would have to download an application designed by their local authority.
   2. A broad iOS and Android system update to integrate the contact tracing tool into each smartphone’s operating system – users can opt in and activate contact tracing through a menu option in phone settings.

Video source: PACT, Massachusetts Institute of Technology 2020 [33].

Decentralization is a key aspect of the Apple-Google solution – the match identification process would take place locally on the smartphones of users. The decentralization would allow users to store Bluetooth contact logs in the form of anonymized identifiers on their smartphones for up to 14 days [32]. The contact logs would be compared to the anonymized identifiers of other users pulled from a provider hosted application server based on a history of proximity [32].  The voluntary opt-in option might limit adoption of the Apple-Google solution, however the Bluetooth plan unlike GPS wouldn’t track the physical location of users, and only pick up the signals of nearby phones at 5-minute intervals [30]. Apple and Google hope that others will see benefits of adopting their approach which has user privacy and security baked into the design of the Bluetooth Low Energy technology for contact tracing.

The Need For Stronger Data Protection Laws

The repurposing of mass surveillance technologies to fight COVID-19 has raised new concerns about privacy and the trade-offs between health, community well-being, and individual rights [6]. The adaptation of these methods for contact tracing has reinvigorated public discourse, leading many to ask: Does the benefit of the surveillance outweigh its costs to privacy, speech, and equal opportunity? Countries such as China have a governance model that is notorious for bypassing the need for user consent and a lack of data protection rights for citizens [12]. On the other side, European countries have chosen to operate within the universal human rights framework to protect the rights of citizens through laws such as the General Data Protection Regulation (GDPR) [25].

Which approach will prevail on privacy, the uses of personal data and algorithmically enhanced decision-making at a time when the COVID-19 virus is quickly spreading across the globe?

Once again, the properties of these mass surveillance technologies are not a problem, as a matter of fact, the technologies offer an opportunity for countries to track and curb the spread of COVID-19 with speed and accuracy not possible during prior pandemics [14]. The European Union has already shown support for the Apple-Google initiative and praised it for its “privacy by design” approach, due to the use of anonymized Bluetooth identifiers, decentralization and a 14-day expiration on the storage of users digital location track records [32]. Countries that require adherence to GDPR, such as Finland, have already introduced measures to ensure that the private sector is aware that data collection at this time should be led by public health experts. Below is an infographic on legislation concerning the processing of health data and employee data [34]. 

Image source: NIXU, Cybersecurity 2020 [34].

The early days of the COVID-19 outbreak were marked by tight censorship of health workers in Wuhan China, and automated content moderation by the local governments to control the narrative about the pandemic. This kind of censorship sparked debate on free speech and information control [7]. The country did not address such issues but rather repurposed its mass surveillance capabilities for contact tracing, which worked to flatten the curve of infection and transmission but failed to balance public health and civil rights [7,8,]. The intrusive data collection methods of governments and mobile data brokers are well  documented; and are often carried out on an uninformed and non-consenting populace. There are several examples from around the world on how in times of crisis, emergency powers can outlive emergencies [14]. For instance, another example is the USA Patriot Act – although it was intended to prevent terrorist attacks it was accused of infringing civil liberties. It got repealed through the 2015 Freedom Act and it took 14 years for the U.S government to install restrictions on how surveillance programs could be used [14]. The rest of the world has an opportunity to look at these examples and to implement consent-based surveillance and “privacy-by-design” approaches during this global health crisis. Currently, it is civil society groups and GDPR campaigners who are pushing for a framework that could install safeguards and enforceable rules to limit the use of the collected data to the limited purpose of COVID-19 contact tracing.

Conclusion

Different countries across the world are repurposing mass surveillance technologies in an effort to fight the COVID-19 outbreak. The adoption of anonymized Bluetooth technology for contact tracing and digital surveillance is evidence that the properties of these technologies is not a problem. Privacy-by-design and privacy-preserving contact tracing that is led by public healthcare agencies from across the globe, such as the Korean Center for Disease Control and Prevention or the National Health Command Center in Taiwan, have led to high adoption rates of mass surveillance technologies repurposed for health outcomes. In contrast, there is intensified public outcry in countries such as China and Israel where the repurposing of mass surveillance technologies in response COVID-19 has been led by security agencies and law enforcement. Furthermore, civil society groups are concerned about function creep and the undermining of civil liberties due to the lack of safeguards to ensure that data collected during COVID-19 is limited to use during the global pandemic. It is without doubt that the world will have a great deal to reflect upon after the COVID-19 global health crisis, and more will be written on the best models for the repurposing of mass surveillance technologies to fight future pandemics.

References:

[1] Liza Lin and Timothy Martin (2020). “How Coronavirus is Eroding Privacy,” The Wall Street Journal.

[2] Sam Biddle and Lee Fang (2020). “LOCATION,-TRACKING FIRM HELPING TRUMP GET REELECTED NOW WANTS TO CASH IN ON CORONAVIRUS,” The Intercept. 

[3] Frank Bajak and Matt O’Brien (2020). “Apple, Google to harness phones for virus infection tracking,” Associated Press (AP News). 

[4] Byron Tau and Michelle Hackman (2020). “Federal Agencies Use Cellphone Location Data for Immigration Enforcement,” The Wall Street Journal. 

[5] The Economist (2020). “App-based contact tracing may help countries get out of lockdown.”

[6] Genevieve Bell (2020). Volume 123 Number 3, Issue: Navigating a World Reshaped By COVID-19,”The Benevolent Panopticon”, page 16. MIT Technology Review.

[7] China Media Bulletin 142 (2020). “China Media Bulletin: Coronavirus-era repression, propaganda, censorship, surveillance and more (No. 142),” Freedom House.

[8] Ryan Broderick (2020). “The WeChat App Messaging App Has Been Censoring Coronavirus Content Since January,” BuzzFeed.News.

[9] World Health Organization (2017). Contact Tracing. “What is contact tracing and why is it important.”

[10] Ferretti, Luca; Wymant, Chris; Kendall, Michelle; Zhao, Lele; Nurtay, Anel; Abeler-Dörner, Lucie; Parker, Michael; Bonsall, David; Fraser, Christophe (2020-03-31). “Quantifying SARS-CoV-2 transmission suggests epidemic control with digital contact tracing”.

[11] Alice Park (2020). “Here’s How Scientists and Public-Health Experts Recommend the U.S. Gets Back to ‘Normal,” TIME.

[12] Anna Mitchell, Larry Diamond (2018). “China’s Surveillance State Should Scare Everyone,” The Atlantic.

[13] South China Morning Post (2020). “How big data helps China track the coronavirus…and its people.”

[14] Sam Biddle (2020). “PRIVACY EXPERTS SAY RESPONSIBLE CORONAVIRUS IS POSSIBLE,” The Intercept. 

[15] Dave Gershgorn (2020). “We Mapped How the Coronavirus Is Driving New Surveillance Programs Around the World,” OneZero.

[16] WorldOmeter(2020). COVID-19 Coronavirus Pandemic, Accessed on April 30th, 2020.

[17] Vittorio Bertola (2020). “Can a coronavirus tracking app be both effective and privacy-centric?” VentureBeat. 

[18] Your Dictionary (2020). “Geotracking – Computer Definition,” Accessed on April 30th, 2020. 

[19] Tim (2020). ” The complete guide to Coronavirus geotracking Apps and Time Series Databases Analysis,” IUNERA. 

[20] Jerry Hildenbrand (2018). “How does GPS work on my phone?” Android Central. 

[21] Chirs (2019). “Bluetooth: How it works and why we still need it,” Medium. 

[22] Scientific American, Tech (2007). “How does Bluetooth work?” 

[23] Jeanette Ferrara (2016). “How do Bluetooth devices work?” ScienceLine. 

[24] Tim (2020). “Top 5 ways how smartphones and Big Data help to fight the Coronavirus pandemic,” IUNERA.

[25] Russell Brandom and Adi Robertson (2020). “Apple and Google are building a coronavirus tracking system into iOS and Android,” The Verge. 

[26] John Toon (2020). “Will Smartphones Help Us Keep COVID-19 Under Control?” Georgia Tech, News Center. 

[27] Leo Kelion (2020). “Coronavirus: Apple and Google team up to contract trace Covid-19,” BBC Technology. 

[28] Bernard Marr (2020). “The Vital Role of Big Data In The Fight Against Coronavirus,” Forbes. 

[29] Beth Duff-Brown (2020). “How Taiwan Used Big Data, Transparency and a Central Command to Protect Its People from Coronavirus,” FSI News, Stanford University.  

[30] Frank Bajak and Matt O’Brien (2020). “Apple, Google to harness phones for virus infection tracking,” Associated Press (AP News). 

[31] Rich Haridy (2020). “Google & Apple join forces to develop smartphone COVID-19 contact tracing,” Mobile Technology, New Atlas. 

[32] AppleInsider Staff (2020). “CDC contact tracing criteria give nod to Apple-Google approach,” AppleInsider. 

[33] Private Automated Contact Tracing (2020). “PACT,” Massachusetts Institute of Technology. 

[34] Tuisku Sarrala (2020). “INFOGRAPHIC FOR EMPLOYERS: COVID-19 & PRIVACY,” Nixu Cybersecurity.  

[35] Ravi Aron (2020). ” Combating COVID-19: Lessons from Singapore, South Korea and Taiwan,” Technology, Wharton. University of Pennsylvania. 

[36] David M. Halbfinger, Isabel Kershner and Ronen Bergman (2020). ” To Track Coronavirus, Israel Moves to Tap Secret Trove of Cellphone Data,” The New York Times. 

[37] Nir Hasson (2020). “Amid Coronavirus Crisis, Israel Tells Palestinians to Download App that Tracks Phones.” HAARTEZ. 

 

 

Image source: Fingent 2019

The Triple Bottom Line: Multi-Benefit Application of AI in the Healthcare Industry

By: Alie Fordyce

Abstract

         Artificial Intelligence (AI) is a powerful technology that has the potential to provide positive societal change. There is major concern that its growing capabilities could lead to unintended consequences or be misused by bad actors that threaten its development.  However, considering the significant opportunity that AI represents to offer multi-benefit application across all industries supports the continued development of suitable AI safeguards and regulation to pave the way for its ethical use promoting a sustainable future. Focusing specifically on AI and the health care industry in this paper, the two main suggestions for AI’s application in healthcare are the following: 1) streamlining healthcare systems and drug development to reduce costs and 2) making big data more meaningful and accessible for healthcare practitioners to better patient care, medical processes, and reduce cost/increase resource efficiency.

1  Introduction

1.1 The Triple Bottom Line

The triple bottom line (TBL) is a sustainability framework for evaluating business investment that values social and environmental factors on an equal playing field with economic profitability. The TBL factors are commonly referred to as the three P’s: people, planet, and profit (University of Wisconsin Sustainable Management n.d.). Under the 3 P framework, economic decisions are a means to an end in creating an effective economic strategy while also striving to fulfill humans needs within ecological constraints (Sustainably Illustrated 2014). John Elkington, who first coined the phrase “triple bottom line” in 1994, fine-tuned its meaning in 2018 in order to elevate it from being a simple accounting tool (Elkington 2018) to become a holistic purposed-based framework. He reminds us that success isn’t (or shouldn’t be) simply a measure of profit, but rather should balance social wellbeing and the health of the planet as equally important elements in our long-term development activities (Elkington 2018). This is even more important now, as recognition of the need for sustainability has not yet reduced the rate of environmental degradation or helped to mitigate increases in the wealth gap. The original TBL concept has been watered-down by the business community to promote what seems to be a balancing act of the three Ps but in actuality favors commercial profit.  This watering down means that the TBL has so far failed to reshape our ‘hard-wired cultural problem’ and only disguises a retention of profit-motive dominated ethos (Elkington 2018). Because of the lack of progress toward a balanced framework, Elkington reignited the original conception of the TBL to provoke deeper thought about the future of capitalism (Elkington 2018).  The reinforced TBL pushes for a transformation of capitalism as a whole, requiring a complete system change (Elkington 2018). This shift requires a material change in objectives, new innovative approaches and policy supported TBL metrics that reflect a change of attitude to drive dramatic change in the trajectory for a sustainable balance between society, environment and profit. Without valuing the three on an equal playing field, the fear is that the betterment of one (e.g. economic profit) will come at the detriment of other sectors, hence pushing an agenda to demonstrate that focusing on all three sectors – the TBL – will result in a benefit to all. 

Video source: Sustainability Illustrated

Image source: Matt Chase/Harvard Business Review

1.2 Why AI? Why Healthcare?

Elkington reported in 2018 that the sustainability sector was approximately $1 billion in annual revenues globally, but that it was primed for high growth in the near future (Elkington 2018).  Elkington’s sector growth estimate is predicated on the idea that ignoring sustainability will result in increasing cost as social and environmental degradation resulting from current approaches will begin to significantly negatively impact economic potential. In fact, Elkington et al. estimated that the sustainability sector could become a $12 trillion a year market by 2030 (Elkington and Roberts 2017). This enormous economic potential combined with the promise of beneficial social, environmental, and positive economic outcome represents an investment opportunity that should command the attention of business, government, and social institutions intent on setting us on a path of a sustainable future.  However, the difficulty in assessing cause and effect within the TBL framework, along with the complex interactions between inputs and outputs in economic, social, and environmental outcomes, means we need innovative new ways to guide evaluation, investment and prioritization of efforts in building a sustainable future.  Artificial intelligence (AI) offers a powerful approach that can support Elkington’s call for innovative thinking to help us pursue holistic, balanced, and effective solutions for a livable future.

Healthcare is an example of an industry in need of disruptive thinking to counter unsustainable cost increases from the needs of an aging population, an explosion in chronic disease, and increasing environmental-linked health challenges like antibiotic resistance and proliferation of infectious disease. Further, the lack of availability of basic healthcare for the poor is an additional moral issue warranting investment to improve the lives of millions.  Effective solutions to these issues offer strong return potential for investors while simultaneously offering positive benefits to society:

“Lack of access to affordable and quality healthcare products and services is an acute problem among the bottom-of-the-pyramid (BoP) population in emerging markets, the majority of which are disproportionately affected by increasing rates of chronic disease, high mortality rates from infectious disease, traffic accidents, and maternal and child deaths… At the same time demand for healthcare products and services (and customer willingness and ability to pay for them) has increased as a result of a growing middle class and better awareness of health and healthcare.” (Haile-Mariam and Spector 2017, 4)

Considering the economic, societal and personal value placed on good health, combined with the absolute scale and the complex factors affecting healthcare decisions, AI is a tool that offers potential to help the healthcare sector innovate for social good and economic return. Some impact investors have already shown interest in the healthcare sector because they see “significant opportunities to support innovative, low-cost models that make healthcare more accessible and affordable while also benefitting from widening gaps between healthcare demand and supply in emerging markets” (Haile-Mariam and Spector 2017, 4). The primary areas of investment within the healthcare sector of emerging markets, to date, are in electronic patient records, telemedicine, delivery services, product manufacturing/distribution, and health insurance. This leaves existing gaps in access to quality data, integrating impact goals with that data and the business decision-making processes (Haile-Mariam and Spector 2017, 8). AI, as an approach to tackling the most pressing global issues, has the potential to be of significant value in filling this gap because of its ability to process large amounts of data finding cause and effect relationships that can lead to new approaches and solutions.

The body of this paper is divided into two parts: the first demystifies AI (on a general level), its use, and its challenges. The second part analyzes the healthcare industry, specifically the gaps where AI could enhance its potential and reach.

2  Demystifying AI

2.1 AI: Past and Present

In the recent decade, AI has garnered significant interest and its use is becoming increasingly widespread. Yet, there are valid concerns about the risk of using AI in our everyday lives, relative to issues including: privacy, use of AI-based insights to benefit the profit-seekers rather than general society and the fact that AI’s power is evolving faster than regulatory policies to mitigate risk.  It’s important to first unmask AI in order to understand its potential in more mainstream social impact usage. AI has become a tool to automate large-scale tasks: online social media content being filtered to fit a user’s preferences or Google using analytics to optimize travel directions or tailor online shopping suggestions (Basak 2017). The inherent fear that aligns with this feeling of being tracked is not completely unfounded. Although AI is in general used to make products and services more efficient in ways that benefits users, there is something unsettling about a future that could be ‘governed’ by AI. An additional legitimate concern of the public is the threat of large-scale job losses to AI services: a McKinsey report projected that 400 million jobs will face automation by 2030 and suggested that in the United States between 16 and 54 million could lose jobs between 2016 and 2030 (Rinehart and Edwards 2019). This creates an inherent fear of AI’s misuse and its potential negative impact on the livelihoods of displaced workers. A panel of AI experts shared their primary concerns on the future of AI (Robitzski 2018):

• “Sorting out how to keep AI responsible is a very tricky question; it has many more dimensions than just the scientific. That means all of society does need to be involved in answering it… Humans have always struggled with not letting new technologies be used for nefarious purposes” – Kenneth Stanley, Professor at University of Central Florida, Senior Engineering Manager and Staff Scientist at Uber AI Labs

• “I think the most dangerous thing with AI is its pace of development. Depending on how quickly it will develop and how quickly we will be able to adapt to it. And if we lost that balance, we might get in trouble… criminals or large terrorist organizations using it to disrupt large processes or simply do pure harm. Terrorists could cause harm via digital warfare… AI is a tool. It is a powerful tool, and this powerful tool could be used for good or bad things” – Irakli Beridze, Head of the Centre for Artificial Intelligence and Robotics at UNICRI, United Nations

• “I think we should watch out for drones. I think automated drones are potentially dangerous in a lot of ways… in five or ten years, I can imagine that a drone could have onboard computation sufficient enough that it could actually be useful” – John Langford, Principal Researcher at Microsoft

• “I don’t think there is bad technology, but there will be bad people. It comes down to who has access to the technology and how we use it” – Hava Siegelmann, Microsystems Technology Office Programs Manager at DARPA

• “When there’s a lot of interest and funding around something, there are also people who are abusing it… [there are some] AI examples when their systems are basically over-optimizing a single path that maybe anyone didn’t even care about before… They are more like magician tricks” – Tomas Mikolov, Researcher Scientist at Facebook AI

Part of the vilification of AI undoubtedly comes from its negative portrayal in pop-culture (Basak 2017). In reality, most effort in AI is to find harmony between the growing capabilities of AI technology and holistic benefits it can provide, however there is inherent risk with AI that comes with any new technology as there will be bad actors that use AI to promote antisocial and illegal activities.  The risk of misuse of AI means we must work to understand its potential and as we understand, construct regulations and use practices that balance the benefits against the risks.

Technically speaking, AI can be dated back as far as the 14^th century where Ramon Llull, Catalan poet and theologian, used mechanical means to create new knowledge from combining concepts (Press 2016). More commonly cited pioneers of AI are Thomas Bayes (Bayesian inference) in 1763 and George Boole (Boolean logic) in 1854 (Press 2016). Bayes helped develop a framework of reasoning based on probability and Boole developed logical reasoning useful in resolving solutions in complex systems (Press 2016). More recently, Alan Turing conceptualized a test of AI effectiveness, which is now known as the Turing Test (Press 2016). Today, Turing machines refer to any mathematical model of computation that defines a machine; in essence any computer is a Turing machine at its core. In 1955, the term artificial intelligence was first coined in a project proposal submitted by John McCarthy of Dartmouth College (Press 2016). This 1955 proposal is considered as the official birth of the new field, AI, used for automated reasoning: the first AI program, the Logic Theorist, was create by Herbert Simon and Allen Newell in 1955 (Press 2016).

Today’s use of AI has greatly developed from that of the mid-20^th century and with its increasing ability comes concern relative to its potential misuse or unintended consequences of its use that create risks for society. Multi-national corporations, including Apple, Facebook, Google and Amazon, have invested heavily in AI research and development with little regulatory oversight governing how AI is implemented into their products and services (Adam 2017).  Adding to the risk of potential misuse, the concentration of AI’s use in global technology giants means it is contributing to strengthening an industry monopoly which is marginalizing competitors. A lack of industry competition is harmful for multiple reasons: including lack of innovation due to complacency and inflated product costs thereby hurting the public. The lack of regulation has sparked controversy concerning individual user privacy; the European Commission filed a five-billion-dollar lawsuit against Google for breaching EU antitrust rules (Cassidy 2018). Google’s Android operating system is estimated at running around 85% of the world’s smartphones, making it effectively a monopoly in mobile communications software (Cassidy 2018). This was a major case in EU’s cracking down on giant American tech companies, in an effort to more aggressively regulate their practices in place of the laissez-faire American regulatory agencies (Cassidy 2018). This highly publicized lawsuit further sparked conversation on how much of our personal data these tech giants can access and use without permission. This, in part, reinforced vilification of AI as a risk to personal privacy and an enabler of tech company dominance. The EU lawsuit created the basis for discussion between the EU and tech giants around upgrades in regulation and policy around AI. Sundar Pichai, Alphabet’s (parent company to Google) chief executive, stated, “regulation of artificial intelligence was needed to ensure proper human oversight… ‘there is a balance to be had’ to ensure that rules do not stifle innovation” (Satariano 2020). This new effort toward enhancing regulatory measures has placed global attention on the capabilities of AI:

“Artificial intelligence – where machines are being trained to learn how to perform jobs on their own – is seen by technologists, business leaders and government officials as one of the world’s most transformative technologies. Yet it presents new risks to individual privacy and livelihoods – including the possibility that the tech will replace people in their jobs.” (Satariano 2020)

The proposal for the policy outlines the healthcare industry specifically as a riskier use of the technology. Margrethe Vestager, executive vice president of the European Commission, highlights that while AI presents itself as one of the most promising technologies of today, “it presents many dangers because it requires trusting complex algorithms to make decisions based on vast amounts of data” (Satariano 2020).

         One of the dangers Vestager is referring to is AI’s threat of becoming overly human. Overly human refers to its propensity to reinforce societal inequalities and mimic racist and gendered prejudices. Google’s unveiling of BERT – an artificial intelligence technology that systematically learns how people write and talk – uncovered AI’s ability to pick up on human biases in its programming (Metz 2019). It’s AI system learns from digitized information which harbors decades of biases, which are now being recycled into modern thinking: for example, BERT is more likely to associate computer programming with men, one program associated almost anything written about President Trump with being negative (even if the content was intended to be flattering), in 99 cases out of 100 words – ‘jewelry’, ‘baby’, ‘horses’, ‘houses’, ‘money’, and ‘action’ – were associated with men (the only outlier being ‘mom’), and Google and Amazon’s cloud-computing services both failed to recognize ‘hers’ as a pronoun, but correctly identified ‘his’ (Metz 2019). These discoveries don’t account for all the biases we have to detect or that we are even able to recognize.

Image source: Tahir 2019

Image source: Financial Services Storytelling 2018

Another major danger of AI technology is its use of big data and the privacy issues that result from its ability to disaggregate inputs and thus connect data back to ‘anonymous’ sources. Moore’s law dictates that computer power doubles every 18 – 24 months, but with massive increases in individual devices and high-speed global networks, a new law is introduced: Metcalfe’s Law (Kerry 2020). Metcalfe’s law dictates that increases in networks have a compounding effect on growth of information, meaning more data (big data) makes computing power much more powerful (Kerry 2020). However, this pushes individual user privacy to the epicenter of computing growth because individual user data is its driving force. AI is accentuating this concern because of its large-scale use of big data in “search algorithms, recommendation engines, and adtech networks”. “As AI evolves, it magnifies the ability to use personal information in ways that can intrude on privacy interests by raising analysis of personal information to new levels of power and speed” (Kerry 2020). Facial recognition has become a central part of this concern, creating calls for banning of the use of facial recognition due to the inherent privacy issues and potential for abuse.  Critics have cited the example, of China using facial recognition as a means to support authoritarian control (Kerry 2020). There is an ongoing debate between legislators and business leaders to find a balance of privacy protection and technological growth. The main policy options for privacy protection take two primary main forms: 1) targeting discrimination directly – which is reflected in the Consumer Online Privacy Rights Acts introduced to the Senate in 2019, and 2) a more indirect approach to create accountability measures to identify discrimination in the handling of personal data (Kerry 2020).

In furthering these discussions regarding user protection and increased regulation, it is possible that a balance between sensible regulation and innovation can drive AI’s future toward positive social impact. Ideally, policy regulations will be abided by without stunting technological growth.

2.2 The Future of AI

AI’s continued development offers new insight to its potential implementation and its where its use is most ethical and appropriate. Google heavily values the potential of AI: in order to “provide new ways of approaching problems and meaningfully improve people lives” they have designated an internal program to fund AI specific research in the hopes of tackling the world’s biggest challenges (Google AI). After its optimistically welcomed discovery in the 1950s, the following 60 years of steady research caused both positive and negative public and governmental reactions, bringing us to the present day debate on where and how to use AI for good and where to limit its use to avoid abuse (Hager et al. 2017). AI’s future is at a malleable stage allowing us to decide how it will continue to impact our future; it is important to push it in a positive direction that supports societal development. A pivotal element of its evolution relies on what young researchers are inspired to explore (Hager et al. 2017). While we can’t ignore the policy decisions that continue to shape AI and the ethical and safety concerns that surround it, we can’t forget the important societal implications AI currently has and the potential it has on shaping our future for the better (Hager et al. 2017).

AI could strongly support the sustainability mission: this does not solely refer to that of the environmental agenda, but rather it encompasses “all aspects of sustainable biological, economic, and social systems that support human wellbeing” (Hager et al. 2017). Deloitte reports that 8 in 10 leaders consider AI critically important to their success within the next two years (Austin). The reason for this is because AI is anticipated to be the source of tremendous productivity gains into the future, fundamentally changing the use of general technology across industries and changing society as a whole (Austin). McKinsey’s 18 identified capabilities of AI fall under 3 major categories of AI application: computer vision, natural-language processing, and speech recognition (Chui et al.). Some of the capabilities they cite relating specifically to healthcare are (they denote these in overall categories as “health and hunger”, “public and social sector” and partially as “crisis response”): deep learning on structured data, natural language processing, image and video classification, object detection and localization, language understanding, sound detection and recognition, sentiment analysis, language translation, tracking, emotion recognition, speech to text, content generation, and reinforcement learning (Chui et al.).

Although AI will be replacing many jobs, its use will also be creating jobs. In 2017, Gartner predicted that as of 2020 more jobs will be created by AI than eliminated: “AI will create 2.3 million jobs in 2020, while eliminating 1.8 million” (Gartner 2017). Additionally, they cite the healthcare industry as one that will see continuously growing job demand, while others like manufacturing will see the most job loss (Gartner 2017). It’s a falsehood that AI is synonymous with automation, the greatest growth of AI implementation relies on “AI augmentation – a combination of human and artificial intelligence, where both complement each other” (Gartner 2017). McKinsey reports that one of the major challenges to AI implementation is a lack of field-specific talent (Chui et al.). This creates demand for talent and education in this field, reinforcing the importance of inspiring young researchers to fill the gaps where they are most needed. Some of the capabilities of AI cited above do not require extensive AI experience or knowledge, whereas others are more complex and case specific calling for more vigorously trained AI experts (Chui et al.). Therefore, progress in this field relies heavily on the recruitment of capable and talented people to program these specific AI capabilities that hold potential for great social benefit. A necessary focus of employers going into the future is “augmenting” people with AI, ensuring their skills are prepared for how AI will disrupt industries (Gartner 2017). Gartner sites that in 2021, “AI augmentation will generate $2.9 trillion in business value and recover 6.2 billion hours of worker productivity” (Garnet 2017).

The following suggestions push beyond the vague rhetoric surrounding ‘AI for social good’ and give concrete suggestions for multi-benefit application of AI in the healthcare industry.

3  AI in the Healthcare Industry

3.1 AI in Healthcare: Laying the Groundwork

Sustainability hinges on the assurance of the health of both ecosystems and human societies long into the future (Hager et al. 2017). To briefly lay some groundwork on ways AI is already being implemented in the healthcare industry today, the following share some successful examples of AI’s positive impact on long-term health:

• “Current methods for gathering population-scale data about public health through surveys of medical providers or the public are expensive, time consuming, and biased towards patients who are already engaged in the medical system. Social media analytics is emerging as an alternative or complementary approach for instantly measuring the nation’s health at large scale and with little or no cost. Natural language processing can accurately identify social media posts that are self-reports of disease systems, even for rare conditions.” (Hager et al. 2017, 8)

• “The massive-transfusion protocol (MTP) CDST [clinical decision support tool] is currently being assessed under a two-year clinical trial… This CDST uses evidence-based predictive analytics to help physicians identify which patients genuinely require a massive transfusion, thereby reducing complications associated with over-transfusion or the needless expenditure of blood products.” (Hager et al. 2017, 8)

• “Robots can analyze data from pre-op medical records to guide a surgeons instrument during surgery, which can lead to a 21% reduction in a patients hospital stay. Robot-assisted surgery is considered “minimally invasive” so patients won’t need to heal from large incisions. Via artificial intelligence, robots can use data from past operations to inform new surgical techniques… One study that involved 379 orthopedic patients found that AI-assisted robotic procedure resulted in five times fewer complications compared to surgeons operating alone.” (Marr 2018)

• “From interacting with patients to directing patients to the most effective care setting, virtual nursing assistants could save the healthcare industry $20 billion annually… Most applications of virtual nursing assistants today allow for more regular communication between patients and care providers between office visits to prevent hospital readmission or unnecessary hospital visits.” (Marr 2018)

• “Currently, image analysis is very time consuming for human providers, but an MIT-led research team developed a machine-learning algorithm that can analyze 3D scans up to 1,000 times faster than what is possible today. This near real-time assessment can provide critical input for surgeons who are operating. It is also hoped that AI can help improve the next generation of radiology tools.” (Marr 2018)

AI’s use in healthcare, although already in existence, is still in its infancy stages. There are many untapped areas of growth for AI’s use in healthcare processes. Some near-term and longer-term opportunities that Hager et al. share in their paper are as follows:

Near-Term Opportunities: 

• Targeted therapy decisions: “Many chronic diseases are difficult to treat because of high variation among affected individuals. This makes it difficult to choose the optimal therapy for a patient. Developing systems that support targeted therapy decisions from large-scale observational data is an emerging and exciting area of research… By analyzing longitudinal databases of clinical measurements and health records we can develop decision support tools to improve decision-making.” (Hager et al. 2017, 9)

• New sensors, new healthcare delivery: “AI can be used to analyze social media data and discover and suggest behavioral and environmental impacts on health… [In addition to methods mentioned previously] Social media as well as social networks can also be used to address the informational and psychological needs of individuals… Related also is this opportunity for cost-effective interventions for addressing mental health, addition, and behavioral health issues using modern low-cost sensing technologies. Data gathered routinely during healthcare delivery can be leveraged to reduce hospital-acquired infections. Low fidelity sensors, some of which are diagnostic, together with AI and internet technologies can enable low barrier telemedicine for example for chronic healthcare. Advances in natural language processing and machine reading can be used to synthesize, integrate and appropriately disseminate new medical knowledge…” (Hager et al. 2017, 9)

Longer-Term Opportunities:

• Personalized health: “… The major opportunity is to pivot from personalized medicine to personalized health, to keeping people from getting to the hospital in the first place, and to dealing with life issues and not just specific diseases. For this, we need to move to modeling health of individuals and populations by using integrated data sets –  electronic health records data and other data gathered within the health system with genomic, socio-economic, demographic, environmental, social network and social media and other, non-traditional data sources, such as social service and law enforcement data… From this can come personalized, longitudinal treatment plans to improve an individual’s health.” (Hager et al. 2017, 10)

• Addressing bias: “An important challenge that arises in fitting models from observational health data sources is that the data may be influenced by many phenomena, including some unrelated to target disease of interest and arising from the structure of the health ecosystem. For example, information may be missing on conditions that are not reimbursed. Not accounting for these biases can lead to models that cause harm as provider practice patterns change… in other words, what is the measurement process and how did it affect the data that were generated?” (Hager et al. 2017, 10)

 

The opportunities presented above consider the TBL by valuing cost-effectiveness as well as environmental and social well-being as equal parts to a successful whole. Further use of a TBL approach provides a framework groundwork for evaluating the most effective and sustainable uses of AI in healthcare.  In the next section, the TBL evaluation of some potential uses of AI in healthcare are considered.

3.2 Greatest Challenges in Healthcare Today

Although the healthcare industry has seen massive changes and improvements in recent history, it is constantly evolving and with changes in climate, growing populations, and new discoveries, new challenges and opportunities are constantly developing. Much of the future of healthcare improvements and innovation rest in technological development and effective implementation targeted at addressing specific healthcare challenges that provide wide reach to large populations while effectively using limited resources. The following section highlights healthcare’s greatest challenges to date – this provides a framework for how to most effectively target AI technology implementation to have the greatest social impact possible.

1. Uncertainty: Uncertainty, like for many industries and in its most general sense, is one of the greatest challenges acting as a barrier to the success of the healthcare industry. Actors that can thrive under the pressures of uncertainty are the strongest bets for the growth of the healthcare industry (Singhal et al. 2018).

2. Costs: The costs of healthcare are only predicted to increase, making it inaccessible to certain groups. The Kaiser Family Foundation predicts that the annual cost of health insurance per family will surpass $20,000 (Appold 2020). This is because of increased drug prices, high-cost therapies, and industry consolidation (Appold 2020). In particular, specialty drug prices (drugs used to treat specialty and rare diseases) have majorly increased – they cost up to 50 times that of traditional drugs (Appold 2020). The key solution to this is focusing efforts towards innovative solutions, such as technological advances, to help mitigate the rising costs of healthcare.

3. Data – Access and Analytics: Healthcare data is becoming more and more prevalent (extremely useful data), but there are few systems in place to collect and consolidate the data and a lack programs to make the data useful (Sullivan 2018). If big data could be harnessed and made available to healthcare providers, it could support data-driven decision-making (Sullivan 2018). The key to tackle this challenge is not only to harness the data, but to make it meaningful and ethically sourced.

AI can play a role in tackling all three of these major challenges that act as barriers to the global success of the healthcare industry.

3.3 AI-Specific Recommendations Targeting Healthcare Challenges

AI technology has the ability to target the aforementioned challenges of healthcare, in particular that of increased costs and data handling. This will contribute to ensuring that the healthcare system is robust and can withstand uncertainty moving forward in the future.

The first recommendation targets the rising costs of the healthcare industry. The OECD (Organization for Economic Cooperation and Development) estimates that around 20% of healthcare spending is wasted globally (Bernaert and Akpakwu 2018). This amounts to (based on the top 15 countries by healthcare expenditure) an estimated waste of between $1,100 and $1,700 per person annually (Bernaert and Akpakwu 2018). This coupled with the overall rising costs in the healthcare industry (particularly in drug costs and insurance) makes healthcare increasingly inaccessible to large portions of the population. AI can help to eliminate some of these cost inefficiencies by creating more stream-lined process for common healthcare procedures and data networks, making healthcare more affordable, accessible, and resource efficient. The following are the key points of cost reduction in healthcare using AI:

• Repetitive, pattern-based tasks (e.g. analysis of CT scans) can be done more accurately by AI programs – this eliminates physician error and allows earlier diagnoses (Bernaert and Akpakwu 2018). According to Wired, AI can be 30 times faster in analyzing mammograms; this allows early condition detection and saves time and cuts down on resource waste (Bernaert and Akpakwu 2018).

• Effectively harnessing data can also begin to standardize and streamline medical processes: all the way from diagnoses to treatment. AI programs can help analyze large data sets to identify patterns in condition diagnosis and treatment outcome, allowing clinical decision-making to be more informed and streamlined (Bernaert and Akpakwu 2018). This eliminates wasteful treatment options and helps surpass trial and error phases of treatment. By doing so, one of the highest resource-consuming consequences of treatment will be eliminated (or heavily reduced) by decreasing unnecessary treatments and post-treatment complications (Bernaert and Akpakwu 2018).

• One of the major areas AI can help reduce costs (where they are predicted to heavily increase without intervention in the coming years) are in drug trials and development. AI can enable faster development of drugs, both saving countless lives and eliminating large costs and resource depletion (Bernaert and Akpakwu 2018). Biomarker monitoring platforms allow AI to optimize drug development by allowing gene-level identification of diseases and the quick processing of millions of patient data points (Bernaert and Akpakwu 2018). Researchers, consumers, drug developers, policy makers, and the healthcare industry benefit from these opportunities of growth and development.

There is undoubtedly something unsettling about the possibility of taking medical advice from a robot, but in reality, these AI programs have not replaced their human counterparts, instead they have augmented human caregivers so that they can more effectively and thoroughly manage a large number of patients (Bernaert and Akpakwu 2018).

Video source: World Economic Forum 2018

Second, AI can be used to make big data more  useful and accessible for healthcare practitioners, bettering patient care, medical processes, and cost/resource efficiency. Additionally, understanding big data can help combat the detriments of uncertainty by shaping our understanding of illness patterns, patient similarities, treatment comparisons, predictive programming, and so forth. The use of AI in capitalizing on the use of data was already touched on in the previous recommendation, as it could be beneficial in all sectors of the healthcare industry. The following are areas of extreme growth potential in capitalizing on big data using AI in the healthcare industry:

• Using data analysis, AI can be used to power predictive care (Kriwet 2020). Patient data ranging all the way from place of birth, diet, and employment (“social determinants of health”) to medical history and genetically predisposed diseases can be used to predict and preempt medical care (Kriwet 2020). This will not only save lives, eliminate the need for medical procedures, reduce the detriments of uncertainty on our health and resources, but also focus medical care towards prevention rather than treatment.

• Another majorly useful area for AI to fill a gap in healthcare is by creating a centralized health records nationally, and even globally. AI can help build one globally accessible database for health records and patient information (abiding by privacy laws and regulations). All physical hospital locations can be connected via digital infrastructure, monitoring supply and demand, using AI to determine or predict at-risk patients, eliminate systemic bottlenecks that prevent patients from access or complications, informing potential telemedicine capabilities, and create a highly cohesive system bound by digital footprint rather than physical location (Kriwet 2020).

The above recommendations tie in Hager et al.’s near-term and long-term opportunity suggestions and tackle what are considered to be some of healthcare’s most pressing challenges moving forward.

3.4 Barriers to AI Implementation

There are a few barriers that challenge the seamless implementation of AI in the healthcare industry that are noteworthy and listed here briefly. Hager et al. identify some of the barriers as 1) the need for cross-disciplinary training (programs that train scientists in developing AI methods for complex socio-technical systems), 2) needing better methods of working with data and ensuring privacy along with increased data sharing, and 3) making a sustainable system that incorporates AI’s ability to observe data and make more informed decisions, while reactively making the needed interventions and the ability to learn from this feedback loop (Hager et al. 2017, 10  – 11). The privacy issues Hager et al. mention, coincide with the difficulty of FDA approval related to certain AI algorithms (Greenfield 2019). Although algorithms have shown to have major benefits for both patients and doctors in medicine and medical devices, it has proven to be difficult to regulate these algorithms (Greenfield 2019). In addition to this, a McKinsey report introduces some of the main hurdles to effective implementation of AI in healthcare as the shortage of talent to develop AI solutions, lack of data accessibility for certain regions, and the ‘last-mile’ challenge (Chui et al. 2018). The ‘last-mile’ challenge refers to NGOs and social-sector organizations that don’t have constant access to a certain level of AI-related skill in keeping up with the function of new technologies (Chui et al. 2018). This challenge occurs when technology providers disappear after the implementation of the solution without creating a system that ensures its long-term sustainability (Chui et al. 2018). This occurs almost exclusively in poorer and/or rural regions, deepening the socio-economic divide with the implementation of beneficial technology. The possibilities of AI are far too great to allow hurdles to become unsurpassable barriers; therefore, each hurdle must be tackled concurrently with the development and implementation of AI.  

4  Conclusion

AI is integral to the future of social, political, environmental, and economic growth. When implemented appropriately and with consideration to its most optimal utilization, specifically considering the TBL (triple bottom line, i.e. people, planet, and profit) framework, it can have immensely positive social impact. As with all new technologies, it has the potential to be used unethically by bad actors. It is because of this potential for misuse that now is the time to shape the future of AI for social good. This paper provides background on AI, outlines its potential to be used for social good and highlights risks that come with the power of this new technology.  Additionally, specific examples of AI use in the healthcare industry are outlined to show that when used in the right way, the approach offers great benefit to social wellbeing. The two main recommendations for capitalizing on AI in the healthcare industry are: 1) streamlining healthcare systems and drug development to reduce costs and 2) harnessing meaningful data to improve diagnoses and treatments and eliminating factors of uncertainty. AI for social good is and should be the future of technological development, providing investors with opportunities to both benefit on an economic level, as well as a social and environmental one.

Works Cited

Adams, R.L. 2017. “10 Powerful Examples Of Artificial Intelligence In Use Today.” Forbes. January 2017. https://www.forbes.com/sites/robertadams/2017/01/10/10-powerful-examples-of-artificial-intelligence-in-use-today/#3c209fe1420d.

“AI for Social Good.” n.d. Google AI. https://ai.google/social-good/.

Appold, Karen. 2020. “Top 6 Challenges Healthcare Executives Will Face in 2020.” Managed Healthcare Executive. January 2020. https://www.managedhealthcareexecutive.com/news/top-6-challenges-healthcare-executives-will-face-2020/page/0/2.

Austin, Shelby. n.d. “Capitalizing on the Promise of Artificial Intelligence.” Deloitte. https://www2.deloitte.com/ca/en/pages/deloitte-analytics/articles/global-perspectives-ai-adoption.html#.

Basak, Suryoday. 2017. “On the Negative Connotations of AI and Its ‘Responsible Growth.’” Medium. September 2017. https://medium.com/@suryodaybasak/on-the-negative-connotations-of-ai-and-its-responsible-growth-d0ee9896bfcb.

Bernaert, Arnaud, and Emmanuel Akpakwu. 2018. “Four Ways AI Can Make Healthcare More Efficient and Affordable.” World Economic Forum. May 2018. https://www.weforum.org/agenda/2018/05/four-ways-ai-is-bringing-down-the-cost-of-healthcare/.

Cassidy, John. 2018. “Why Did the European Commission Fine Google Five Billion Dollars?” The New Yorker, July 2018. https://www.newyorker.com/news/our-columnists/why-did-the-european-commission-fine-google-five-billion-dollars.

Chui, Michael, Martin Harrysson, James Manyika, Rogers Roberts, Rita Chung, Pieter Nel, and Ashley van Heteren. n.d. “Applying Artificial Intelligence for Social Good.” McKinsey & Company. https://www.mckinsey.com/featured-insights/artificial-intelligence/applying-artificial-intelligence-for-social-good.

Elkington, John. 2018. “25 Years Ago I Coined the Phrase ‘Triple Bottom Line.’ Here’s Why It’s Time to Rethink It.” Harvard Business Review, June. https://hbr.org/2018/06/25-years-ago-i-coined-the-phrase-triple-bottom-line-heres-why-im-giving-up-on-it.

Elkington, John, and Richard Roberts. 2017. “Sustainability: A $12 Trillion a Year Market by 2030.” European Business Review, October. https://www.europeanbusinessreview.com/sustainability-a-12-trillion-a-year-market-by-2030/?__cf_chl_jschl_tk__=c8c2f08a52ae71d7391285271dac2dff55cfc049-1587665131-0-AavznFyMrk6Bl_-VluD-EK3hdJL0m_TLX4dkLFZcUbLSoTXlXiHZxbH8y5p4iJtoRujo1gsm5coVhsrLKmzBB0TLBvDbNTccoioSxWy7z9apT5HL2gJzGsXjNkXbTYEeluJoky_9o8_sNzFULExkFLKd52Tlxm5NqwI4NKm3hcguWJO2gwgjIf-hdOSnvEvnQ7pu-306y9WOeBC8KiFY21ao6NPDaxWQAApWOk_KVkOhfwf3c3bxwMSIJK7TktRyJenTaUE56-WNv4RPxMkqo6BGv4q8Wa9QJZJ1Un3MCcJKJmE5nxf7OOzKe5LlHfDcv3ctN0chsva3yhGsnLGc5bs.

Financial Services Storytelling. 2018. “Perpetuating the Past? AI’s Battle Against Bias.” Medium. April 2018. https://medium.com/the-future-of-financial-services/perpetuating-the-past-ais-battle-against-bias-915852c97b31.

“Gartner Says By 2020, Artificial Intelligence Will Create More Jobs Than It Eliminates.” 2017. Gartner. December 2017. https://www.gartner.com/en/newsroom/press-releases/2017-12-13-gartner-says-by-2020-artificial-intelligence-will-create-more-jobs-than-it-eliminates.

Greenfield, Daniel. 2019. “Artificial Intelligence in Medicine: Applications, Implications, and Limitations.” Harvard University: The Graduate School of Arts and Sciences. June 2019. http://sitn.hms.harvard.edu/flash/2019/artificial-intelligence-in-medicine-applications-implications-and-limitations/.

Hager, Gregory D., Ann Drobnis, Fei Fang, Rayid Ghani, Amy Greenwald, Terah Lyons, David C. Parkes, et al. 2017. “Artifical Intelligence for Social Good.” Computing Community Consortium, March, 1–23.

Haile-Mariam, Hana, and Allison Spector. 2017. “Impact Measurement in the Healthcare Sector.” GIIN: Global Impact Investing Network, March, 1–31.

Kerry, Cameron F. 2020. “Protecting Privacy in an AI-Driven World.” Brookings, February. https://www.brookings.edu/research/protecting-privacy-in-an-ai-driven-world/.

Kriwet, Carla. 2020. “Here Are 3 Ways AI Will Change Healthcare by 2030.” World Economic Forum. January 2020. https://www.weforum.org/agenda/2020/01/future-of-artificial-intelligence-healthcare-delivery/.

Marr, Bernard. 2018. “How Is AI Used In Healthcare – 5 Powerful Real-World Examples That Show The Latest Advances.” Forbes. July 2018. https://www.forbes.com/sites/bernardmarr/2018/07/27/how-is-ai-used-in-healthcare-5-powerful-real-world-examples-that-show-the-latest-advances/#7018946a5dfb.

Metz, Cade. 2019. “We Teach A.I. Systems Everything, Including Our Biases.” The New York Times, November 2019. https://www.nytimes.com/2019/11/11/technology/artificial-intelligence-bias.html.

Press, Gil. 2016. “A Very Short History Of Artificial Intelligence (AI).” Forbes. December 2016. https://www.forbes.com/sites/gilpress/2016/12/30/a-very-short-history-of-artificial-intelligence-ai/#1b3b2fd56fba.

Rinehart, Will, and Allison Edwards. 2019. “Understanding Job Loss Predictions From Artificial Intelligence.” American Action Forum. July 2019. https://www.americanactionforum.org/insight/understanding-job-loss-predictions-from-artificial-intelligence/.

Robitzski, Dan. 2018. “Five Experts Share What Scares Them the Most About AI.” Futurism. September 2018. https://futurism.com/artificial-intelligence-experts-fear.

Saratchandran, Vinod. 2019. “5 Ways Big Data Is Changing the Healthcare Industry.” Fingent. January 2019. https://www.fingent.com/blog/5-ways-big-data-is-changing-the-healthcare-industry.

Satariano, Adam. 2020. “Silicon Valley Heads to Europe, Nervous About New Rules.” The New York Times, February 2020. https://www.nytimes.com/2020/02/16/technology/europe-new-AI-tech-regulations.html.

Singhal, Shubham, Brian Latko, and Carlos Pardo Martin. 2018. “The Future of Healthcare: Finding the Opportunities That Lie beneath the Uncertainty.” McKinsey & Company. January 2018. https://www.mckinsey.com/industries/healthcare-systems-and-services/our-insights/the-future-of-healthcare-finding-the-opportunities-that-lie-beneath-the-uncertainty.

Sullivan, Hugh. 2018. “6 Major Challenges Facing the Healthcare Industry in 2020.” Medium. November 2018. https://medium.com/@MailMyStatement/5-major-challenges-facing-the-healthcare-industry-in-2019-60218336385f.

Sustainability Illustrated. 2014. Triple Bottom Line (3 Pillars): Sustainability in Business. YouTube. https://www.youtube.com/watch?v=2f5m-jBf81Q&feature=emb_title.

Tahir, Osama. 2019. “When Machines Know Sin: The Algorithmic Bias of Technology.” Hackernoon. February 2019. https://hackernoon.com/when-machines-know-sin-the-algorithmic-bias-of-technology-82402b70dfd0.

“The Triple Bottom Line.” n.d. University of Wisconsin: Sustainable Management. https://sustain.wisconsin.edu/sustainability/triple-bottom-line/.

Sacha Qasim: 2020 

Image: TechCrunch

Abstract

Advances in information and communications technologies have greatly benefited our daily lives. Technology has likewise been at the forefront in fighting the virus during the global COVID-19 pandemic. Governments across the globe are using contact tracing apps to track the physical proximity between individuals using Bluetooth transceivers. Apple and Google have announced a joint effort to develop a contact tracing feature for the iOS and Android systems, which will help individuals, governments, and health institutions combat the virus. Despite the promise of technologies like contact tracing, concern and alarm has been raised because of how they are “vulnerable to electronic surveillance and interception.”[1] Misuse of surveillance threatens individuals and their digital privacy rights. This paper will delve into the development and deployment of contact tracing apps, de-blackbox how they work, and discuss their implications for privacy.

Introduction

A full-spectrum of business crises have emerged from COVID-19. While markets oscillate between bullish and bearish trends, factories and businesses have whirred to a pause, forcing nearly 30 million US citizens to claim unemployment (as of April 23, 2020). In isolation, our personal and professional lives intermingle complexing our need for change, hours Zoom by, and simple acts like handshaking and hugging have become dangerous and unwelcome. With government-mandated lockdowns across the world, essential workers must constantly expose themselves to illness, while simple chores require most everyone to leave safety.

The adoption of contact tracing apps will become part of our daily lives beyond COVID-19 days. Part of Apple and Google’s partnership is centralized around how it is designed, focusing on the privacy and security implementations. The main objective of this research paper is to delve into the legal documents of the Exposure Notification (Apple and Google’s rebrand of contact tracing) engine and de-blackbox the system through its users terms and agreements. Subsequently, governments that are building contact tracing apps, presents a dichotomy in staying home or being surveilled. This design will be available to over 260 million smartphone users in the United States[2], which inevitably bears the question, how is our privacy prioritized? 

________

Apple and Google’s Contact Tracing Bluetooth Spec for COVID-19 Explained by Hussein Nasser

________

How will Apple | Google’s Contact Tracing app work?

Apple and Google’s unprecedented collaboration will propel advances of technology to reduce the spread of COVID-19. Many governments are keen to loosen up lockdowns and reopen economies. To do so, we need to be sure citizens are infection-free and can safely resume life mingling with others. In assuring the safety of citizens, contact tracing is a novel idea. The proposals and released versions are eminent: the iOS and Android systems will generate a database using Bluetooth transceivers, allocating keys to devices that are in Bluetooth range. Bluetooth range can be measured from 100-200 feet. These capabilities will make it possible for an individual who tests positive for the coronavirus, notify the system. The system will anonymously deploy the keys to any individual who has been in close proximity to the infected individual in the past 14 days. The exposed user will then receive an “exposure notification” and will be alerted that they are at risk. The exposed user should then self-isolate and get tested.

The application programming interface (API) will release enabling the interoperability between iOS and Android devices. Apple and Google are making the Bluetooth transceivers accessible to the third parties such as the government and health agencies after complying with The Exposure Notification Entitlement Request[3] which includes an addendum of strict licensing and restrictions. To use the Exposure Notification API’s “You must be a government entity, such as a government health services organization, or a developer who has been endorsed and approved by a government entity to develop an Application on behalf of a government for COVID-19 response efforts”[4] explicitly reported in the Apple Developer Program License Agreement.

Who are the Users?

The ExposureNotification framework defines users, who have consented the system with opting in. The main goal in the design process of the new system is to be sure privacy is preserved in the implementation process. Apple and Google have acknowledged two types of users:

Affected User: The affected user will have the prerogative to notify the system if they have been tested positive for the coronavirus. To notify the system, they will be given a unique code by health agencies to enter into the system[5]. This will reduce fake positives by malicious attackers. When a user is tested positive, their diagnosis keys (within the framework of API) will be shared with other users to alert them to potential exposure.

Exposed User: Given a set of positively affected users, the framework allows you to determine whether those Daily Tracing Keys were observed locally by the user, indicating potential exposure, if so, additional information such as date and duration may also be retrieved. But the affected user will always remain anonymous.

In addition, another unnamed user is one who remains healthy and never tests positive, the Daily Tracing Keys never leave the device and their keys stay within the database. Also, while not a user, but individuals who choose not to download the Exposure Notification app can pose an undetermined risk. These outliers can be those who fear over privacy and surveillance concerns.   

Bluetooth Exposure Notification Applications:

Permitted government health agencies can build the apps with the Exposure Notification service through the Bluetooth Low Energy wireless technology enabled. Official apps will be available to be downloaded through respective app stores. These apps employ systems that notify users of exposure through “random, rotating keys and identifiers to convey positive diagnosis in addition to data such as associated symptoms, proximity, and duration.”[6]

Within the system, sets of keys are generated when consent is given by the user. The keys are enabled to receive information through Bluetooth signals. Every day, the Daily Tracing Key is generated for the 24-hour window. Later, if a user is infected, the Daily Tracing Keys are retrieved from the device and sent to the subset of keys: Diagnoses Keys. The Rolling Proximity Identifier is the main privacy-preserving identifiers that are sent through Bluetooth transceiver keys. These are implemented as layers in preserving individual privacy.

The image above, produced by Apple visualizes how the different keys will be processed through the system. The amount of time and purpose are allocated for specific purposes. Note: This image was one of the first models of how the Key Schedule will work but has been updated since then. This report will use the original picture for it being simplified comparatively.

The intent is forging ahead with Bluetooth contact tracing platform to be built into the underline platforms in the devices. Making this is a more robust solution than an API that allows more individuals to participate through their choice in opting in. This venture will also enable more participation through the scope of the general ecosystem of apps and government authorities with easy accessibility.

De-Blackboxing Apple and Google’s Sample Code for Building a Contact Tracing App:

Apple and Google have released a sample code that can be used as a guideline for developers across the world who are developing applications for their governments. In the process, there are eight crucial steps to solidify the functionality of the app as well as following the strict protocol warranted by Apple and Google to use the Exposure Notification software.

1. Authorize Exposure Notifications

Here the user will explicitly authorize the software engaging exposure notification. Apple and Google’s specific ENManager class has been developed to specifically provide information pertaining to the objectives of the user’s authorization for proceeding with the protocol.

The ENManager class is the legal blueprint that defines variables and methods applied broadly in the system. A generic class can be visualized as seen below:

Image: The Java Tutorial[7]

Every time the user launches the app, the ENManager object will detect whether the user has authorized exposure notification. Since the mission of launching these systems is detrimental to society and governments, the user will be prompted with an interface (upon launching the app) to authorize the service each time. The interface will prompt the user a second time, pressing importance to opt-in. In every case, the user will still have control over their privacy and rights and may choose to decline to activate the system.

2. Store User Data Locally

This step is where app stores information about test results and high-risk exposures in the user defaults directory.[8] This will prompt the system to upload the diagnosis keys to the server. This is the process where the system scans the device’s sent/received Bluetooth beacons between other devices for matched interactions with any other diagnosis keys.

In this process, the data is transformed and uses a custom property wrapper to encrypt the data and notifies the app when the local data has been modified.

3. Share Diagnosis Keys with the Server

This is where many people, fear their privacy will begin to be abused if this step in programming is not secured carefully. In storing valuable information such as a user who has been diagnosed by a doctor for COVID-19, the user can upload their diagnosis keys to the server by authorizing it to do so. Side note: The United Kingdom is the only country in the world that lacks a professional diagnosis supplying the user with a unique code to enter into the system, warning others that they may be at risk. This is causing havoc since people can enter fake symptoms, inducing fear in society and attacks from trolls and malicious actors.[9]

With the infected user notifying the system, sending diagnosis keys to the server, the app will provide the keys on demand through a separate server that they diagnosis keys were originally sent to.

4. Ask Users to Share COVID-19 Indicators

After a user authorizes the app with information that they have been tested positive for COVID-19 the app will preserve this data on the device. The app will then ask the user for consent to share the keys to potentially affected users. When given permission to proceed with the monumental rotation and share of keys (all in pursuit of identifying those at risk sooner), the app will retrieve the list of diagnosis keys that have been exposed on rotation through the singleton ENManager objects method. This process will look similar to the sample code:

5. Create Background Task to Check for Exposure

One of the biggest obstacles Apple was challenged with in the past weeks was how the iOS system would not let Bluetooth transceivers signals interact with user’s devices when the phone was turned off. Singapore’s TraceTogether app was deterrent to users because Apple “will not allow Bluetooth-based track-and-tracing to be carried out in the background.”[10] Meaning that contact tracing apps screens had to be on, which hindered users from access to other tools on their phones such as making phone calls, drainage of battery, or general inconveniences for having the screen on and it sliding to another- no longer tracing the user as it should. France too, has been stomping their feet, butting heads with Apple in attempt to have them loosen privacy restrictions between user and government data. French digital minister Cedric O claims that France’s app will be released as early as June 2, 2020 and since has worked around not using Apple and Google’s contact tracing API. This is because the French government finds having user’s data an instrumental piece in combatting COVID-19 through contact tracing apps.[11]

Apple and Google have been working promisingly on working around the problem of background tracking. This step shows how. Intermittently, a background task checks the keys of potential exposure to a user’s keys who is diagnosed with COVID-19. To be sure the app captures the most signals of potential exposure, the operating system is designed to launch itself into the background. The background task is timed and done in fast intermetals to provide the most up-to-date information.

6. Download Diagnosis Keys

The following will observe how the sample code predicts how the app will obtain and submit diagnosis keys to the framework.

The app will be used as a vehicle to download diagnosis keys from the server and send it to the framework the analyze. This is a double-check method that will scan each diagnosis key. The app retrieves URLs of the keys files from the last generated scan. The URLs are used to access downloadable files to the device.  

7. Configure Criteria to Estimate Risk

Many hone on the concern for accuracy in the Bluetooth tracing technology. For how long and proximity distance can one consider close enough or too far for exposure? These are questions that may be up to the government health agencies, varying between each country. In this step of the process, the framework will use data derived from both the local level and diagnosis key. If the two matches (asserting risk of exposure) this will then take into account, the distance between the individuals and time spent in the vicinity. Evaluating the risk, ENExposure Configuration object is implemented into the code which gathers more accurate bits of data to make the guidelines.

The sample code by Apple and Google measure the level of risk between local systems and the diagnosis keys are as follows:

8. Submit Diagnosis Keys to the Framework

After the above steps are structured to capacity, the final blackbox is the submission of diagnosis keys into the framework. After many complex layers of sending, retrieving, wrapping, etc the information is subject for another round of evaluating the level of risk the keys indicate. “The app maps the exposure information to its own structure so that it can save the data to the app’s local store.”[12]  

And to “wrap” up the process, the app pings the finish method to complete its search (for risk and keys). “The finish method updated the local store with the new data, including any exposures, the date and time the app executed the search, and the index for the key file to check next time”[13]

Privacy Specifications:

Apple and Google have bullet pointed their privacy specifications and dedicated an entire page in their Exposure Notification preliminary. The tech companies hone on the delicate intent and purpose they masterfully have developed and state that privacy is an “essential requirement in the design.”[14]

The protocol warrants privacy eloquently in the preliminary by first assessing location tracking and debunking many concerns. The Exposure Notification Bluetooth Specification will not use location tracking for tracking individuals. The specification “strictly uses Bluetooth beaconing to detect proximity.”[15] Grave concern arose around the dystopian reality of tech giants using location tracking. Location tracking is the surveillance of location using Global Position System (GPS) that can accurately track specific location, speed, and navigational compass of device. These technologies are “constantly sending and receiving signals and tracking every single movement.”[16] Some governments have adopted these invasive technologies veiling their mass surveillance by justifying its intent is utilitarian- for the greater good of society in combatting COVID-19.

On May 4, 2020 Apple and Google announced that they will ban location tracking in their contact tracing apps. This is particularly amid fears of government tracking and to “Avoid fragmentation between different systems and instead allow all these phones to work together.” [17] This announcement calmed users that their privacy is still being prioritized and “staving off potential abuse.” [18] Which Nellis et als. responds with warnings from privacy experts who state that “any cache of location data related to health issues could make businesses and individuals vulnerable to being ostracized if the data is exposed.”[19]

In efforts to reduce the risk of privacy loss from broadcasting the identifying factors, behaviors in how often the keys change is crucial. Each Rolling Proximity Identifier changes nearly every 15 minutes, depending on the population density in the user’s region. The Temporary Exposure Key will “be correlated to a contact.”[20] 

“Proximity identifiers obtained from other devices are processed exclusively on device”[21] is bulleted to reaffirm the steps taken in how your data is being processed. Your data that processed, is exclusively done so on your device. Meaning that the cloud, developers, agencies cannot access your data.

The preliminary’s privacy section focus’ on consent in two parts: First, users are entitled to preserving their privacy and whether they want to contribute and opt-in to exposure notification. Second, if the user is tested and diagnosed with COVID-19, “users must provide their consent to share Diagnosis Keys with the server.”[22] So a user has the choice to opt-in to the system and be notified in the event they have been exposed. In addition, they have the rights and privacy on whether they want to notify the system that they have been diagnosed with COVID-19. This two-step consent is a new foreground that Apple and Google have stepped into, receiving rounds or applause from some and skepticism from others.

And alas, Apple and Google have single-handily tied privacy and transparency together. Acknowledging the two biggest buzzwords in the tech world, trepidation fills society. Upon the release of the legal documents of the Exposure Notification details, curious individuals immediately punch in Command/Ctrl + F (depending whether they are on a Mac or Windows PC) searching for these keywords that will defend their digital rights.  

Image: Apple-Google Exposure Notification FAQ Document

Privacy Considerations for Cryptography:

Apple and Google have also provided a detailed technical specification for cryptographic key scheduling which will be used in the Bluetooth protocol supporting the mission of preserving privacy upon the release of Exposure Notification. Similar to the privacy guidelines set in the Bluetooth specifications, keys will hinder the municipality or any other attackers from intersecting user’s data. Therefore, the key schedules are established by the specific components developed within the operating system. This system will prevent applications from digesting information that is predictable and that could allow third parties to track the user.

The Temporary Exposure Key and Rolling Proximity Identifiers correlate together sharing valuable information of timing, beacons that have been sent, and received. This is another way that can avail privacy loss from broadcasting the identifiers. This is possible because the Temporary Exposure Keys, any bad actor computationally can not access the Rolling Proximity Identifier due to the complexity of the system that lies with barriers because of rotating keys. “A wide range of replay and impersonation attacks”[16] are also prevented with these privacy principles.

When an infected user reports, the Diagnosis Key- correlated to the Rolling Proximity Identifiers has a time-frame of 24 hours until it rotates because of the Temporary Exposure Keys that are implemented into the system and programmed to change daily. It is ESSENTIAL that the server not retain or send any metadata collected by the clients.

_______

What Contract Tracing Means For Your Privacy by Bloomberg

_______

Basic Principles of Information Protection:

For a classic approach in how design principles should include information protection, Jerome Saltzer and Michael Shroeder, presented monumental pieces of work. Saltzer et als.recognoizes the weight of responsibility computers have on society as they integrate sharing more information, across more mediums as the economy and society expands.

The term “privacy” denotes a socially defined ability of an individual (or organization) to determine whether, when, and to whom personal (or organizational) information is to be released (Saltzer et als.)[23]

Mechanisms that are used to preserve privacy are deeply applied within the database. Access is controlled through the developer under the jurisdiction of said corporation. Saltzer and Schroeder developed five specific criteria with understanding a plethora of schools of thought in how to engineer designs.

Principle	Directive
Economy of Mechanism	Keep the design simple and small
Fail-safe defaults	Deny access by default; grant access only by explicit permission
Complete meditation	Check every access to every object
Open design	Do not depend on attackers being ignorant of the design
Separation of privilege	Grant access based on more than one piece of information
Least privilege	Force every process to operate with the minimum privileges needed for its task
Least Common mechanism	Make shared state information inaccessible to individual processes, lest one corrupt it
Psychological acceptability	Protection should be easy to use, at least as easy as not using it

Image: Figure 10.1, Information Protection Principles of Saltzer and Schroeder

The above table represents the principles that Saltzer et als. believed to be “descriptions of skills and strategies that software designers follow when making design decisions. This strategizes almost always lead them in the direction of designs that meet the criteria”[24]

Tracking Coronavirus Tracing Apps:

The deluge of apps are growing globally but needs regulation. MIT Technology Review has identified automated contact tracing apps that are supported by local governments. The apps use different paradigms in how they use Exposure Notification technologies to mandate regulations in combatting COVID-19. So far, data has collected 25  individual automated contact tracing apps that have been deployed by the government. This review analyzes apps by awarding up to five stars on the app based on the Voluntary, Limits, Data Destruction, Minimized, and Transparency of the app.[25] 

Image: PreventionWork

Notable countries include the following with the name, type of technology and awarded stars by MIT Tech Review:

Australia: COVIDSafe  4 STARS  Tech: Bluetooth

China: Chinese health code system NO STARS Tech: Location, Data mining

France: StopCovid 1 STAR Tech: Bluetooth

Germany: Corona App  2 STAR Tech: Bluetooth, Google/Apple

India: Mask.ir 1 STAR Tech: Location

Israel: HaMagen FIVE STARS Tech: Location

Italy: Immuni FIVE STARS Tech: Bluetooth, Google/Apple

Singapore: Trace Together FOUR STARS Tech: Bluetooth

UK: NHS COVID-19 App  ONE STAR Tech: Bluetooth, Location

The countries listed above is where the coronavirus is most clustered. These countries have a global responsibility in asserting conditions in how they will suppress the spread of the coronavirus.

However, many countries with the most tested and reported positive do not have contact tracing apps in place yet. May 15, 2020 Johns Hopkins University has confirmed over 4.5 million cases globally. With a quarter of the cases reported by the United States alone at 1.4 million.[26]

Human Rights During Global Pandemic:

Fear looms over many as technologies are being released every day in how they will combat the coronavirus. While contact tracing apps are promising, countries who have deployed them lack transparency which leads to discomfort and lack of interest in participating in downloading these apps.

The Human Rights Watch group believes contact tracing is invasive and poses a risk to human rights. Thought HRW does not directly acknowledge Apple and Google’s partnership, they discuss the vulnerability and trust issues we have with governments who are developing these apps. Though they acknowledge there is some sacrifice to be made to each individual in light of the greater good. “Some restrictions on people’s rights may be justifiable during a public health emergency, but people are being asked to sacrifice their privacy and turn over personal data for use by untested technologies,”[27] HRW digital rights researcher, Deborah Brown expresses. The HRW states that the privacy of individuals is paramount of concern to everyone involved. Their skepticism towards tech companies is because of how they have undermined a plethora of other rights.

In this instance, it must be observed that privacy rights should not be prioritized over another’s life but the rebutted quote follows “Years of progressive encroachments on privacy have numbed many Americans, and some even consider them fair trade-offs for the convenience and security of modern life… the erosion of privacy weakens democracy.”[28] While we must consider the potential power the government has over us, we have proof that Apple and Google are doing what they can to be as transparent and prioritize individual privacy at the core of its design.

Conclusion:

The number of cases reported globally has no justification for its magnitude. The United States alone is responsible for the outrageous 1.4 million cases. Due to a lack of national testing, PPE, contact tracing solutions is an unwarranted excuse. For this reason, contact tracing implementation needs to be deployed ASAP. This conceptually is unfathomable in the structure system of majority governments globally continuing to see cases of COVID-19 spark up. Time is of the essence and to prevail during this global pandemic, any measure of combat should be applied. This report shows Apple and Google have taken charge and great responsibility in blocking any intersection of privacy invasion in their contact tracing APIs.

Fight to defeat this-

Works Cited:

[1] https://www.ohchr.org/en/issues/digitalage/pages/digitalageindex.aspx

[2] https://www.forbes.com/sites/tjmccue/2020/04/24/iphone-and-android-app-for-covid-19-contact-tracing-will-be-strictly-opt-in-only/#170e0d95a322

[3] https://developer.apple.com/documentation/exposurenotification

[4] https://developer.apple.com/contact/request/download/Exposure_Notification_Addendum.pdf

[5] https://www.latimes.com/opinion/story/2020-05-12/coronavirus-tracing-app-apple-google

[6] https://developer.apple.com/documentation/exposurenotification

[7] http://www.iitk.ac.in/esc101/05Aug/tutorial/java/concepts/class.html

[8]https://developer.apple.com/documentation/exposurenotification/building_an_app_to_notify_users_of_covid-19_exposure

[9] https://www.dailymail.co.uk/news/article-8289061/Experts-warn-theres-stop-people-lying-NHSs-coronavirus-tracking-app.html

[10] https://www.bbc.com/news/technology-52366129

[11] https://www.businessinsider.com/france-attacks-apple-contact-tracing-app-2020-5

[12] https://developer.apple.com/documentation/exposurenotification/building_an_app_to_notify_users_of_covid-19_exposure

[13] https://developer.apple.com/documentation/exposurenotification/building_an_app_to_notify_users_of_covid-19_exposure

[14] https://covid19-static.cdn-apple.com/applications/covid19/current/static/contact-tracing/pdf/ExposureNotification-BluetoothSpecificationv1.2.pdf

[15] https://covid19-static.cdn-apple.com/applications/covid19/current/static/contact-tracing/pdf/ExposureNotification-BluetoothSpecificationv1.2.pdf

[16] https://www.privacytrust.com/news/location_tracking.html

[17] https://www.technologyreview.com/2020/05/04/1001060/google-and-apple-lay-out-rules-for-contact-tracing-apps/?truid=662988d130df8626624f5414311223ac&utm_source=the_download&utm_medium=email&utm_campaign=the_download.unpaid.engagement&utm_content=05-12-2020

[18] https://www.technologyreview.com/2020/05/04/1001060/google-and-apple-lay-out-rules-for-contact-tracing-apps/?truid=662988d130df8626624f5414311223ac&utm_source=the_download&utm_medium=email&utm_campaign=the_download.unpaid.engagement&utm_content=05-12-2020

[19] https://www.reuters.com/article/us-health-coronavirus-usa-apps/apple-google-ban-use-of-location-tracking-in-contact-tracing-apps-idUSKBN22G28W

[19] https://covid19-static.cdn-apple.com/applications/covid19/current/static/contact-tracing/pdf/ExposureNotification-BluetoothSpecificationv1.2.pdf

[20] https://covid19-static.cdn-apple.com/applications/covid19/current/static/contact-tracing/pdf/ExposureNotification-BluetoothSpecificationv1.2.pdf

[21] https://covid19-static.cdn-apple.com/applications/covid19/current/static/contact-tracing/pdf/ExposureNotification-BluetoothSpecificationv1.2.pdf

[22] https://covid19-static.cdn-apple.com/applications/covid19/current/static/contact-tracing/pdf/ExposureNotification-CryptographySpecificationv1.2.pdf

[23] http://web.mit.edu/Saltzer/www/publications/protection/Basic.html

[24] Denning, Peter J. et als. Great Principles of Computing. The MIT Press. 2015.

[25] https://www.technologyreview.com/2020/05/07/1000961/launching-mittr-covid-tracing-tracker/

[26] https://coronavirus.jhu.edu/map.html

[27] https://coronavirus.jhu.edu/map.html

[28] https://www.hrw.org/news/2020/05/13/covid-19-apps-pose-serious-human-rights-risks

[29] https://www.theatlantic.com/politics/archive/2020/04/coronavirus-pandemic-privacy-civil-liberties-911/609172/

Explore the Music World: Categorize Music by Mood

Shuxin Zhao

Abstract

Music mood classification is now becoming more and more important for most music streaming services. Mood classification system improves users’ experience with online music streaming services; besides, advertisers could analyze the relationship between music, users’ mood, and their behaviors to re-understand their target consumers emotionally. This article aims to explore how music mood classification system works technically, including the taxonomy and identification of the music mood, as well as the mechanism of machine learning on music mood classification system. Music mood classification becomes one of the best evidence of the music industry’s increasing reliance on artificial intelligence.

1. Introduction

With the booming development of network technology, there are more and more online streaming services. As the music consumption paradigm moves towards online streaming services, users have access to increasingly large online music library. In this case, the music classification system begins to play an important role in music discovery for users. Currently, the common classification method is mostly based on the external characteristics of music, such as singer, band, year, album, and so on. Such a method is convenient, but the disadvantage is also obvious. It ignores users’ direct feelings about the music, therefore, in the most time, this kind of classification cannot provide users with the most accurate recommendation of what they want. With the increasingly urgent need for high-quality music classification, many content-based music classification methods have been proposed, which extract features from the music itself, such as the melody, the performance style, etc. Among all these classification methods, music mood classification is one of the most popular methods, and many music companies have begun to work on this. For example, Gracenote started its sonic emotion classification about 10 years ago (Roettgers & Roettgers, 2017). Spotify acquired Echo Nest to define audiences by moods and activities using the music mood data (A “Hit” for Every Mood — Spotify’s Analysis of Our Emotional States, 2019). Users of Rok music service could use a mood grid to play songs based on their feelings (Hayes et al., 2014). 

However, a challenging problem that arises in this domain is that classifying hundreds of millions of songs in the music library manually is not practical at all. Manual classification is time-consuming work. Besides, it seems to be inevitable for people to make mistakes when classifying music personally. Therefore, the researchers turn to the power of AI for help to do this work. Briefly speaking, researchers will firstly set a dataset to train the machines, helping them detect different kinds of music mood. Researchers will extract features from music, and then put those features into a classifier to identify different kinds of music mood. With the advancement of AI, the algorithms iterate over and over again, and now there are various algorithms to achieve the goal of music mood classification.

2. Mood Taxonomy

When using music apps, users could find a variety of mood tags on the home page. For example, Gracenote has developed its mood taxonomy of more than 400 emotional qualities (Roettgers & Roettgers, 2017). However, too many emotional tags will make the mood classification complex. Thus, in most existing researches of music mood classification, the music moods are normally divided according to the two-dimensional emotion model. This model consists of two dimensions: valence (negative/positive) and arousal (low/high). Every single emotion can be placed on these two dimensions graphic (Munoz-De-Escalona, 2017). For example, happy is positive valence with medium arousal; calm is low arousal with neutral valence.

Based on this model, psychologist Robert Thayer proposed another model of mood in 1989 (Nuzzolo, n.d.). The model suggests that emotions are influenced by two factors: stress (happy/sad) and energy (energetic/calm). “Energy” corresponds to the arousal in the two-dimensional emotion model, while “stress” values to the valence. Besides, these two lines also divide music mood into four clusters: contentment, depression, exuberance, and anxious/frantic.

3. De-Blackbox the Music Mood Classification System

To achieve the goal of automatic music mood classification using computers, machine learning is indispensable. This section is going to discuss the algorithm, design, and complete approach of the mood classification system using content analysis, then providing a deeper understanding of how machine learning works in this area. Generally speaking, machine learning has the following four steps: input, feature extraction, classification, and output.

3.1 Music database

Before starting to do machine learning, researchers should firstly find a music database to train the model to detect the music mood. The quality of the music database is crucial for the model to get accurate prediction results. An appropriate music database must have music samples with different music moods. The dataset can be divided into a training set and a testing set. The larger the dataset, the more accurately the machine learning model predicts the music mood. There are two common music databases: Million Song Dataset and MIDI dataset. The Million Song Dataset is a freely-available collection of audio features and metadata for a million contemporary popular music tracks. The size of the dataset is around 280 GB. MIDI dataset comprises of 130,000 MIDI files. Unlike .mp3 or .wav files, the MIDI file doesn’t contain actual audio data, and it only explains how long and loud each note is played for. With the Python library music21, data and features can be extracted from MIDI files (Khurana, 2018).

3.2 Feature Extraction

Feature extraction is an important step in the process of machine learning. It reduces the data dimension of the original input and recombines the original features for subsequent use. In short, feature extraction has two functions: reducing data dimensions and sorting out existing data characteristics. As is known to all, most songs are composed of melody and lyrics. Therefore, to extract music mood effectively, we can do feature extractions from two aspects: audio and lyrics.

3.2.1 Audio Feature Extractions

Usually, there are four facets of audio features to detect the music mood: intensity, timbre, pitch, rhythm (Nuzzolo, n.d.). These four features correspond to physical quantities such as frequency, duration, amplitude, and spectrum distribution of vibration. In the mood map, intensity and timbre are associated with energy while rhythm and pitch are in combination with stress in Thayer’s mood model.

Intensity is the loudness of a song. It is considered to be the average volume across its entirety. Intensity is determined by the amplitude of the waveform. To be specific, intensity could be numeric by calculating the RMS (root mean square) of the signal. The RMS is found by taking the root of the average of several amplitudes squared. A higher RMS usually indicates a higher intensity, and a lower RMS would suggest a lower intensity with a calmer piece. (Nuzzolo, n.d.)

Timbre is the quality of the music, and existing results show that the timbre of the sound is determined primarily by the spectral information. Different timbres always have distinctive characteristics of waveform. There are some sub-features for extracting timbre: zero-cross, centroid, roll-off, entropy, and MFCCs (Mel Frequency Cepstral Coefficients) (Singh et al., 2010) (Laurier, 2011).

Pitch is determined by the frequency of sound. Faster vibration, higher pitch, and vice versa. As is expected, higher-pitched songs usually are happier, more exuberant and energetic, while lower-pitched songs fall into the categories of anxious/sad, calm or depressed (Nuzzolo, n.d.).

Rhythm is the music’s pattern. To calculate rhythm, there are five sub-features: average beat-spectrum, average tempo, average regularity, average positive autocorrelation, and average negative autocorrelation. Among these five features, tempo, beat-spectrum, and regularity are more important than the other two. High tempo and regularity in a song mean positive emotion while low rhythmic values mean negative emotion. (Singh et al., 2010).

One engineering challenge of mood feature extraction is to develop and improve the algorithms to analyze these music features of an audio signal. For example, to get the MFCCs feature, firstly we should pre-emphasize, frame, and window the audio signal. For each short-term analysis window, we could get the corresponding spectrum by FFT (fast Fourier transform). Next, we could obtain the Mel spectrum by processing the spectrum in the Mel-filtering. Last but not least, we could do a discrete cosine transform (DCT) on the Mel spectrum, then obtaining MFCC (Laurier, 2011). After all these steps, audio can be described by a series of cepstral vectors, and each vector corresponds to each frame.

3.2.2 Lyrics Feature Extractions

The aim of the lyric analysis is to refine the results of audio content analysis. As the lyrics do not follow any particular grammar rule in favor of making it more rhyming, in most cases, semantic analysis is not so helpful. Therefore, the Bag-of-Words technique combined with the Bi-gram model is used to extract mood from lyrics. Both techniques are not based on a semantic analysis of the text. These techniques are dependent on adjectives or a combination of two adjectives. It is also worth noting that lyrics analysis is more associated with valence, so the one-dimensional model is better for lyrics analysis rather than a two-dimensional model. In the one-dimensional model, lyrics are classified into two categories: positive emotions and negative emotions (Singh et al., 2010).

Bag-of-Words (BOW) is a collection of words where each word is assigned tags from a dictionary. A word can have different tags. Some tags are predefined as positive or negative according to mood, such as happy words, sad words, etc, while other tags are tagged based on previous tags (Singh et al., 2010). The lyric of each song is represented as a set of the 20 most frequent words (stems) in the song (Brilis et al., 2012), then we could calculate the emotional value based on the positive and negative word counts.

Bi-gram analysis can be considered as a pseudo-semantic analysis. It uses the same tags as of BOW but analyzes the combination of two words. Bi-gram could generate meaningful emotional value. Each combination of words is marked with a certain value lying between the range (-10, 10) depending on the mood they will elicit when combined together (Singh et al., 2010).

3.3 Classifier

The classifier is the most important part of the machine learning model, which needs to be trained with the previously extracted features. It is responsible for classifying the music into different music moods. There are some different types of classifiers that are often used: k-Nearest Neighbors (k-NN), support vector machine (SVM), neural networks.

 k-NN is one of the simplest classifiers and is easy to implement. It is a non-parametric, instance-based, lazy supervised algorithm. In k-NN, for each sample X in the testing set, select several nearest samples in the training set, and the mood of the music sample is determined by the majority of these “neighbors” (Khurana, 2018).

SVM is a common method. In machine learning, it is a supervised learning model commonly used for pattern recognition, classification, and regression analysis. SVM aims to find a hyperplane to divide the samples into two categories with the largest interval. The w represents the hyperplane coefficient that we need to find.

Neural Networks are very effective in most machine learning models. There are many layers in neural networks helping to classify the moods. And selected learned music mood features are reported with interpretations. This network depends on the complexity of the system, and it processes information by adjusting the connections between nodes. The artificial neural network has the ability of self-study, and it could learn the potential rules between the input and output by analyzing the data in advance. When there is new data input, it will predict the output according to the obtained rule. (Choi et al., 2016).

4. Use of Music Mood Classification

4.1 Music Recommendation System

The music mood classification system could improve the algorithms for identifying similar songs for online radio services, basing the similarities in the music mood rather than similar artists. Spotify has 85 playlists under the “mood” category. If users listen to the music on a weekday morning, they could choose the playlists like “happy wake-up songs”, “perfect day” and “get ready for the day”. If users are feeling down, then the playlist like “sad melody” would be a good choice for them. Over the years, streaming music services have promoted such playlists which are based on emotions. They have shown to their users that selecting playlists based on specific emotions can effectively help users find what they want in the massive music library.

4.2 Marketing

Music mood classification system could help users customize their own playlists, but there’s also a marketing boost behind the personalization process. Nowadays, for Spotify, music is no longer a commodity, but the emotion and behavior data attached to the music is the commodity. Spotify claims that with the users’ data, their bargaining power to sell sentiment and behavior data has promoted a lot. Spotify provides unique value for WPP and its customers by analyzing the relationship between music and users’ emotions and behaviors. The attributes of music, such as rhythm and strength, have been proved to be highly correlated with people’s emotions, which will allow advertisers to re-understand their target consumers in a new emotional dimension.

Spotify not only wants to monitor and explore users’ emotions but also wants to understand users’ future behaviors. Its ultimate goal is to predict users’ behavior through music. For example, if a user is playing a playlist with light music in the morning, then Spotify would speculate that the user may be doing yoga or meditating. Based on such speculation, Spotify could push advertising content that matches this scenario (音乐选择折射了你的情绪，而 Spotify 正利用这些数据大赚, 2019).

However, it comes with another ethical issue: is it right to sell users’ emotions to make a profit? Does such monitoring of users’ moods invade users’ privacy? Spotify has seen the potential value of music data, which might greatly stimulate their ambition to further collect, store, and recycle users’ personal data. They could even use these data to influence users’ behavior and daily decisions. In those cases, ensuring users’ data not being misused has become an important issue.

5. Conclusion

This article systemically explores how music mood classification works technically. It begins with an introduction of mood taxonomy, then de-blackboxing the machine learning on music mood classification in detail, including the dataset, feature extraction, and classifier. It is undeniable that AI and big data are playing an increasingly important role in the world, which helps people to live a more positive and convenient life. However, at the same time, how to protect our privacy in such an AI era has also become an urgent problem.

Bibliography

A “Hit” for Every Mood—Spotify’s analysis of our emotional states. (2019, August 5). MAEKAN. https://maekan.com/2019/08/a-hit-for-every-mood-spotifys-analysis-of-our-emotional-states/

Brilis, S., Gkatzou, E., Koursoumis, A., Talvis, K., Kermanidis, K. L., & Karydis, I. (2012). Mood Classification Using Lyrics and Audio: A Case-Study in Greek Music. In L. Iliadis, I. Maglogiannis, H. Papadopoulos, K. Karatzas, & S. Sioutas (Eds.), Artificial Intelligence Applications and Innovations (Vol. 382, pp. 421–430). Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-642-33412-2_43

Choi, K., Fazekas, G., & Sandler, M. (2016). Explaining Deep Convolutional Neural Networks on Music Classification. ArXiv:1607.02444 [Cs]. http://arxiv.org/abs/1607.02444

Hayes, T., Hayes, T., & Hayes, T. (2014, June 24). Ever Wonder How Music Is Matched To Your Mood? Fast Company. https://www.fastcompany.com/3032320/ever-wonder-how-music-is-matched-to-your-mood

Khurana, S. (2018, December 23). MUSIC CLASSIFICATION USING ARTIFICIAL INTELLIGENCE. Medium. https://medium.com/@sukantkhurana/music-classification-using-artificial-intelligence-3d21c59c5cb2

Laurier, C. (2011). Automatic Classification of musical mood by content-based analysis.

Munoz-De-Escalona, E. (2017). Online Measuring of Available Resources. https://doi.org/10.21427/D7DK96

Nuzzolo, M. (n.d.). Music Mood Classification | Electrical and Computer Engineering Design Handbook. Retrieved April 21, 2020, from https://sites.tufts.edu/eeseniordesignhandbook/2015/music-mood-classification/

Performance analysis of isolated Bangla speech recognition system using Hidden Markov Model. (n.d.). Retrieved April 25, 2020, from https://www.ijser.org/paper/Performance-analysis-of-isolated-Bangla-speech-recognition-system-using-Hidden-Markov-Model.html

Roettgers, J., & Roettgers, J. (2017, August 1). Sad Songs, Artificial Intelligence and Gracenote’s Quest to Unlock the World’s Music. Variety. https://variety.com/2017/digital/news/gracenote-ai-machine-listening-1202511912/

Singh, P., Kapoor, A., Kaushik, V., & Maringanti, H. B. (2010). Architecture for Automated Tagging and Clustering of Song Files According to Mood. 7(4), 7.

梅尔频率倒谱系数（MFCC） 学习笔记—BaroC – 博客园. (n.d.). Retrieved April 25, 2020, from https://www.cnblogs.com/BaroC/p/4283380.html

音乐选择折射了你的情绪，而 Spotify 正利用这些数据大赚. (2019, June 19). 爱范儿. https://www.ifanr.com/1226127