Try to imagine 2 billion of anything. It’s genuinely too hard for the human brain to comprehend such a gargantuan number, and yet Facebook serves 2.27 billion monthly users (Abbruzzese). With the dizzying array of opinions and demands that 2.27 billion global users have of the platform, how does Facebook decide who to please and why to please them? How does Facebook moderate the massive amount of user-generated content on its platform? How is artificial intelligence used to automate content moderation on Facebook?
Why does Facebook moderate content?
First and foremost, Facebook is a business that aims to make a profit. Most of Facebook’s revenue is gained from selling advertisements to third parties, as Mark Zuckerberg concisely explained during a congressional hearing in April 2018:
What we allow is for advertisers to tell us who they want to reach, and then we do the placement. So, if an advertiser comes to us and says, ‘All right, I am a ski shop and I want to sell skis to women,’ then we might have some sense, because people shared skiing-related content, or said they were interested in that, they shared whether they’re a woman, and then we can show the ads to the right people without that data ever changing hands and going to the advertiser. (Gilbery)
Within this business model, one can summarize the ultimate goal of Facebook in its relationship with its users into two steps: (1) To keep users engaged and on the platform so that the advertisements can be seen and engaged with and (2) to promote users to generate content so that Facebook can extract more detailed behavioral insights with which to target ads. Basically, Facebook operates within the model of surveillance capitalism to make a profit (Laidler).
Thus, Facebook has a bona fide economic incentive to maximize the number of its users who feel that they are safe to post what they want without fear of censorship or peer-mediated attack. Additionally, as Facebook is a network in which users create the content that other users consume, Facebook needs users to trust that they will not be offended each time they open the site. These incentives to maximize the amount of user-generated content on its platform are reflected in the descriptions of the principles that Facebook includes within its public-facing Community Standards.
When discussing the concept of safety and why it’s important to Facebook, Facebook says “People need to feel safe in order to build community,” suggesting that the reason threats and injurious statements are not welcome on the platform is because this type of content chills the process of community formation (Community Standards). The concept of increasing the number of opinions and ideas that can exist on the platform again resurfaces in Facebook’s description of “Voice” as a defining principle of its community standards. Facebook states, “Our mission is all about embracing diverse views. We err on the side of allowing content, even when some find it objectionable, unless removing that content can prevent a specific harm.”
So, economically at least, there is a reason that Facebook is heavily invested in content moderation. Facebook wants its platform to be a pleasant place to retain users so that Facebook can sell more ads.
Issues of free speech
This section could be short. Constitutionally in the United States, Facebook has no legal mandate to remove or allow speech on its platform. On the internet, many interactive service providers, also referred to as platforms, have traditionally backed the idea that the internet should be a place open to free expression and the marketplace of ideas, but they have no legal mandate to do so (Snider).
The First Amendment states that “Congress shall make no law respecting an establishment of religion, or prohibiting the free exercise thereof; or abridging the freedom of speech.” Notably, the First Amendment does not protect citizens from the actions of other private parties. The Supreme Court confirms this interpretation in Hurley v. Irish- American Gay Group of Boston 515 U.S. 557, 566 (1995), in which it states, “the guarantees of free speech . . . guard only against encroachment by the government and ‘erect] no shield against merely private conduct.’” Even though the first amendment does not explicitly protect users from having their speech censored by Facebook, users are still angry and invoke rhetoric that suggests their rights are being mutilated when Facebook censors them.
Alex Abdo, a senior staff attorney with the Knight First Amendment Institute references an idea of a “broader social free speech principle” and summarizes this frustration as the result of a societal expectation in the United States: “There is this idea in this country, since its founding, people should be free to say what they want to say” (Snider).
The lines between Facebook censorship and government censorship are understandably blurred when even the Supreme Court of the United States uses language that equates social media sites to the traditional concept of the “public forum.” In PACKINGHAM V. NORTH CAROLINA 137 S.Ct. 1730 (2017), a case in which North Carolina made accessing many common social media sites a felony for registered sex offenders, the Supreme Court extended the idea that in many ways, contemporary social media acts as a public forum: “While in the past there may have been difficulty in identifying the most important places (in a spatial sense) for the exchange of views, today the answer is clear. It is cyberspace—the ‘vast democratic forums of the Internet’ in general, Reno v. American Civil Liberties Union, 521 U.S. 844, 868 (1997), and social media in particular.” (Grimmelmann, 2018).
Section 230 immunity
Compounding this societal confusion around the legality of content moderation decisions on Facebook is the immunity that Facebook receives under Section 230 of the Communications Decency Act. Under Section 230, Facebook is completely immune from liability for most non-illegal, non-copyright that users upload onto its platform. Additionally, Section 230 contains a “Good Samaritan” provision that allows “interactive computer services” to take somewhat of an editorial stance by removing content that they deem offensive in their platforms without accumulating any liability (COMMUNICATIONS DECENCY ACT, 47 U.S.C. §230). This law is the reason that Facebook can “develop their own community guidelines and enforce them as they see fit” (Caplan).
Mounting pressure for Facebook to do something
No legal imperative exists for Facebook to moderate its content outside of copyrighted and illegal content. The legal imperative, however, is not what many users think about, and as we’ve established that Facebook makes its money from its users, it is easy to understand that Facebook will want to listen to their demands.
There is an increasing anxiety about Facebook’s massive scope coupled with the dependence people have on Facebook and the platform’s ultimate ability to filter speech. As the Supreme Court alluded to in Packingham v. North Carolina, the internet – especially social media sites – have become the main place for people to express themselves and their ideas in contemporary society. With over 2 billion users, Facebook is larger than many sovereign countries and has ultimate power over all user content, and users have no source of structural system to contest decisions or advocate for themselves within this system.
Additionally, following the 2016 United States presidential election, many people were frustrated with Facebook’s apparent negligence in controlling the spread of misinformation on the platform. Users demanded that Facebook do more to stop the spread of false information on the platform because in this role of a news delivery source – in which Facebook’s newsfeed algorithm makes editorial choices about what to show a user – rather than strictly a social networking platform, some users believe that Facebook has the obligation to ensure a certain standard of news content. Following the political violence in Myanmar and the rise of white nationalism on the platform to name a few instances, some people are also calling for Facebook to do more to moderate content that contributes to political radicalization (Mozur).
How does Facebook currently moderate content?
Facebook uses artificial intelligence tools like machine vision and language processing to flag content that might violate its Community Standards, and from this point, the content is sent to one of the company’s over 15,000 human moderators (Community Standards). At the root of Facebook’s content moderation execution, separate from its public-facing Community Standards, is an ad-hoc system of PowerPoint slides that contain rules which attempt to distill ethically and politically vague dilemmas into binary moderation decisions. This simplification of difficult moderation decisions is part of an attempt to uniformly train its over 15,000 human content moderators to deal with the absolute avalanche of content that needs to be checked each day on Facebook.
Some of these moderators are contracted workers with Facebook and may be forced to work on moderating content that is in a language that the moderator does not understand or situated within a foreign country. The New York Times reported that the rules moderators get to execute moderation decisions are “apparently written for English speakers relying on Google Translate, suggesting that Facebook remains short on moderators who speak local languages” (Fisher).
In addition to the Powerpoint slides of moderation rules, Facebook has an excel-style spreadsheet of groups and individuals that have been banned from the platform for being a “hate figure.” Moderators are instructed to remove any content on Facebook that is praising, supporting, or representing any of the listed figures. This blanket-coverage strategy is meant to make moderation simpler for human moderators, but drawing hard lines on content regardless of context can chill political speech or work to maintain a status quo for certain groups in power. As Max Fisher reports in the New York Times, “In Sri Lanka, Facebook removed posts commemorating members of the Tamil minority who died in the country’s civil war. Facebook bans any positive mention of Tamil rebels, though users can praise government forces who were also guilty of atrocities.”
Facebook, in taking a stance that content can be shared on its platform depending on the context of the post around it, has limited its ability to fully automate certain aspects of content moderation. On Thursday, May 2, 2019, Facebook banned Alex Jones and all InfoWars-related content from its platform with the caveat that content from this publisher could be shared if the commentary about the content is critical of the message. While AI systems have the capability to conduct sentiment analysis human moderation is required to accurately moderate content according to a viewpoint-based policy (Martineau).
Facebook’s use of stringent Powerpoint-delivered rules coupled with the ultimate subjectivity of a human moderator suggests that Facebook would like to combine the best features of context sensitivity and consistency in content moderation. But, as the continued societal outrage against pretty much every content moderation decision Facebook makes suggests, the current model is not successful for them.
Content moderation experts Tarleton Gillespie and Robyn Caplan have narrowed down most content moderation categories into three groups according to size organization and content moderation practices. (1) The artisanal approach is a tactic in which around 5 to 200 workers govern content moderation decisions on a case-by-case basis. Most social media sites begin their moderation with this approach, and then are forced to adapt the process as a case-by-case scale becomes overwhelming. (2) The community-reliant approach, seen on sites like Wikipedia and Reddit, combines formal policy made at the company level with volunteer moderators from the site’s community. (3) Finally, the industrial approach is the model Facebook uses, in which “tens of thousands of workers are employed to enforce rules made by a separate policy team. (Caplan, 2018).
At all levels, content moderation must deal with the tension between context sensitivity and consistency, and accept different trade-offs between the two. Facebook’s industrial approach, as shown in the global reach of its Community Standards, is one that greatly favors consistency over context sensitivity. In her report on online content moderation, Robyn Caplan confirmed Facebook’s approach to consistency over context sensitivity with one of her interviewees from Facebook:
One of our respondents said the goal for these companies is to create a “decision factory,” which resembles more a “Toyota factory than it does a courtroom, in terms of the actual moderation.” Complex concepts like harassment or hate speech are operationalized to make the application of these rules more consistent across the company. He noted the approach as “trying to take a complex thing, and break it into extremely small parts, so that you can routinize doing it over, and over, and over again.”
Thus, Facebook’s eagerness to adopt AI for content moderation makes a lot of sense. One of the largest trade offs organizations make when they choose to automate content moderation is for consistency over context sensitivity.
This eagerness for Facebook to adopt AI is not hidden at all. Mark Zuckerberg publically has high hopes for the role of artificially intelligent automation in content moderation, as he stated in response to questioning about Facebook’s role in allowing harmful content in Myanmar, “Over the long term, building AI tools is going to be the scalable way to identify and root out most of this harmful content” (Simonite). Automation at this scale makes a lot of sense from a logistical standpoint. With around 2.3 billion monthly users, the idea of solely using human moderators is ludicrous. Additionally, Mike Schroepfer, Facebook’s chief technology officer said that he thinks “most people would feel uncomfortable with that” in reference to purely human content moderation. Schroepfer went on to say, “To me AI is the best tool to implement the policy—I actually don’t know what the alternative is” (Simonite).
Conversely, some may feel that absolute automated content moderation is just as unnerving. Facebook’s former chief security officer, Alex Stamos, warned that increasing the demand for AI content moderation is “a dangerous path,” and that in“five or ten years from now, there could be machine-learning systems that understand human languages as well as humans. We could end up with machine-speed, real-time moderation of everything we say online” (Lichfield).
Creepiness aside, Facebook has already begun to implement AI moderation into its content moderation process in several ways. In all categories of moderated content, Facebook utilizes AI filters to flag content for review by its legion of human moderators. In most of these categories – spam, fake accounts, Adult Nudity and Sexual Activity, Violence and Graphic Content, Child Nudity and Sexual Exploitation, and terrorist propaganda – 95-99.7% of content was actioned upon before other Facebook users reported it. Most of these categories involve clear-cut definitions of what is and is not objectionable, so AI is easily trained to recognize offensive categories. Violence and spam are, generally speaking, globally recognized and not prone to metamorphosing definitions. Training data does not have to include myriad cultural contexts and definitions to pin down definitions of these types of offensive content. Conversely, only 14.9% of Bullying and Harassment was found and actioned upon before Facebook users reported it. The cultural definitions of bullying and harassment change constantly, and different words and even emoji can suddenly morph into offensive slurs and harassment (Community Standards). Because Facebook’s AI content filters require a lot of manual training and labeling, it isn’t yet plausible to produce an AI filter that can respond and adapt to the changing cultural contexts that endeavor to forever evolve the definitions of bullying and harassment.
Facebook is gigantic. The scale at which the platform operates, within so many countries and with so many stakeholders, requires that the platform moderate certain types of content to keep its users safe and placated within the platform. Using an industrial approach to content moderation, Facebook values consistency in its content moderation over case-by-case considerations for context of the content. This consistency-favoring approach shows in Facebook’s enthusiastic adoption of AI-enhanced content moderation.
Abbruzzese, J. (2018, October 30). Facebook hits 2.27 billion monthly active users as earnings stabilize. Retrieved May 5, 2019, from NBC News website: https://www.nbcnews.com/tech/tech-news/facebook-hits-2-27-billion-monthly-active-users-earnings-stabilize-n926391
Caplan, R. (2018, November 14). Content or Context Moderation? | Data & Society. Retrieved from https://datasociety.net/output/content-or-context-moderation/
COMMUNICATIONS DECENCY ACT, 47 U.S.C. §230
Community Standards. (n.d.). Retrieved May 5, 2019, from https://www.facebook.com/communitystandards/
Fisher, M. (2018, December 27). Inside Facebook’s Secret Rulebook for Global Political Speech. The New York Times. Retrieved from https://www.nytimes.com/2018/12/27/world/facebook-moderators.html
Gilbery, B. (2018, April 23). Facebook says its users aren’t its product – Business Insider. Retrieved May 5, 2019, from Business Insider website: https://www.businessinsider.com/facebook-advertising-users-as-products-2018-4
Grimmelmann, J. (2018). Internet law: cases and problems(Eigth edition). Lake Oswego, OR: Semaphore Press.
Laidler, J. (2019, March 4). Harvard professor says surveillance capitalism is undermining democracy. Retrieved May 5, 2019, from Harvard Gazette website: https://news.harvard.edu/gazette/story/2019/03/harvard-professor-says-surveillance-capitalism-is-undermining-democracy/
Lichfield, G. (n.d.). Facebook’s leaked moderation rules show why Big Tech can’t police hate speech. Retrieved May 5, 2019, from MIT Technology Review website: https://www.technologyreview.com/f/612690/facebooks-leaked-moderation-rules-show-why-big-tech-cant-police-hate-speech/
Martineau, P. (n.d.). Facebook Bans Alex Jones, Other Extremists—but Not as Planned | WIRED. Retrieved May 5, 2019, from Wired website: https://www.wired.com/story/facebook-bans-alex-jones-extremists/
Mozur, P. (2019, March 4). A Genocide Incited on Facebook, With Posts From Myanmar’s Military. The New York Times. Retrieved from https://www.nytimes.com/2018/10/15/technology/myanmar-facebook-genocide.html
Simonite, T. (n.d.). AI Has Started Cleaning Up Facebook, but Can It Finish? | WIRED. Retrieved May 5, 2019, from Wired website: https://www.wired.com/story/ai-has-started-cleaning-facebook-can-it-finish/
Snider, M. (2018, August 9). Why Facebook can censor Infowars and not break the First Amendment. Retrieved May 5, 2019, from USA Today website: https://www.usatoday.com/story/tech/news/2018/08/09/why-facebook-can-censor-infowars-and-not-break-first-amendment/922636002/