Harm:

Hate Speech

Definition: Explicitly advocating violence, discrimination, or prejudice toward individuals or groups based on immutable characteristics.
Motivation:
Ideological
Legal Status:
Illegal in some jurisdictions
Platform ToS:
Violates Policy
Victim Visibility:
Unaware
Classification:
Inherent in Content
TSPA Abuse Type:
Offensive & Objectionable Content: Hateful Content

Hate speech is generally understood as language that promotes or incites violence, discrimination or hostility against groups based on personal attributes like race, ethnicity, religion, gender, sexual orientation, or other protected or sensitive characteristics. While such speech may be protected by law in most liberal democracies, it's unmitigated spread and amplification on digital platforms is both unwanted by the platform, and deeply harmful to the communities that are targeted.

The relative anonymity and self-sorting that online platforms provide fosters an environment where users can feel comfortable expressing or reveling in prejudices and animosities that are outside of the realm of mainstream acceptability. While these sentiments may build up and circulate in small online communities, their impact rarely remains there - hateful and dehumanizing speech online emboldens the readers and writers of it to action in the physical world. Even worse, the attention (and outrage) maximizing algorithms that social networks use to amplify engaging content can easily have the impact of amplifying hateful, bigoted, or violent rhetoric. As a particularly appalling example, hate speech against the Rohingya (an ethnic minority) spread and amplified by Facebook between 2012 and 2017 contributed to and accelerated the genocide that occurred in 2017. Hate speech on platforms is not a theoretical concern.

What features facilitate Hate Speech?

By maximizing attention (and by proxy, outrage), recommendation systems can amplify hate speech.
Recommendation
A platform proactively inserting content into a user's view.
In providing mechanisms for private, ideologically homogeneous conversations, platforms can enable speech that isn't tolerated in the mainstream.
Messaging
Enable users to exchange text in real time.

How can platform design prevent Hate Speech?

Third repost has to manually enter the content
Slow the spread of virality by adding friction on repost/re-share functionality.
No Comments for Fresh Accounts
Don't accept comments from newly created or barely used accounts.
Affinity To Comment
Require a history of interaction between users before they're allowed to interact in comments.
Anonymous Limitations
Require users to create an account before they can use features that create data or interact with others.
Comment Tone Check Popup
Before posting content containing vulgarities, prompt a user to think twice.
Is something missing, or could it be better?
Loading...