A project of the Integrity InstituteViolent extremism (VE) poses a significant challenge for online platforms. These platforms inadvertently serve as conduits for extremists from various ideological backgrounds to connect, share propaganda, and coordinate acts of violence. Due to algorithms that prioritize engagement, borderline or provocative content often gets amplified, leading users down rabbit holes that can end in extreme beliefs - many of which include acceptance of violence for political or ideological ends.
Platform design choices complicate the battle against VE. For instance, features like end-to-end encryption, while intended to ensure privacy, can shield extremists from scrutiny and make it difficult for trust and safety professionals to monitor and intervene in harmful activities. On the other end of the visibility spectrum, platforms giving users the power to livestream has been used by violent extremists to spread their message, and the terror it intends to produce (most notably in Christchurch NZ, in the aftermath of which the organization Christchurch Call, which focuses squarely on issues of VE on online platforms, was founded).
While violent extremism predates the internet, the digital landscape has evolved to inadvertently facilitate it. It is increasingly critical for platforms to look beyond simple content moderation, and ensure that their design decisions do not actively support or amplify the causes of violent extremists.