Content Moderation on End-to-End Encrypted Systems: A Legal Analysis

Authors

Charles Duan, American University Washington College of LawFollow
James Grimmelmann

Document Type

Article

Publication Date

2024

Journal

Georgetown Law Technology Review

Volume

8

First Page

1

Last Page

92

Abstract

Online messaging platforms like Signal and Google’s Messages increasingly use end-to-end encryption (E2EE), in which messages are encrypted on the sender’s device and decrypted on the recipient’s, so that no one else—not even the platform itself—can read them. Although E2EE protects privacy and advances human rights, the law enforcement community and others have criticized its growing use. In their view, E2EE prevents platforms and government authorities from responding to abuses and criminal activity, including child exploitation, malware, scams, and disinformation. At times, they have argued that E2EE is inherently incompatible with effective content moderation.

Computer science researchers have responded to this challenge with a suite of technologies that enable content moderation on E2EE platforms. These technologies—message franking, forward tracing, homomorphic encryption, and automated client-side scanning— preserve some of the essential privacy guarantees of E2EE while enabling the targets of abuse to detect and report it. These technical advances, however, raise legal questions. If E2EE messages are supposed to be private from a messaging platform, and the platform participates in detecting whether those messages are abusive, is that an “interception” of an “electronic communication” prohibited under the Wiretap Act?

This Article analyzes these new E2EE content moderation technologies in light of six major federal communication statutes: the Wiretap Act, the Stored Communications Act, the Pen Register Act, the Computer Fraud and Abuse Act, the Communications Assistance for Law Enforcement Act, and the PROTECT Our Children Act. While generally we find that these content moderation technologies would pass muster under these statutes, the answers are not as clearcut as one might hope. The advanced cryptographic techniques that these new content moderation strategies employ raise multiple unsettled questions of law under the communication privacy regimes considered. This legal uncertainty arises not because of the ambiguous ethical nature of the technologies themselves, but because the decades-old statutes failed to accommodate, or indeed contemplate, the innovations in cryptography that enable content moderation to coexist with encryption. To the extent that platforms are limited in their ability to moderate end-to-end encrypted content, then, those limits may arise not from the technology but from the law.

Recommended Citation

Charles Duan & James Grimmelmann, Content Moderation on End-to-End Encrypted Systems: A Legal Analysis, 8 Georgetown Law Technology Review 1 (2024).
Available at: https://digitalcommons.wcl.american.edu/facsch_lawrev/2252