This course aims to teach learners how to effectively measure and prioritize abuse on the internet, specifically focusing on spam, phishing, and fake accounts. The course covers the challenges of detecting abuse, the importance of good measurement in prioritizing work and analyzing impact, and various approaches used by Facebook's integrity teams for measurement and prioritization. Learners will also learn about the "Abuse Uncertainty Principle" and strategies for ensuring metrics inform key decisions. The teaching method includes a lecture format with real-world examples and scenarios. This course is intended for individuals interested in online security, abuse detection, and data analysis in the tech industry.
The Abuse Uncertainty Principle, and Other Lessons Learned from Measuring Abuse on the Internet - David Freeman - USENIX Enigma Conference - 2020
USENIX Enigma Conference via YouTube
Overview
Syllabus
Intro
Sometimes I wish I worked on ad ranking...
Abusive content on Facebook
Measuring abuse is hard!
Measuring abuse: The Dark Ages
Measuring Abuse: The Renaissance
Benefits of Measurement (in addition to a quieter office)
How do we get there?
Count what you blocked These things have to be bad, right?
Have users do the labeling Crowdsource the work via reporting/appeals flows.
Human labeling Get experts to decide for you.
Deployment at Facebook Used for internal direction and/or external reporting of
Deployment at Facebook Used for internal direction on
Comparison of approaches
Example: Rate Limiting
"Proof" of Goodhart's Law
Corollary: "Abuse Uncertainty Principle"
Living with the Uncertainty Principle For spam detection at Facebook, we split signals into two classes
Other coping mechanisms
Challenge: Build an Abuse Adapter!
Open questions
Thank you!
Taught by
USENIX Enigma Conference
