A feature published by The New Yorker on Oct. 23, 2026, describes Meta’s school-threat detection as a sequence: automated systems flag possible warning signs, human investigators review broader activity, and they may refer information to law enforcement when they find evidence of substantial steps toward violence. The feature reports around 800 such referrals worldwide in 2025. That is a count of referrals, not attacks prevented.
How Meta’s reported referral process works
The automated flags are a starting point for human review. Investigators consider a person’s posts, shares, interactions and prompts to AI chatbots. Meta’s law-enforcement-outreach leader, Emily Vacher, described referrals when investigators find substantial steps, such as a target list or manifesto.
Vacher helped assemble Meta’s investigative group in 2018 to study past attackers’ online activity and behavioral patterns. The group worked with engineers on automated detection systems.
Two reported cases show the handoff to authorities
In Texas, a Meta team supplied the initial tip that led to an FBI inquiry involving two teenagers in February 2025. The New Yorker dates an FBI agent’s call to Spring Branch Independent School District police chief Larry Baimbridge to Feb. 18. The feature says the teenagers later pleaded guilty to separate offenses, based on its account of FBI information.
In a separate case in fall 2025, Meta reportedly flagged an Instagram post by a student at Benjamin N. Cardozo High School and passed information to the FBI. The FBI notified the New York City Police Department. Police found a loaded pistol in the student’s backpack and took the student into custody just over two hours after the post.
Why threat detection remains difficult
Coded references, changing images and symbols, and the insider knowledge of online communities can make potential threats harder for moderation systems to interpret. The New Yorker also describes the risk of overreporting, which could draw young people into the criminal-justice system for morbid expression rather than credible threats.
Katherine Keneally, a threat-analysis and prevention director at the Institute for Strategic Dialogue U.S., raised a broader concern: referrals address individual cases, while platforms’ recommendation systems can also spread violent material.
