What I thought AI would solve. And what it actually taught me.

TL;DR

• Started with a statistical method at work — effective, but no explanatory power
• Rebuilt it with AI after leaving — accurate, but still a black box
• Ran three analyses to understand the data — found that no single method is enough
• Redesigned around evidence-first decision support instead of automated classification
• The hard part was never the model. It was knowing what question to ask.

Overview

This project started as a real operational problem: how do you handle review reports that nobody can verify?

On the platform I worked on, users could flag reviews for reasons like "false information" or "posted by someone affiliated with the community." These are impossible to confirm — there's no way to know if a review is truly false, or if the person who wrote it has a hidden conflict of interest. As a result, the abuse team routinely ignored these reports.

I tried to solve this three times. Each attempt taught me something the previous one couldn't.

My Role

This project spans two contexts: work I did as an Associate Product Manager at Internet Brands, and personal projects I built after leaving to explore AI-assisted moderation design.

The Problem

The platform had nine flag reasons. Five of them — toxic content, promotional material, privacy violations, COVID misinformation, off-topic content — could be reviewed and acted on by the abuse team. The other four couldn't:

• It contains false information