OpenAI has a program, reported as "Project Lily," where hundreds of contractors read real ChatGPT conversations and rate the model's responses to improve it. That's the core of 404 Media journalist Joseph Cox's September 14, 2026 report, based on leaked internal documents — and it confirms something most of ChatGPT's 900+ million weekly users likely assume isn't happening to their conversations.
None of this is exotic or malicious. It's ordinary RLHF (reinforcement learning from human feedback) work, the same technique used to shape most production chat models. But the gap between what people assume about a "private" AI conversation and what actually happens to it is the real story — and it's a smaller-scale preview of a gap most organizations have with their own AI usage.
TL;DR
| Question | What's confirmed |
|---|---|
| What is Project Lily? | An OpenAI program where contractors read full ChatGPT conversation threads to rate response quality |
| Who reported it? | 404 Media's Joseph Cox, September 14, 2026, based on leaked internal documents |
| Do reviewers see who I am? | OpenAI says no usernames, but says sensitive details can still slip through filters |
| Is this OpenAI-specific? | No — Anthropic has confirmed the same practice for Claude |
| What can I do about my own ChatGPT use? | Disable "Improve the model for everyone"; avoid pasting identifying detail |
| What's the bigger version of this problem? | Organizations have little to no visibility into what employees paste into any AI chat or agent tool — chat or coding — day to day |
What Project Lily actually is
According to the leaked documents Cox reviewed, contractors working on Project Lily get access to complete conversation threads — not isolated, anonymized snippets — and rate how well the model's responses perform. The stated goals are specific: reduce the model's tendency toward anthropomorphization (talking or acting like it has feelings), and decrease sycophancy, where the model over-agrees with a user rather than pushing back.
Sycophancy isn't an abstract quality issue here. The reporting connects it directly to real-world harm: it's been cited in wrongful-death lawsuits alleging that ChatGPT's over-agreeable behavior contributed to user suicides. Reviewing what the model actually said in real conversations is, in a narrow sense, the same fix OpenAI would need to make regardless — you can't tune a model's tendency to over-agree with people without someone reading examples of it doing that.
The tension is that fixing it requires exactly the kind of access most users assume doesn't exist: a human being reading what they actually typed.
What OpenAI says about privacy safeguards
OpenAI told 404 Media that contractors reviewing conversations under Project Lily don't see the usernames attached to them. But the same reporting notes OpenAI's own acknowledgment that sensitive details — health information, financial details, other identifying content — can still slip through the filters meant to screen conversations before a contractor sees them.
That's a meaningful, honest caveat, not a loophole framed as a feature: text-based content filtering on free-form conversation is inherently imperfect. A user doesn't fill out a "sensitive information" field before typing; they just type, and whatever filter runs afterward is working against unstructured prose written by someone who had no reason to think about what a filter would catch.
This is not just an OpenAI story
The 404 Media reporting includes confirmation that Anthropic uses human review of real Claude conversations too, for similar quality and safety purposes. That context matters: this isn't a scandal unique to one company's practices, it's how RLHF-based consumer chat products are built across the industry. Improving a model on real usage requires looking at real usage.
That doesn't make the privacy tradeoff disappear — it just means the honest framing is "this is standard practice you should account for," not "this is one company behaving badly."
What to actually do about your own ChatGPT use
The practical guidance from the reporting is straightforward:
- Turn off "Improve the model for everyone" in ChatGPT's data controls — this is the setting that opts your conversations into this kind of review and training pipeline.
- Don't treat a chat window as a diary. Avoid pasting identifying health, financial, or otherwise sensitive personal detail into a consumer AI chat unless you'd be comfortable with a stranger reading it verbatim.
- Assume the same is true of whatever chat tool you use next. Given Anthropic's confirmation, switching products doesn't remove the practice — it just changes who's reading.
The version of this problem organizations actually have
Project Lily is a story about a company reviewing conversations it controls, with a stated purpose and (imperfect) filtering in front of it. Most organizations have a worse version of the same exposure, with none of that structure: employees using AI chat tools and coding agents on personal accounts, with no policy, no review, and no record of what left the building.
That's the pattern behind 2026's DLP telemetry showing employees paste something sensitive into an AI chatbot roughly every three days, mostly on unmanaged personal accounts a company has zero visibility into — not through any single dramatic leak, but through the accumulated, unlogged ordinary use of tools that read exactly like the ones behind Project Lily.
The same blind spot extends past chat windows into what AI coding agents do on a developer's machine: the shell commands they run, the files they touch, the credentials they can reach. If a consumer chatbot with a stated privacy policy and content filters still needs a caveat about what slips through, it's a reasonable baseline assumption that an unmonitored coding agent with filesystem and network access deserves at least as much scrutiny — and today, most organizations have none.
This is the problem Beam is built around: recording what an AI agent actually does — locally, on the device, before anything is a matter of trusting a vendor's own review pipeline — rather than finding out only when something goes wrong. Beam Sentinel, Beam's browser extension, checks text against known-sensitive patterns before it's sent to ChatGPT, Claude, Gemini, Perplexity, Copilot, or Grok, and pauses high-severity matches for an explicit decision instead of letting them go out silently. On the coding-agent side, Beam's local collector keeps a record of an agent's shell commands, file writes, and MCP tool calls, so "what did the agent actually do" has an answer that doesn't depend on asking the agent.
None of that requires trusting Beam with the data either — the checks and the recording happen locally, not in a cloud pipeline reviewed by anyone else's contractors. The goal isn't a different company reading your conversations; it's giving the organization actually responsible for the risk a record it can inspect itself.
Related reading
- How much sensitive data do people share with AI chatbots? The 2026 numbers
- What is AI agent monitoring? Definition, scope, and tooling
- Enterprise data agents are the new access-control problem, not just a new feature
- MCP security: a practical guide
- Original reporting, via explainx.ai: OpenAI's Project Lily: human review, ChatGPT, and privacy in 2026
Facts about Project Lily are as reported by 404 Media (Joseph Cox, September 14, 2026) and summarized by explainx.ai, current as of this post's publish date. Beam Sentinel's and Beam's local collector's capabilities reflect chrome-apps/beam-sentinel and apps/sentinel-collector's READMEs as of September 15, 2026.