OpenAI is using hundreds of contractors to read real ChatGPT conversations and evaluate AI-generated answers, according to a September 14, 2026 investigation by 404 Media. Internal documents identify the initiative as Project Lily. The material reaching reviewers can include sensitive personal information.
The relevant permission is ChatGPT’s “Improve the model for everyone” setting, which OpenAI says is enabled by default in personal Free, Plus, and Pro workspaces. Switching it off excludes new conversations from training, but it is not a retroactive withdrawal of everything previously shared.
The story exposes an important distinction: removing an account name, restricting employee access, and preventing human review are different privacy protections.
Project Lily Grades the Chatbot’s Behavior
The reported workflow involves more than checking conversations for prohibited content. Contractors read a user’s prompt, summarize what the person wants, and assess four possible chatbot responses on a one-to-seven scale. Their work produces detailed judgments about how ChatGPT should answer real requests.
According to 404 Media, reviewers are instructed to reduce sycophancy, meaning excessive agreement or flattery, and discourage the chatbot from presenting itself as human. Those are behavioral training goals, rather than simply decisions about whether a conversation violates a safety policy.
The distinction matters because an ordinary, permitted conversation can still be useful for model improvement. A request does not have to be suspicious to become evaluation material when training is enabled. OpenAI’s consumer documentation explicitly distinguishes access for improving model performance from access for investigating abuse.
These evaluations also should not be confused with contractors secretly composing every live ChatGPT answer. The reporting describes people reviewing prompts and generated responses to improve models, not a human-operated replacement for the chatbot. Nor does the reported workforce size establish that every eligible conversation is read.
Removing Names Does Not Make Chats Anonymous
OpenAI says reviewers do not receive ChatGPT usernames and that conversations pass through a version of its Privacy Filter before reaching contractors. Some tasks nevertheless include a “user memories summary”, providing context from previous interactions that can indicate where someone lives.
Privacy Filter detects and masks categories of personally identifiable information, including names, addresses, email addresses, phone numbers, account numbers, and secrets such as passwords. It identifies relevant stretches of text rather than rewriting the entire conversation into a privacy-safe abstraction.
Sources
No external sources are listed for this article.
