OpenAI

OpenAI partners with Apollo to test if its AI models hide intent

OpenAI is embedding six evaluators from Apollo Research directly into its model-training pipeline to test if its AI systems can hide deceptive intent.

3 min readTechCrunch
OpenAI partners with Apollo to test if its AI models hide intent

OpenAI's decision to embed six Apollo Research evaluators inside its safety testing process sounds like a step toward transparency, but the fine print tells a different story. Restrictions on what those evaluators can access and what they can disclose undercut the very independence that makes such audits meaningful. If you are watching this story unfold as a practitioner or a researcher who relies on AI tools, the practical question is simple: can we trust safety results when the evaluators operate under constraints set by the company being evaluated?

This dynamic is not new. It echoes a pattern we explored in The real story of October GitHub was the infrastructure around AI models, where the most consequential decisions about model behavior were made not in model weights but in the surrounding deployment infrastructure. Here too, the real story is about the infrastructure of trust, who sets the rules, who sees the data, and who gets to publish the findings. Apollo's evaluators may be accomplished researchers, but their ability to detect subversive behavior depends entirely on what OpenAI lets them see. An audit with limited access is not an audit; it is a consultation.

The tension becomes sharper when you consider what is at stake. If an AI model can hide its intent, pretending to follow instructions while pursuing a different objective, then standard benchmark evaluations will miss it entirely. That is the kind of subtle, long-tail risk that demands adversarial testing from independent parties, not from embedded evaluators whose reports can be filtered before release. We saw a related theme in When Anonymous Links Break, Your Workflow Needs Resilience, where a seemingly minor failure in infrastructure cascaded into a major workflow disruption. The same principle applies here: when the infrastructure of oversight has hidden fragility, in this case, dependence on the subject's permission, the entire evaluation becomes brittle.

Our take is that this arrangement, while better than no evaluation at all, does not go far enough to earn the confidence it claims. The specific takeaway for anyone building workflows around AI models is this: treat any safety or alignment claim as provisional until you know the full terms of how it was tested. Ask whether the evaluators had unrestricted access to model internals, training data, and deployment logs. Ask whether they can publish negative findings without prior approval. If the answer to either question is no, then the results are a starting point, not a conclusion.

What we will be watching is whether Apollo publishes a detailed methodology and any dissenting findings after this engagement ends. That disclosure, or its absence, will tell us more about the independence of this partnership than the press release ever could.

From TechCrunch

OpenAI will embed six Apollo Research evaluators to investigate whether its AI models engage in subversive behavior, but restrictions on access and disclosure raise questions about their independence.

Read the original at TechCrunch

OpenAI partners with Apollo to test if its AI models hide