OpenAI opens its internals to four outside assessors: Apollo, METR, Transluce and Goodfire move on-site to probe alignment
Synopsis
OpenAI has signed agreements with four independent assessors—Apollo Research, METR, Transluce and Goodfire—for a first phase of embedded assessments in which they work on-site at OpenAI offices on OpenAI devices and reach model checkpoints, source code, training and evaluation data, monitoring systems and the researchers behind the work, focusing on alignment and safeguards for models in training and internal use. Earlier external testing was episodic and limited in access; this round lengthens both access and duration, but no assessment findings have been released.
Interpretation
For the first time, independent outside organizations enter OpenAI as embedded assessors to evaluate alignment and safeguards for models during training and internal use, rather than testing only released models from outside. Prior third-party evaluation was largely episodic testing; OpenAI says this phase builds on its earlier third-party assessment work and follows a recent pilot assessment of its internal monitoring systems and an investigation of the Hugging Face incident. Four signed agreements assign distinct tasks: Apollo Research examines whether models could covertly work against developer intentions and whether safeguards would detect it; METR assesses misaligned behavior in the largest reinforcement-learning run to date and separately conducts a monitor assessment; Transluce reviews the quality and coverage of post-training alignment evaluations; Goodfire analyzes model internals with its platform.
Assessors receive access to some of the same internal resources OpenAI researchers use, including model checkpoints, source code, training and evaluation data, monitoring systems, technical documentation, and discussions with the researchers and engineers responsible for the work. That depth is meant to let assessors pursue independent evaluations and uncover failures OpenAI may not have anticipated, rather than answer only pre-set questions. Assessors needing sensitive confidential information work on site at OpenAI offices on OpenAI-provided devices and follow at minimum the same security standards as OpenAI employees; employee discussions are off the record and cannot be published or attributed, while assessors may request OpenAI-attested statements through a separate pathway.
Perspective
The report covers only the first-phase embedded assessment arrangements OpenAI published: each organization's assessment target, the resources made accessible, on-site and security requirements, oversight and dispute resolution, employee discussions and attestation, conflicts and cool-off periods, and publication and responsible-disclosure periods. No assessment results, findings or conclusions have been released, and this report makes no measured judgment about model performance or safety.
OpenAI says the protocols will need to evolve as it learns, and that early stages may bring technical access challenges, space constraints, evolving security protocols and other unexpected issues. The scope of a second monitor assessment with METR is pending mutual agreement; METR's alignment assessment here is less expansive than the broad misalignment survey its own proposal envisions; and the roughly eight-week follow-on with Transluce is set for early 2027 and remains an intent.
