Definition
Third-party safety evaluators given ongoing, employee-like access inside frontier AI labs — desks, badges, laptops, training-pipeline visibility — to verify safety practices, report incidents, and publish key findings without editorial control (subject to narrow security/legal redactions).
Key Points
- 2026-09-12: Step 1 of dario-amodei’s pace-the-frontier-framework; anthropic unilaterally commits; metr named as example evaluator (2026-09-12-dario-amodei-pace-frontier-primary)
- Replaces episodic pre-deployment audits with continuous in-lab oversight — modeled on banking regulatory supervisors
- Sam Altman said OpenAI will adopt similar embedded evaluator model
- Key architectural change: evaluators observe development as it unfolds, not snapshot audits
Related
- pace-the-frontier-framework
- dario-amodei
- anthropic
- metr
- ai-safety
- ai-governance
- frontier-lab-eval-safety
Operational Tier (October 2026)
- 2026-10-15: metr becomes first tier-3 evaluator at anthropic — six researchers with employee-like access and publication rights without veto (narrow redaction only); ties to White House Accord on Super Intelligence pledge (2026-10-15-anthropic-metr-employee-level-access)
- SecureBio remains tier-1 (scoped reviews, Anthropic approval to publish); tier-2 can publish without approval with less internal access
First Named Partner (September 2026)
- 2026-09-18: accenture Faculty staff embed inside anthropic — red-teaming, alignment assessment, safeguard testing (2026-09-18-anthropic-accenture-embedded-evaluator, embedded-evaluation)
- $1B+ five-year commitments; nonexclusive; metr pilots in dialogue