Anthropic has started giving six researchers from METR, a nonprofit that evaluates frontier AI models, what the company calls "employee-like access" to its systems. The arrangement, set out in an Anthropic post dated October 15, is the deepest form of independent AI evaluation any major lab has described in public so far. The METR team gets company laptops, desks in Anthropic's offices and access to internal tools roughly on par with Anthropic's own risk assessment staff. METR can also publish what it finds without Anthropic signing off.
The move matters because the debate over AI oversight has largely been about who gets to look inside the labs, and on whose terms. Until now, most outside testing of frontier models has happened through narrow, pre-agreed projects, often with the lab holding a veto over what gets published.
What the independent AI evaluation deal gives METR
According to Anthropic, the terms are as follows:
- People: six METR employees work under the arrangement.
- Equipment and space: they receive company hardware and office space.
- Systems: their access to internal tools and systems is described as roughly on par with Anthropic's own risk assessment teams.
- Information: Anthropic staff are encouraged to share information with them, apart from three defined categories: customer data, attorney-client privileged material and model architecture details.
- Publication: METR will publish its analysis of risk levels, incidents or practices, and Anthropic cannot veto it.
There is a catch. Anthropic can still require redactions that fall inside those three protected categories, and Anthropic decides what counts as falling inside them. The company says METR remains free to state how much access it had and what information it received, so readers can judge how complete a given report is.
The scope is also tight. Only the small embedded team works under the agreement, which Anthropic says is enforceable. Other METR staff, including people working with Anthropic on separate projects, are walled off. Anthropic stresses that the setup is not a route for other organisations to obtain its information or its customers' data.
A three-tier model for outside evaluators
The METR deal sits at the top of a three-level framework Anthropic laid out in the same post.
- Tier 1 covers narrowly scoped risk investigations. Anthropic decides what access to give, and publication needs its approval.
- Tier 2 lets evaluators publish without approval and gives them more information. Anthropic can still demand removal of specific categories, but evaluators may say publicly whether they think a redaction was justified.
- Tier 3 is the METR arrangement: employee-like access with publication rights similar to Tier 2.
Anthropic says early-stage partners will get more limited information and publication rights. It treats architecture details as a high leak risk with usually low relevance to safety questions, so it will share them only when they are highly relevant and only with explicit internal approval.
Who else is in the queue
Anthropic named several other organisations at different stages:
- SecureBio is reviewing the chemical and biological weapons chapter of Anthropic's August 2026 Risk Report. SecureBio had already published a pilot review of the company's February 2026 report. Anthropic says it will consider giving SecureBio more access after the current review.
- Accenture and Faculty are working toward a broader partnership covering evaluations, red-teaming, alignment assessments, safeguard testing and capability building.
- Hugging Face is listed among several organisations in early discussions.
METR itself is not new to Anthropic. Over the past year it reviewed several of Anthropic's Risk Reports, according to the company.
Why this lands now
Anthropic ties the arrangement to two earlier commitments. In March it promised stronger independent, third-party assessment. It later signed the White House Accord on Super Intelligence, under which participating companies pledged to work with independent external auditors or evaluators.
The timing also matters for regulators. US scrutiny of the frontier labs has been rising, from the Federal Trade Commission's information demands to the administration's new incident-reporting expectations. We covered both in our reports on what the FTC subpoenas to OpenAI and Anthropic ask for and the first meeting of the Super Intelligence Force. An embedded evaluator with its own publication rights gives Anthropic a ready answer when officials ask how its safety claims are checked.
Anthropic is explicit that the programme is voluntary. "We're embedding evaluators voluntarily, and we believe every frontier AI company should do the same," the company wrote.
The open questions
Three issues will decide how much weight METR's eventual reports carry.
Who draws the redaction line. Anthropic alone decides what falls into the protected categories. If a future METR report is heavily redacted, readers will have to rely on METR's own statement about what it saw and what it was denied.
Independence over time. Embedded evaluators sitting in a company's offices, on its hardware, for long periods, face a familiar risk from financial auditing: closeness can blunt scepticism. Anthropic's post does not say how long the arrangement runs or how it can be ended.
Whether rivals follow. The post does not mention government bodies such as the UK AI Security Institute, and no other frontier developer has announced a matching deal. A single lab inviting inspectors in is a useful experiment. It only becomes a governance standard if others adopt it or regulators require it.
The first test will be METR's first published assessment under the new terms, and how much of it survives Anthropic's redaction rights.
