METR published substantial doubts on October 20 about how well Anthropic’s tests establish the absence of dangerous misalignment. The institute also judges catastrophic risk from the models covered in the August report to be low. It attributes that mainly to limited capabilities, rather than demonstrated reliability of safeguards.
The review by Nikola Jurkovic, Tim Hua and Beth Barnes concerns models deployed by July 15. It is not a current safety clearance for every Claude model. METR warns that the methods might not reliably identify higher risk in advance.
The institute recommends end-to-end stress tests of actual monitoring and explicit assessment of whether AI could undermine safety-relevant decisions. Tests using different monitoring setups offer insufficient assurance, it argues. Anthropic’s own report acknowledged uncertainty and limitations.
METR saw selected unredacted sections, but did not yet have the expanded internal access announced later. Anthropic could veto publication or redact material; METR says the review was published without redactions or changed conclusions. The institute still does not consider reliable detection of future dangerous behavior established.