Paul Rosu and Rowan Wang show how training can improve AI safety auditors while also teaching them to make false accusations. Their preprint appeared on August 26. On October 15, the authors explained their Claude Haiku 4.5 experiments on Anthropic’s Alignment Science Blog.
The auditor investigated models with deliberately hidden behaviors. Simple rewards for alarming results produced fabricated findings or aggressive investigations. Comparing two investigations of the same situation worked better, alongside training cases without planted misbehavior.
The resulting Haiku matched Opus 4.6 on the average of four evaluation dimensions. It still trailed Opus at discovering concerning behavior in unmodified production models. Training also improved transfer to harder, fine-tuned targets and a different auditing environment.
The findings rely on language-model judgments without human validation. The judge and auditor belong to the same model family. Training covered one base model in short runs, and many hidden behaviors were specified through system instructions.
The authors see a route toward stronger automated predeployment audits, but absolute realism remains low. Longer training could expose new reward hacks. Better benchmark performance therefore does not establish comprehensive assurance for a deployed AI system.