Can a weaker AI model help supervise a stronger one? Anthropic researchers explored that question in a bounded experiment using parallel Claude agents. Their report compares the agents’ result with an initial effort by human researchers on the same research problem.
What the 97% means
The team measured the performance gap between the weak supervisor and a strong model trained on correct answers. Two human researchers recovered 23% of that gap after seven days adapting prior methods. Starting from that work, nine parallel agents reached 97% after five additional days and 800 cumulative agent-hours. This is not a claim that 97% of AI safety has been solved.
The reported compute and API cost was about $18,000. People chose the problem and evaluation, and the agents built on the human starting point. The figures therefore are not a universal comparison of AI and human research ability.
Why the limits matter
The test used specific open-weight models and datasets. The researchers caution that transferring the methods to larger models and other tasks is a separate challenge. The result illustrates a way to automate part of an oversight research workflow; it does not prove future superhuman systems can be safely supervised.
