A new experiment using unpublished NeurIPS papers challenges claims by Anthropic and OpenAI that AI agents can conduct autonomous AI research. The study, led by Princeton and the UK AI Security Institute, found that while frontier models can handle research engineering, they fail at critical research tasks. Two AI-generated papers were rejected by original authors as conference reviewers would, with one receiving a 'Strong Reject' due to poor data, experiments, and lack of new contributions. The researchers argue that existing evaluations either test narrow tasks or rely on peer review, which they describe as 'overstretched, stochastic, and suffers from poor review quality'.
The study introduced 'Shadow Evaluation,' where agents received core research questions from unpublished papers and were evaluated by the original authors. The team partnered with authors of two NeurIPS 2026 submissions, one examining personality steering in language models and the other developing a method called TabPFN. The agents used Claude Opus 4.8 with Extra-High Reasoning, given six days, $3,000 in API credits, a GPU budget, and full access to a virtual machine and the open web. The agents ran inside a scaffold, a software environment that orchestrated model calls and provided tools, allowing the model to delegate to subagents and monitor its own resource usage. The researchers used OpenClaw, an open-source, vendor-neutral agent framework built by Austrian developer Peter Steinberger.
The original authors reviewed the finished papers as conference reviewers and rejected both. One got a 'Strong Reject,' criticizing poorly motivated data, unreadable prose, and no new contributions. One reviewer called the reasoning a 'proof by example' fallacy that was 'highly non-scientific.' Another called the experiment choices 'bizarre' and the results clearly the product of 'post hoc choices.' Analysis of agent logs revealed systematic weaknesses, including poor judgment on publishable research, failure at creative problem-solving, and inability to backtrack effectively. Both agents gave up their most ambitious research goals within the first ten hours. The engineering work was completed without human help, with only three interventions needed: a scaffold bug fix, a deadline extension, and a request to rewrite for readability. The researchers found no significant reward hacking, as agents did not manipulate results or skew data to chase better scores.
Source: thedecoder
