What changed
Analytics firm SemiAnalysis has released GDPx, a benchmark that tests AI agents on 206 tasks in realistic work environments and professional software over several hours. The scores are low, with no model scoring above 12 percent. Claude Opus 5.5 leads the field with just 12 percent, followed by Google's Gemini 4 Carbon (10.9 percent) and GPT-6 Astra (9.6 percent).
What this means in practice
The GDPx benchmark simulates knowledge work by having AI agents operate professional software like Siemens NX, search data rooms with thousands of files, or interact with live enterprise systems. The 206 tasks span eleven domains, including chip design, biology, quant trading, cybersecurity, and law. No model fully solves the finance and legal tasks, as a single wrong value in a spreadsheet makes the entire output useless in practice. SemiAnalysis counts the share of individual requirements met instead of requiring perfect task completion.
Limitations and availability
Critics question the benchmark's meaningfulness, citing distortions from safety filters, secrecy around test tasks, and unequal conditions created by each provider's own agent software. The tasks come from 14 specialized partners who also sell training and evaluation data to AI labs. Usually only one example task per partner is public, with the rest remaining behind closed doors. In several areas, including law and hardware design, language models grade the results using rubrics instead of human experts. Outsiders cannot fully verify the difficulty or grading quality. SemiAnalysis plans to update the benchmark every three to six months.
