A researcher at the University of Pennsylvania used OpenAI's GPT-5.6 Sol Pro to solve a long-standing open problem in statistics, disproving a 30-year-old conjecture about the reliability of the Benjamini-Hochberg method for controlling false positives. The breakthrough challenges the assumption that the widely used method reliably works with correlated data, a key issue in fields like genomics. According to Edgar Dobriban, an associate professor at the University of Pennsylvania's Wharton School, the AI model completed the task in roughly 90 minutes, while its predecessor, GPT-5.5, failed to produce a valid solution even after more than 20 hours of computation. Dobriban used the AI to construct a statistical model where the actual false discovery rate provably exceeds the target level, with simulations confirming the result. He also published the accompanying code. The finding matters more in theory than in practice for now, as the gap above the target level is relatively small (0.104 vs 0.1), so the result mainly matters for theory at this point. Practical effects still need further study, and the finding doesn't mean the BH procedure is generally unusable. The result is still significant for statisticians because AI solved the problem quickly after humans had failed. Dobriban says GPT-5.6 Sol Pro took about 90 minutes. GPT-5.5 couldn't find a solution even after roughly 20 hours of work with several agents. "So the capability improvement is quite real. Exciting times to live in!" he writes. The full chat and prompt are available here.
Source: thedecoder
