What changed
Microsoft’s AI diagnostic system, MAI-DxO, correctly diagnosed complex medical cases in a simulation more than twice as often as generalist physicians, according to a study published in Science. The system, which coordinates several AI agents to act like a panel of doctors, achieved an 83% accuracy rate in complex cases, compared to 36% for primary care doctors and general internists. The study used complex cases from the New England Journal of Medicine, and the results suggest that the AI’s performance is significantly better than human doctors in these scenarios.
What this means in practice
The AI system, MAI-DxO, is designed to assist in medical diagnosis by questioning patients, ordering tests, and debating diagnoses. Microsoft tested the system with models from several companies, including OpenAI, Google, Anthropic, Meta, and DeepSeek, but its best results came with OpenAI’s GPT-5 and GPT-5.5. On complex cases, DxO increased GPT-5.5’s accuracy score from 72% to nearly 87% at a slightly cheaper cost. However, the gains did not show up on emergency department cases, where both models scored about 85% at a similar cost. The system is intended to help with complex cases and less capable AI models, such as Gemini 2.5 Flash, which saw a significant improvement from 46.4% to nearly 69% with DxO.
Limitations and availability
The study’s data suggests that much of the improvement comes from more powerful underlying AI models, with Microsoft’s orchestration system adding gains on top. Microsoft is working with the Mayo Clinic to train a foundation model for healthcare, with release planned for next year. The company also has a new partnership with Beth Israel Deaconess Medical Center in Boston to evaluate DxO’s clinical reasoning capabilities across different hospital settings. The study will compare DxO’s recommendations against human doctors’ in real-world settings, including triage, specialist consultations, and treatment decisions. Microsoft’s researchers noted that accuracy gains naturally shrink as models approach the theoretical maximum, and the DxO orchestration helps with the broader challenge of optimizing both accuracy and cost.
