Microsoft's AI Panel Beats Doctors on Hard Cases

Microsoft’s AI system, MAI-DxO, correctly diagnosed complex medical cases in simulations more than twice as often as generalist physicians. The results, published Thursday in Science, mark a significant step for Mustafa Suleyman, the company’s AI CEO. He described the achievement as a move toward what he calls medical superintelligence. The system coordinates multiple AI agents to act like a panel of doctors, questioning patients and debating diagnoses.

The company has been refining these tools for years. Suleyman joined Microsoft in 2024 to lead its AI unit after co-founding DeepMind. His vice president, Dominic King, followed him shortly after. Both executives have spent careers trying to apply AI to healthcare. Their earlier work included launching DeepMind Health at Google. Now, at Microsoft, they are building a foundation model for healthcare in partnership with the Mayo Clinic, with a release planned for next year.

In the study, the AI system achieved an 83% accuracy rate on difficult cases from the New England Journal of Medicine. Generalists working individually with reference materials were correct only 36% of the time. Researchers noted that these complex cases are usually handled by specialists or teams, who were not part of this specific comparison. The data suggests that much of the improvement comes from stronger underlying models, with Microsoft’s orchestration adding further gains on top.

Microsoft tested the system with models from OpenAI, Google, Anthropic, Meta, and DeepSeek. The best results came from OpenAI’s GPT-5 and GPT-5.5. On complex cases, the system reduced costs by about $3,000 per case while raising accuracy by four points compared to using GPT-5 alone. However, on emergency department cases, the gains were minimal. The system helped less capable models significantly, but the gap between weak and strong models remained larger than the boost provided by the orchestration.

Suleyman believes the next phase involves integrating these tools directly into clinical workflows. He predicts a coding-style breakthrough for medical records by 2027. This would allow for streaming highly accurate predictions about how a medical record might unfold. He views this foresight as essential for clinicians. The company aims to make healthcare models an order of magnitude better within the next 24 to 36 months.

To test this in the real world, Microsoft is partnering with Beth Israel Deaconess Medical Center in Boston. The study will evaluate the system’s clinical reasoning across different hospital settings. Patients will receive standard care while those who consent will also interact with the AI. Researchers will then compare the AI’s recommendations against those of human doctors. King called this the biggest study of its kind, aiming to provide compelling evidence for wider deployment.

The release coincides with a manifesto defining medical superintelligence as surpassing the strongest human-led health systems. Microsoft hopes to move beyond simple benchmarks like answering exam questions. Instead, it wants to integrate with clinicians and healthcare systems for better outcomes. The company already sells dictation and radiology tools to doctors. It sees this new diagnostic system as a major expansion of those efforts, aiming to hold the number one spot in clinical decision support.