AI Panel Beats Doctors on Complex Cases
Microsoft's new diagnostic system outperformed human physicians in a study, but experts note it still requires significant cost and accuracy improvements for real-world use.
By Elijah Fitzgerald ·
The race to automate medical diagnosis is intensifying as major tech firms release tools that rival human expertise. On Thursday, a new study published in the journal Science revealed that Microsoft’s latest AI system, known as MAI-DxO, correctly diagnosed complex medical scenarios significantly more often than generalist doctors.
The system works by coordinating multiple AI agents that function like a panel of specialists. These agents question patients, order tests, and debate potential diagnoses. In simulations using difficult cases from the New England Journal of Medicine, the AI achieved an 83% accuracy rate. By comparison, primary care doctors and general internists working individually were correct only 36% of the time. It is important to note that these cases are typically handled by specialists or teams, who were not included in the study's comparison group.
Mustafa Suleyman, CEO of Microsoft AI, described the results as a step toward "medical superintelligence." He noted that the company is currently partnering with the Mayo Clinic to train a specialized healthcare foundation model, which is planned for release next year. Suleyman believes this will allow the company to make significant leaps in predicting how medical records unfold, a capability he calls "gold dust" for clinicians.
However, the study indicates that much of the performance boost comes from using powerful underlying models rather than just the orchestration system. Microsoft tested the system with models from various companies, including OpenAI, Google, and Anthropic. The best results were achieved using OpenAI’s GPT-5 and GPT-5.5. While the AI improved accuracy on complex cases, the gains were minimal for emergency department scenarios. Additionally, the system increased the estimated cost per case compared to using a single model with a basic prompt.
The technology performs best when assisting less capable models. For example, a weaker model called Gemini 2.5 Flash saw its accuracy rise from 46.4% to nearly 69% when paired with the new system. Despite these gains, the gap between strong and weak models remained larger than the boost provided by the orchestration. Microsoft researchers explained that accuracy improvements naturally shrink as models approach their theoretical maximum, but the system helps optimize both cost and precision.
To move beyond simulations, Microsoft is partnering with Beth Israel Deaconess Medical Center in Boston to evaluate the system in real hospital settings. This study will compare the AI’s recommendations against those of human doctors in emergency and urgent care environments. Dominic King, vice president of Health at Microsoft AI, stated that this will be the largest study of its kind, providing compelling evidence for future large-scale deployment.
This release follows similar moves by Google and OpenAI, which have recently claimed their AI can match or exceed doctor performance. Microsoft aims to integrate these tools directly into clinical workflows, moving beyond simple task benchmarking. Suleyman emphasized that the company is committed to accelerating the introduction of this technology, viewing it as a key differentiator for Microsoft in the competitive landscape of frontier AI labs.
