Microsoft AI System Diagnoses Complex Cases at 83% Accuracy, Surpassing Physicians in Study
A new study in Science reveals that Microsoft's multi-agent AI diagnostic tool, MAI-DxO, significantly outperforms generalist physicians in solving intricate medical scenarios while pursuing a goal of medical superintelligence.
By Luana Sato ·
Microsoft's AI diagnostic system demonstrated an 83% success rate in solving complex medical cases during a simulation, a figure more than double the 36% accuracy achieved by generalist physicians working individually. The results, published this Thursday in the journal Science, mark a significant milestone for Mustafa Suleyman, CEO of Microsoft AI, who characterized the achievement as a crucial step toward developing medical superintelligence. The study highlights a growing disparity between the demand for medical expertise and the available supply, a challenge Microsoft's technology aims to address.
The system in question, referred to as MAI-DxO or simply DxO, functions by orchestrating a panel of multiple AI agents. These agents interact by questioning a patient, ordering necessary tests, and debating potential diagnoses before reaching a conclusion. While the study utilized complex cases typically reserved for specialists or physician teams to test the system's capabilities, Microsoft's technology is designed to coordinate these steps to assist in medical decision-making. Dominic King, Vice President of Health at Microsoft AI, noted, "There is very substantial and exciting here."
The timeline of this development involves a progression from standard AI models to specialized medical applications. The study's data indicates that while much of the performance gain stems from powerful underlying AI models, the DxO orchestration system provides additional accuracy on top of that foundation. Microsoft tested the system using models from various providers, including Google, Anthropic, Meta, and DeepSeek, but reported the most significant results when utilizing OpenAI's GPT-5 and GPT-5.5. Regarding cost, running GPT-5 on a basic prompt costs approximately $8,400 per complex case, whereas DxO reduced this cost by about $3,000 while increasing accuracy. However, on emergency department cases, there was little improvement in either accuracy or cost between GPT-5 on its own and with the assist from DxO.
Looking ahead, Microsoft plans to conduct real-world evaluations to test the clinical reasoning capabilities of DxO across different hospital settings. A partnership with Beth Israel Deaconess Medical Center in Boston will facilitate a study where patients receive standard treatment while consenting participants also engage with DxO for comparison. This initiative coincides with Microsoft's broader effort to build medical superintelligence, defined as the capacity to surpass contemporary human-led health systems in medical reasoning. The company is also working with the Mayo Clinic to train a foundation model for healthcare, with release planned for next year.
Suleyman, who has spent his career attempting to use AI to solve medical issues, previously co-founded DeepMind, which was acquired by Google in 2014. He joined Microsoft to lead its AI unit in 2024, and King followed shortly thereafter. Suleyman has stated that healthcare models will get "an order of magnitude better in capability within the next 24 to 36 months" and expects healthcare models to get an order of magnitude better in capability within the next 24 to 36 months. He believes that in 2027, they will be on the verge of a coding style breakthrough for medical records.
