Microsoft’s MAI-DxO, called Diagnostic Orchestrator (DxO) in the paper, achieved 83.2% accuracy with GPT-5 on 304 New England Journal of Medicine (NEJM) clinicopathological cases and 80.3% on 325 MIMIC-IV-ED emergency-department-to-admission encounters. Published in Science on October 22, 2026, the results come from a controlled, text-based diagnosis benchmark, not a clinical evaluation of patient care.
What the benchmark measured
The study evaluated sequential diagnosis: a system starts with limited case details, asks questions, requests tests, and decides when to submit a diagnosis. Its Sequential Diagnosis Benchmark (SDBench) used 304 NEJM clinicopathological conference cases and 325 encounters from MIMIC-IV-ED.
| SDBench case set | DxO with GPT-5 | Physician comparison | Comparison conditions |
| NEJM clinicopathological cases, 304 cases | 83.2% accuracy | 36.1% average accuracy | Physicians used an open-book, text-only setup on a held-out set of 56 cases; DxO was evaluated on all 304 cases. |
| MIMIC-IV-ED, 325 encounters | 80.3% accuracy | 46.8% average accuracy | The paper reports a generalist-physician average for the benchmark comparison. |
The NEJM figures do not come from equally sized test sets: the physician average covers 56 held-out cases, while the AI result covers all 304. The scores describe performance under the study’s benchmark conditions, not a head-to-head trial of doctors and AI in routine care.
How SDBench presents a case
SDBench makes the task interactive rather than asking for a diagnosis from a fixed vignette. A Gatekeeper handles requests for information from the underlying case record. When the record lacks a requested finding, it can generate a diagnostically neutral synthetic result. A Judge then scores the submitted diagnosis using a physician-authored, five-level rubric; scores of 4 or 5 count as correct.
Microsoft physician scientist Xiaoxuan Liu walks through a case in a Microsoft demonstration published in 2025. The walkthrough follows questions, test requests, cost tracking, and changes to the working diagnosis.
What the cost estimates represent
The study estimated test costs by mapping requests to CPT codes and using Kaiser Permanente Washington’s 2023 chargemaster, with a fixed $300 physician-visit cost. For the NEJM cases, reported DxO configurations ranged from 79.6% accuracy at an estimated $1,303 per case to 84.5% at $8,945. The no-budget GPT-5 result was 83.2% at $5,431 per case.
| NEJM DxO configuration | Accuracy | Estimated cost per case |
| Budgeted configuration | 79.6% | $1,303 |
| Budgeted configuration | 80.9% | $2,241 |
| GPT-5, no budget constraints | 83.2% | $5,431 |
| Accuracy-oriented ensemble | 84.5% | $8,945 |
These are standardized benchmark estimates, not patient bills. They do not capture all real-world variation, such as differences in geography, test availability, invasiveness, discomfort, or time to results. For MIMIC-IV-ED, the no-budget GPT-5 configuration reached 80.3% accuracy at an estimated $4,979 per encounter; budgeted configurations reached 73.2% at $2,634 and 69.5% at $2,037.
Where the findings stop
The benchmark used text-based interactions rather than real patients or electronic health records. It excluded healthy and benign encounters, as well as multimodal inputs such as imaging, waveforms, and pathology slides. Its scoring assessed diagnostic correctness, not treatment safety, therapeutic appropriateness, or longitudinal patient management.
The physician comparison involved individual generalists, not specialist teams. The study’s authors say the results do not establish superiority over specialists or multidisciplinary teams, or performance in clinical practice. The paper also discloses that its authors were Microsoft employees and held Microsoft equity.
The study held out recent NEJM cases from development partly to examine possible memorization. That step does not establish that every benchmark case was absent from the models’ pretraining data.
Why earlier reports used different numbers
A 2025 version of the study focused on OpenAI’s o3 and reported an 85.5% maximum-accuracy configuration and a 20% physician average. The 2026 Science paper reports different models, configurations, and comparison conditions: its main GPT-5 result was 83.2%, while its accuracy-oriented ensemble reached 84.5%. These figures describe separate study versions and setups.
The announced evaluation with patients
On October 22, 2026, Microsoft announced a future prospective evaluation with Beth Israel Deaconess Medical Center. The planned work involves patient interaction and independent physician assessments.
