AI Outperforms Doctors in Emergency Department Diagnoses at Triage, Study Finds

Overview

The Conversation reports on a Science study in which an artificial intelligence system weighed in at different points during patient care in the emergency department, identifying the correct diagnosis or something closely related at triage in 67% of cases. In the same scenarios, two doctors achieved 50% and 55%. The article emphasizes that the AI worked strictly from written text and did not interact with patients or assume responsibility for outcomes, framing the results as a potential second-opinion tool rather than a replacement for clinical judgment.

Key insights

• 67% AI triage accuracy versus 50% and 55% for doctors.
• AI operated only on written notes, not direct patient evaluation.
• Potential benefits include broader diagnostic thinking, but risks include unnecessary testing and overconfidence.
• Calls for careful testing, governance, and NHS-specific considerations before routine adoption.

Overview

The Conversation discusses a Science study that assesses whether an artificial intelligence system can assist with diagnosis in emergency departments by analyzing real patient records. The AI, trained on written notes from a Boston hospital, weighed in at multiple stages of care and achieved 67% accuracy in identifying the correct diagnosis or something closely related at the earliest stage, triage. By comparison, two doctors achieved 50% and 55% accuracy in the same scenarios. The article frames the result as a meaningful step toward AI supporting clinicians, particularly when information is scarce and uncertainty is high, rather than as a replacement for human judgment.

Study design and scope

The AI operated entirely on written text and did not see the patient, hear the patient, examine them, speak to family, or bear any responsibility for subsequent care. It produced a written opinion based on selected information rather than performing emergency medicine. The study uses genuine clinical text from an active emergency department, which the authors argue makes the findings more directly relevant to real-world practice than prior work that relied on exams or synthetic data. The authors also caution about limitations, such as the risk that listing many possible diagnoses could prompt unnecessary tests or erode clinician trust if used improperly. They also acknowledge that some benchmark cases may have been publicly available data that the AI could have encountered during training, which tempers enthusiasm about headline performance.

Context is provided by noting that large language models have demonstrated capabilities on medical licensing exams, but passing an exam is not the same as delivering high-quality ward care. The authors suggest that, with further refinement and governance, these systems could help doctors think through a wider array of possible diagnoses, especially in high-uncertainty situations where missing a serious condition carries substantial risk.

Findings and practical implications

The results show a tangible gap between the AI and human clinicians at the earliest stage of patient care. The authors view AI as a tool that could assist clinicians by expanding diagnostic considerations, potentially enabling care that is faster and safer when properly tested and governed. However, the article emphasizes several caveats: real clinical deployment requires rigorous testing within health systems like the NHS, robust safety and accountability mechanisms, and clarity about how AI recommendations should be integrated into decision-making processes. It also highlights that around 16% of UK doctors report using AI tools daily, with another 15% using them weekly, underscoring that clinicians are already experimenting with these tools even as formal governance and validation frameworks lag behind.

Quote from Ewen Harrison, University of Edinburgh: "The AI was working entirely from written text. It never saw the patient, never examined them, and was not practicing emergency medicine. It was offering a written opinion based on selected information."

These findings are not presented as a directive for practice, but as an indication that AI can offer meaningful second-opinion support. The article notes the potential for longer diagnostic lists to broaden thinking but warns of downsides, including over-testing, unnecessary procedures, or misplaced confidence in plausible but incorrect answers. It also flags concerns about data availability and training data, which can affect performance and generalizability.

Policy, governance, and the road ahead

The hard question is how to test and govern AI tools in real clinical settings such as the NHS, not merely whether AI can assist with diagnosis. The Royal College of Physicians snapshot cited in the piece reveals that a sizable minority of UK doctors are already using AI in daily practice, highlighting the urgency of establishing assessment, training, harm detection, and responsibility frameworks. The article argues for a cautious, measured approach that positions AI as a supportive, second-opinion tool rather than a replacement for clinical judgment and emphasizes patient care outcomes—care that is better, safer, and faster—as the ultimate measure of value.