AI Outperforms Doctors in ER Diagnosis Study

A recent study conducted by Harvard Medical School and Beth Israel Deaconess Medical Center, published in Science this week, indicates that an artificial intelligence (AI) model demonstrated higher diagnostic accuracy than human physicians in emergency room (ER) settings. Researchers compared diagnoses from OpenAI's o1 and 4o models against those provided by two internal medicine attending physicians for 76 ER patients.

The study found that the o1 AI model performed nominally better than or on par with the two attending physicians and the 4o model across various diagnostic stages. This difference was particularly pronounced during initial ER triage, where patient information is limited and rapid, accurate decisions are critical. The AI models were presented with unprocessed electronic medical record data, mirroring the information available to human doctors at the time of diagnosis.

Specifically, the o1 model achieved an exact or very close diagnosis in 67% of triage cases. In contrast, one attending physician reached this level of accuracy in 55% of cases, while the other achieved it in 50% of cases. This outcome suggests a potential for AI to enhance diagnostic precision, especially in high-pressure, information-scarce medical environments.