Physician-aided AI improves detection of acute respiratory distress syndrome
Researchers find that using artificial intelligence in a way that collaborates with physicians, rather than replaces them, could result in higher diagnostic accuracy while also potentially reducing physician workload.
Contact:
Kate Murphy
Marketing Communications Specialist, Weil Institute
mukately@med.umich.edu
---
ANN ARBOR – Acute respiratory distress syndrome (ARDS) is a deadly critical illness that has a high mortality. However, recognition and diagnosis of ARDS is often missed or delayed, and patients do not receive evidenced-based care when ARDS goes unrecognized.
To help physicians identify ARDS faster and more reliably, researchers at the University of Michigan’s Max Harry Weil Institute for Critical Care Research and Innovation and Michigan Medicine developed a deep learning algorithm trained to detect ARDS findings in chest X-rays. Now, in a new study published in npj Digital Medicine, the team examined the unique strengths and weaknesses of this model compared to those of human expert physicians. They also explored how a model and physicians could potentially work together to improve ARDS diagnosis, ultimately improving outcomes for patients.
“Thanks to recent advances in artificial intelligence (AI), we have deep learning systems that can diagnose health conditions based on clinical images with expert-level accuracy,” said Dr. Negar Farzaneh, a Weil Institute Research Investigator and Data Scientist, as well as lead author on the study. “But we’re also seeing a gap between studies describing the capabilities of these systems and efforts to investigate how or when to integrate them in a manner that supports physicians and improves diagnosis. That gap is something we wanted to address in our study.”
Using a reference standard of 414 chest X-rays from adult hospital patients with acute hypoxic respiratory failure, the team deployed the AI model alongside a group of physicians who had expertise in chest x-ray interpretation for ARDS detection. To determine the strengths and weaknesses of both groups, the team measured three factors: overall performance in ARDS detection, accuracy based on difficulty of X-ray interpretation, and level of AI/physician certainty in their interpretations.
Compared to the physicians, the AI model demonstrated higher overall performance in detecting whether ARDS findings were present. But while the model had a stronger showing at first, the team hypothesized that X-ray difficulty may play a key role.
To explore this concept further, the team divided the X-rays based on how challenging they were to classify--with “difficult” defined as those in which there was disagreement among the physicians regarding the interpretations. The researchers found that the AI model outperformed the physicians in interpreting chest X-rays that were not as difficult to review. However, the physicians were better at reviewing the minority of chest X-rays that were more difficult to review. Both physicians and the model also rated their confidences in the chest X-ray interpretation, and the team found that when one was less confident the other performed better.
“It's interesting to see how the AI model and physicians can complement each other's strengths. In situations where physicians lacked confidence in interpreting a chest X-ray, the AI model provided more accurate results, and vice versa,” said Dr. Farzaneh.
"It's interesting to see how the AI model and physicians can complement each other's strengths. In situations where physicians lacked confidence in interpreting a chest X-ray, the AI model provided more accurate results, and vice versa."
Negar Farzaneh, PhD
Research Investigator, Data Scientist,
Weil Institute for Critical Care Research and Innovation
The results of the team’s analysis into the strengths and weaknesses of AI and physician expertise suggested that the two could potentially complement each other and that both types of expertise might decrease rates of ARDS miss-diagnosis. Based on these findings, the team tested several potential strategies where an AI and physician could work together to achieve the best performance. One method that worked well was if the AI system reviewed the chest x-ray first and then deferred to physicians if it was uncertain. In this type of an approach, a physician would only need to review a smaller subset of chest X-rays, thus off-loading the human expert workload and allowing physicians focus on the more challenging cases.
According to study senior author Dr. Michael Sjoding, Associate Director of the Weil Institute and Associate Professor of Pulmonary and Critical Care Medicine, this type of approach could ultimately revolutionize care delivery to ARDS patients in the intensive care unit (ICU).
“Understanding how to effectively operationalize AI systems in the ICU is really important,” said Sjoding. “These systems are becoming more common, but there has not been a lot of work done so far to understand how to bring them to the bedside to help clinicians provide the best care. This work opens the door to a future where AI systems and human experts work together to provide excellent ARDS care to all patients.”
While the team’s analysis demonstrated the power of an AI-physician collaboration, further investigation will be needed to determine how best to integrate such strategies into real-world clinical applications.
“Because medical decisions are often high stakes, we know that patients and clinicians likely won’t accept completely replacing human expertise with AI algorithms,” said Dr. Farzaneh. “However, strategies where the model complements a physician’s diagnosis, rather than replaces it, might be a more reasonable alternative. Our work suggests that these collaborations, when optimized, can result in higher diagnostic accuracy and enable patients to receive more consistent care.”
Paper Cited
“Collaborative Strategies for Deploying Artificial Intelligence to Complement Physician Diagnoses: An Application to Acute Respiratory Distress Syndrome” npj Digital Medicine. DOI: 10.1038/s41746-023-00797-9
Study Authors
Negar Farzaneh, PhD (Weil Institute, Emergency Medicine); Sardar Ansari, PhD (Weil Institute, Emergency Medicine); Elizabeth Lee, MD (Radiology); Kevin Ward, MD (Weil Institute, Emergency Medicine, Biomedical Engineering); Michael W. Sjoding, MD (Weil Institute, Pulmonary and Critical Care Medicine, Internal Medicine), all of Michigan Medicine.
Disclosures
The University of Michigan has filed a US Utility Patent application for the invention “Computer vision technologies for rapid disease detection,” of which Dr. Sjoding and Dr. Ward report being co-inventors and which has been licensed to AirStrip Technologies Inc.
Dr. Farzaneh, Dr. Ansari, Dr. Ward, and Dr. Sjoding are included in an invention disclosure with the University of Michigan’s Office of Innovation Partnerships.
About the Max Harry Weil Institute for Critical Care Research and Innovation
The team at the Max Harry Weil Institute for Critical Care Research and Innovation is dedicated to pushing the leading edge of research to develop new technologies and novel therapies for the most critically ill and injured patients. Through a unique formula of innovation, integration and entrepreneurship that was first imagined by Weil, their multi-disciplinary teams of health providers, basic scientists, engineers, data scientists, commercialization coaches, donors and industry partners are taking a boundless approach to re-imagining every aspect of critical care medicine. For more information, visit www.weilinstitute.med.umich.edu.