Artificial intelligence as a simultaneous second reader in diagnostic radiology: An umbrella review of systematic reviews and meta-analyses.
Mahmoud Omar, Mazzin Elsamaloty, Dottie Yu et al.
✦ AI-curated · Sources linked
In 30 seconds
This umbrella review assessed systematic reviews and meta-analyses of AI used as a simultaneous second reader in diagnostic radiology, comparing AI interpretations with clinician or radiologist interpretations. Ten meta-analyses covering 147 primary studies and more than 722,000 case and 3.6 million control images were included. Diagnostic odds ratios ranged from 30.67 (95% CI; 10.06-102.87) for fracture detection on X-ray to 273.60 (95% CI; 130.51-573.58) for pulmonary nodule detection on CT. Most reviews provided highly suggestive evidence, but substantial heterogeneity and low review quality constrained confidence.
Key findings
- Across applications, diagnostic odds ratios ranged from 30.67 (95% CI; 10.06-102.87) for X-ray fracture detection to 273.60 (95% CI; 130.51-573.58) for CT pulmonary nodule detection.
- Nine of ten meta-analyses provided Class II evidence, with substantial heterogeneity (I² = 89.9%-99.9%).
- AMSTAR-2 quality was critically low in nine reviews and low in one.
Why it matters
The review suggests that AI can show strong agreement with clinician interpretations across several radiologic tasks, supporting interest in its potential as a concurrent reader. However, agreement with clinicians is not equivalent to accuracy against a definitive reference such as histopathology. Marked heterogeneity and weak review quality mean pooled performance estimates should be interpreted cautiously.
What to keep in mind
The evidence was limited by substantial heterogeneity, variation in imaging modalities and model development and validation, and low or critically low quality of the included reviews. Clinician interpretation, rather than a definitive ground truth, served as the reference standard.