Algorithmic Prediction of Delayed Radiology Turn-Around-Time during Non-Business Hours

Authors

Vaibhavi Shah, Radiology and Biomedical Imaging, University of California, San Francisco (UCSF), San Francisco, California 94143; Department of Biological Engineering, Department of Science, Technology, and Society , Massachusetts Institute of Technology (MIT), Cambridge, MA 02139.
Yeshwant R. Chillakuru, Radiology and Biomedical Imaging, University of California, San Francisco (UCSF), San Francisco, California 94143; The George Washington University, School of Medicine and Health Sciences, Washington, DC 20037.
Alex Rybkin, Radiology and Biomedical Imaging, University of California, San Francisco (UCSF), San Francisco, California 94143.
Youngho Seo, Radiology and Biomedical Imaging, University of California, San Francisco (UCSF), San Francisco, California 94143.
Thienkhai Vu, Radiology and Biomedical Imaging, University of California, San Francisco (UCSF), San Francisco, California 94143.
Jae Ho Sohn, Radiology and Biomedical Imaging, University of California, San Francisco (UCSF), San Francisco, California 94143. Electronic address: sohn87@gmail.com.

Document Type

Journal Article

Publication Date

5-1-2022

Journal

Academic radiology

Volume

29

Issue

5

DOI

10.1016/j.acra.2021.05.026

Keywords

Machine Learning; Natural Language Processing; Quality Improvement; Resident Education; Workflow Efficiency

Abstract

RATIONALE AND OBJECTIVES: Radiology turnaround time is an important quality measure that can impact hospital workflow and patient outcomes. We aimed to develop a machine learning model to predict delayed turnaround time during non-business hours and identify factors that contribute to this delay. MATERIALS AND METHODS: This retrospective study consisted of 15,117 CT cases from May 2018 to May 2019 during non-business hours at two hospital campuses after applying exclusion criteria. Of these 15,177 cases, 7,532 were inpatient cases and 7,585 were emergency cases. Order time, scan time, first communication by radiologist, free-text indications, and other clinical metadata were extracted. A combined XGBoost classifier and Random Forest natural language processing model was trained with 85% of the data and tested with 15% of the data. The model predicted two measures of delay: when the exam was ordered to first communication (total time) and when the scan was completed to first communication (interpretation time). The model was analyzed with the area under the curve (AUC) of receiver operating characteristic (ROC) and feature importance. Source code: https://bit.ly/2UrLiVJ RESULTS: The algorithm reached an AUC of 0.85, with a 95% confidence interval [0.83, 0.87], when predicting delays greater than 245 minutes for "total time" and 0.71, with a 95% confidence interval [0.68, 0.73], when predicting delays greater than 57 minutes for "interpretation time". At our institution, CT scan description (e.g. "CTA chest pulmonary embolism protocol"), time of day, and year in training were more predictive features compared to body part, inpatient status, and hospital campus for both interpretation and total time delay. CONCLUSION: This algorithm can be applied clinically when a physician is ordering the scan to reasonably predict delayed turnaround time. Such a model can be leveraged to identify factors associated with delays and emphasize areas for improvement to patient outcomes.

APA Citation

Shah, Vaibhavi; Chillakuru, Yeshwant R.; Rybkin, Alex; Seo, Youngho; Vu, Thienkhai; and Sohn, Jae Ho, "Algorithmic Prediction of Delayed Radiology Turn-Around-Time during Non-Business Hours" (2022). GW Authored Works. Paper 1008.
https://hsrc.himmelfarb.gwu.edu/gwhpubs/1008

Department

School of Medicine and Health Sciences Student Works