Inter-rater reliability of ultrasound quality assurance review of FAST examinations: A cross-sectional study

Elizabeth Dearing, George Washington University, Department of Emergency Medicine, Washington, DC, USA. Electronic address: edearing@mfa.gwu.edu.
Sarah E. Frasure, George Washington University, Department of Emergency Medicine, Washington, DC, USA. Electronic address: sfrasure@mfa.gwu.edu.
Kaitlen Howell, Highland Hospital - Alameda Health System, Department of Emergency Medicine, Oakland, CA, USA. Electronic address: kshowell@fulbrightmail.org.
Megan M. Leo, Boston Medical Center, Department of Emergency Medicine, Boston, MA, USA; Boston University Chobanian & Avedisian School of Medicine, Department of Emergency Medicine, Boston, MA, USA. Electronic address: Megan.Leo@bmc.org.
Matthew Lipton, Vanderbilt University Medical Center, Department of Emergency Medicine, Nashville, TN, USA. Electronic address: matthew.lipton@vumc.org.
Michael Gottlieb, Rush University Medical Center, Department of Emergency Medicine, Chicago, IL, USA. Electronic address: MichaelGottliebMD@gmail.com.
Martin Demant, The University of Arizona, Department of Emergency Medicine, Tuscon, AZ, USA. Electronic address: demant@aemrc.arizona.edu.
Bret P. Nelson, Icahn School of Medicine at Mount Sinai, Department of Emergency Medicine, New York City, NY, USA. Electronic address: bret.nelson@mountsinai.org.
Yiju T. Liu, Harbor-UCLA Medical Center, Department of Emergency Medicine, Torrance, CA, USA. Electronic address: Yliu4@dhs.lacounty.gov.
Keith Boniface, George Washington University, Department of Emergency Medicine, Washington, DC, USA. Electronic address: kboniface@mfa.gwu.edu.

Document Type

Journal Article

Publication Date

7-23-2025

Journal

The American journal of emergency medicine

Volume

97

DOI

10.1016/j.ajem.2025.07.053

Keywords

Competency; Credentialing; FAST exam; Quality assurance; Ultrasound

Abstract

BACKGROUND: Point-of-care ultrasound (POCUS), an integral part of patient care in the Emergency Department, requires a robust Quality Assurance (QA) program for credentialing and providing ongoing feedback to providers. However, there are no set guidelines for QA and the agreement between providers is unknown. OBJECTIVE: This study sought to evaluate the inter-rater reliability of QA for POCUS. METHODS: In this cross-sectional study, eight emergency ultrasound faculty members from eight different institutions completed QA on a total of 25 Focused Assessment with Sonography in Trauma (FAST) exams. The exams were graded on overall quality of image acquisition as well as accuracy of interpretation (true positive, true negative, false positive, false negative, incomplete, and technically limited study). The studies were assessed as to whether they met minimal criteria for credentialing. The overall agreement in meeting this minimal requirement as well as the quality scores alone were each compared between participants using Fleiss' kappa. RESULTS: Inter-rater reliability for whether studies met the minimum criteria for credentialing was slight (Fleiss' κ = 0.058; 95 % CI: -0.016 to 0.132). Image quality grading (scores of 1 and 2 versus 3, 4 and 5) also showed slight agreement (κ = 0.199; 95 % CI: 0.125 to 0.275). There was also no correlation between years in clinical practice and QA review. CONCLUSIONS: This pilot study found there is only slight agreement in QA grading between emergency ultrasound faculty for both overall competency assessment and in image acquisition alone. Developing a tool to standardize QA may improve the inter-rater reliability and create more uniformity for POCUS feedback and credentialing.

Department

Emergency Medicine

Link to Full Text

Share

COinS