Validation of a Novel Statistical Method to Identify Aberrant Patient Logging: A Multi-Institutional Study

Elana A. Min, Elana A. Min, PhD, PA-C, is director of clinical education and an assistant professor in the Physician Assistant Program at the Northwestern University Feinberg School of Medicine in Chicago, Illinois.
Desiree Lie, Elana A. Min, PhD, PA-C, is director of clinical education and an assistant professor in the Physician Assistant Program at the Northwestern University Feinberg School of Medicine in Chicago, Illinois.
Carey Barry, Elana A. Min, PhD, PA-C, is director of clinical education and an assistant professor in the Physician Assistant Program at the Northwestern University Feinberg School of Medicine in Chicago, Illinois.
Amanda Moloney-Johns, Elana A. Min, PhD, PA-C, is director of clinical education and an assistant professor in the Physician Assistant Program at the Northwestern University Feinberg School of Medicine in Chicago, Illinois.
Susan T. Hibbard, Elana A. Min, PhD, PA-C, is director of clinical education and an assistant professor in the Physician Assistant Program at the Northwestern University Feinberg School of Medicine in Chicago, Illinois.
Tamara S. Ritsema, Elana A. Min, PhD, PA-C, is director of clinical education and an assistant professor in the Physician Assistant Program at the Northwestern University Feinberg School of Medicine in Chicago, Illinois.
Mitzi D'Aquila, Elana A. Min, PhD, PA-C, is director of clinical education and an assistant professor in the Physician Assistant Program at the Northwestern University Feinberg School of Medicine in Chicago, Illinois.
Trenton Honda, Elana A. Min, PhD, PA-C, is director of clinical education and an assistant professor in the Physician Assistant Program at the Northwestern University Feinberg School of Medicine in Chicago, Illinois.

Document Type

Journal Article

Publication Date

9-1-2022

Journal

The journal of physician assistant education : the official journal of the Physician Assistant Education Association

Volume

33

Issue

3

DOI

10.1097/JPA.0000000000000437

Abstract

INTRODUCTION: Student patient encounter logging informs the quality of supervised clinical practice experiences (SCPEs). Yet, it is unknown whether logs accurately reflect patient encounters, and the faculty resources necessary to review for potential aberrant logging are significant. The purpose of this study was to identify a statistical method to identify aberrant logging. METHODS: A multi-institutional (n = 6) study examined a statistical method for identifying potentially aberrant logging behavior. An automated statistical Mahalanobis Distance (MD) measurement was used to categorize student logs as aberrant if they were identified as probable multivariate outliers. This approach was validated using a gold standard for aberrant logging behavior with manual review by 4 experienced faculty ("faculty consensus") and then comparing interrater agreement between faculty and MD-based categorization. In secondary analyses, we compared the relative accuracy of MD-based categorization to individual faculty categorizing data from their own program ("own program" categorization). RESULTS: 323 student logging records from 6 physician assistant (PA) programs were included. Compared to "faculty consensus" (the gold standard), MD-based categorization was highly sensitive (0.846, 95% CI: 0.650, 1.000) and specific (0.766, 95% CI: 0.645, 0.887). Additionally, there was no significant difference in sensitivity, specificity, positive predictive value, or negative predictive value between MD-based categorization and "own program" categorization. DISCUSSION: The MD-based method of identifying aberrant and nonaberrant student logging compared favorably to the more traditional, faculty-intensive approach of reviewing individual student logging records. This supports MD-based screening as a less labor-intensive alternative to individual faculty review to identify aberrant logging. Identification of aberrant logging may facilitate early intervention with students to improve clinical exposure logging during their SCPEs.

Department

Physician Assistant Studies

Link to Full Text

Share

COinS