Critical Synthesis Package: The Kalamazoo Consensus Statement Assessment Tools

Authors

Michelle Yoon
Veronica Michaelsen, George Washington University

Document Type

Instructional Material

Publication Date

5-5-2015

Description

This Critical Synthesis Package resource contains 1) a Critical Analysis of the psychometric properties and the application to health science education of the Kalamazoo Consensus Statement Assessment Tools and 2) a copy of each of the three instruments comprising the Kalamazoo Consensus Statement Assessment Tools developed by Elizabeth A. Rider, MSW, MD.

The Kalamazoo Consensus Statement (KCS) Tools are three content-valid, paper-based instruments that assess physician-patient communication skills. The Kalamazoo Essential Elements Communication Checklist (KEECC) is the original instrument. The Kalamazoo Essential Elements Communication Checklist-Adapted (KEECC-A) is a newer, adapted, construct-valid version, which was then further adapted into another multi-rater version, named the Gap-Kalamazoo Communication Skills Assessment Form (GKCSAF). The GKCSAF has high reliability and internal consistency, but modest interrater reliability. All three instruments use Likert ratings, and the GKCSAF also employs two forced-choice and two free-text items on a version adapted for use with multiple raters.

The KEECC and KEECC-A are completed by medical educators who rate learners on seven communication skill competencies: Build A Relationship, Open The Discussion, Gather Information, Understand The Patient's Perspective, Share Information, Reach Agreement, Provide Closure. The GKCSAF is completed by the learner (team if applicable), and one or more additional raters, such as standardized patients, or faculty instructors. Learners are rated on the same seven competencies as the KEECC and KEECC-A, and two additional communication dimensions: 1) Demonstrates Empathy and 2) Communicates Accurate Information.

These instruments were designed for use at all levels of medical education and can be used as formative and summative assessment tools or as a clerkship teaching tool to evaluate actual and simulated patient-physician communication encounters.

Data from the development and adaptation of KCS tools hint at promising psychometric characteristics. Cronbach alpha values indicate reliability. Interrater reliability is modest, and warrants more investigation to understand why, or to strengthen it. Content and construct validity were good, criterion-related validity was modest, and convergent validity is presently unable to be established. Continuing evaluation and research are necessary for establishing validity and reliability, and to advance research in teaching and assessing physician-patient communication skills. We should consider whether the Kalamazoo tool might capture interview efficiency, or be adapted to patient interviews involving multiple family members. Other future directions include using the tool with direct learner observation to strengthen links between communication training and learners' bedside performance. Research and development of the KCS tools has been contextually embedded in graduate medical education by mapping onto ACGME competencies. While these tools are relevant to all levels of learners, there is potential for expanded use in continuing medical education. The tools can be used as professional development for all clinicians and special training for those who receive sanctions, if malpractice links to poor communication. These tools also hold promise in specific situational training, such as difficult conversations, breaking bad news, obtaining informed consent, or scenarios of unanticipated events. Taken together, it encourages earlier incorporation of the KCS tools in training especially given trends towards integration during pre-clerkship years. In conclusion, the Kalamazoo tools are promising teaching and assessment tools for complex contexts of healthcare quality and safety initiatives, integrated clinical training, and increased emphasis on professionalism. The KCS tools may eventually be applicable or adaptable to other health professions. Existing data shows reliability and suggests certain types of validity. Faculty development and training for users may help with interrater differences. In sum, the KCS tools are worth considering for those interested in teaching or assessing communication skills.

Creative Commons License

This work is licensed under a Creative Commons Attribution-Noncommercial 4.0 License

DOI

10.15766/mep_2374-8265.10098

Recommended Citation

Yoon M, Michaelsen V. Critical synthesis package: the Kalamazoo consensus statement assessment tools. MedEdPORTAL Publications. 2015;11:10098. http://doi.org/10.15766/mep_2374-8265.10098