Education Columns

By Suzanne Minor, MD,  Sarah Stumbar, MD, MPH, and Marquita Samuels, MBA, Florida International University

Narrative comments provided by clinical preceptors are important for multiple reasons. Learners benefit more from narrative, specific feedback than from feedback consisting solely of numerical ratings; these more specific comments are easier to assimilate. Narrative comments are also important in justifying high or low numeric ratings, for documenting worrisome student behavior, and for use in the Medical Student Performance Evaluation (MSPE), or the dean’s letter. The quality and quantity of narrative comments can be improved through faculty development emphasizing their importance and essential components. However, faculty development—including in-person visits with faculty, emails, and phone conversations—is often limited by time constraints. A significant body of literature describes the essential components of high-quality narrative feedback: polarity, favorability, strengths or weaknesses, helpfulness to learner, actionability, specificity, trait- or behavior-focused, and entrustment levels.^1-5

Our pilot study strived to determine if simply adding a prompt to the family medicine clerkship student clinical assessment could improve the quality of narrative feedback comments written by preceptors. This intervention required minimal additional time or faculty development. 

In November 2017, we added the prompt: “Please comment on one patient encounter the student had that stands out to you (please try to be as descriptive as possible)” to the family medicine clerkship clinical assessment tool. This prompt was added only to our paper assessment, limiting our data to the six preceptors who regularly use the paper form. The Florida International University Institutional Review Board approved this activity.

We compared the specificity and genericity of all narrative comments, including responses to the new prompt from November 2017 through August 2018 to the prior preceptor comments using the original prompts received by the matched preceptors from April 2017 to November 2017. After individually rating each comment holistically on specificity and genericity, the three authors used an iterative process to come to a consensus. Table 1 details specific examples of specificity and genericity of comments. 

Table 1: Coding Categories and Examples

Specificity referred to the sufficiency of the comment to guide behavior, with “not specific” describing generic personality or character traits. “Semispecific” described a clinician task or described specific clinical situations that evoke clinical tasks. “Very specific” described specific detail.

Genericity referred to the level of detail. “All generic” comments were entirely generic; “partial generic” contained a mixture of generic and not generic comments, and “not generic” had no generic part of the comment. Nongeneric comments may echo the language from the family medicine clerkship assessment tool. 

A total of six faculty regularly complete paper, rather than electronic, assessments and were included in this pilot study. Table 2 compares the preintervention specificity and the postintervention specificity of each preceptor’s comments and Table 3 details the genericity of each preceptor’s comments. Three preceptors (50%) increased the percentage of their narrative comments that were classified as very specific. Two preceptors (33%) continued to have no very specific comments, even after the introduction of the new prompt. One preceptor (17%) had fewer very specific comments. Since we assessed comments in their entirety for specificity and genericity, it was not possible to calculate a specific percent change in either.

One preceptor (17%) increased from 0% to 60% nongeneric comments. Three preceptors (50%) increased the percentage of partially generic comments.

Table 2: Specificity of Each Preceptor’s Comments, Pre- and Postintervention

Abbreviations: NS, not specific; SS, semispecific; VS, very specific.

Table 3: Genericity of Each Preceptor’s Comments, Pre- and Postintervention

Abbreviations: AG, all generic, PG, partially generic^; NG, not generic

This pilot demonstrated that some preceptors improved the specificity and originality of their narrative comments with the addition of the new prompt. Prompting preceptors to provide specific, behavioral details in their narrative feedback comments served as a potentially high-yield, low-effort faculty development intervention.

This pilot was limited by the small number of preceptors who used only paper assessments. On the paper form, preceptors may have been limited by the sizes of the comments boxes. Also, while our assessment of comments was limited to specificity and genericity, we found that the usefulness of evaluating genericity was limited as many comments containing original pieces were labeled as generic or partially generic. Future faculty development can be targeted towards improving the specificity of narrative comments. Since this pilot, the curriculum committee added this prompt to the electronic assessment used for all clerkships. Future research will examine the impact of this comment box across clerkships.

Corresponding Author
Suzanne Minor, MD, Florida International University, seminor@fiu.edu

References

1. Lockyer JM, Sargeant J, Richards SH, Campbell JL, Rivera LA. Multisource feedback and narrative comments: polarity, specificity, actionability, and CanMEDS roles. J Contin Educ Health Prof. 2018;38(1):32-40. doi:10.1097/CEH.0000000000000183
2. Smither JW, Walker AG. Are the characteristics of narrative comments related to improvement in multirater feedback ratings over time? J Appl Psychol. 2004;89(3):575-581. doi:10.1037/0021-9010.89.3.575
3. Warm E, Kelleher M, Kinnear B, Sall D. Feedback on feedback as a faculty development tool. J Grad Med Educ. 2018;10(3):354-355. doi:10.4300/JGME-D-17-00876.1
4. van der Leeuw RM, Overeem K, Arah OA, Heineman MJ, Lombarts KM. Frequency and determinants of residents’ narrative feedback on the teaching performance of faculty: narratives in numbers. Acad Med. 2013;88(9):1324-1331. doi:10.1097/ACM.0b013e31829e3af4
5. Pelgrim EA, Kramer AW, Mokkink HG, Van der Vleuten CP. Quality of written narrative feedback and reflection in a modified mini-clinical evaluation exercise: an observational study. BMC Med Educ. 2012;12(1):97. doi:10.1186/1472-6920-12-97