March Family Medicine Listings

For the Office-based Teacher of Family Medicine

March 2001, Vol 33, No 3

Paul M. Paulman, MD
Feature Editor

Editor’s Note: In this month’s column, John Langlois, MD, and Sarah Thach, MPH, of the Mountain Area Health Education Center in Asheville, NC, present practical information on learner evaluation in the office. I welcome your comments about this feature, which is also published on the STFM Web site at www.stfm.org. I also encourage all predoctoral directors to make copies of this feature and distribute it to their preceptors (with the appropriate Family Medicine citation). Send your submissions to Paul Paulman, MD, University of Nebraska Medical Center, Department of Family Medicine, 983075 Nebraska Medical Center, Omaha, NE 68198-3075. 402-559-6818. Fax: 402-559-6501. ppaulman@unmc.edu. Submissions should be no longer than 3–4 double-spaced pages. References can be used but are not required. Count each table or figure as one page of text.

Evaluation Using the GRADE Strategy
John P. Langlois, MD; Sarah Thach, MPH
(Fam Med 2001;33(3):158-60.)

Many preceptors find evaluation an unpleasant component of their community-based teaching. Evaluation is, however, a critical function of teaching. Evaluation helps assure that future clinicians possess appropriate knowledge, attitudes, and skills. Effective evaluation also helps a learner assess his or her strengths and weaknesses, identify strategies for improvement, and continue professional growth and development.

The key to avoiding evaluation pitfalls (Table 1) is preparing for the evaluation throughout the rotation. When the evaluation process is fully integrated into the rotation, a student’s learning experience is enhanced, and a difficult task for the preceptor is made easier and more effective.

Table 1 Potential Evaluation Pitfalls

• The halo effect	When certain characteristics, either positive or negative, cause preceptors to overlook other important aspects of learner performance, eg, an enthusiastic, caring learner with mediocre skills receiving high marks while a shy student with superior knowledge receives a lower grade
• “Oops:” insufficient evidence	Describing a student’s shortcomings without providing specific incidents and ways in which the student could have done better. At the end of the rotation it can be hard to remember the details of such incidents without a system for recording observations
• “But you never TOLD me that!”	Stating at the end of the rotation that the learner has fallen short of expectations when those expectations were not clearly stated during the rotation
• “But I NEED honors!”	Finding out on the final day of the rotation about the student’s expectations and perceived needs for a particular grade or evaluation on the rotation.
• “Uh-oh, should they pass?”	Realizing at the final evaluation that, despite significant efforts on the preceptor’s part, the learner’s performance has remained substandard throughout the rotation and that he or she should not pass. It is crucial to contact the school early in the rotation to get help.
• The Lake Wobegon effect	Rating all students “above average,” which does not help the school or the student accurately assess the student’s strengths and weaknesses. The learner, future patients, and the profession may suffer.

Evaluation: Making it Work
Many people think evaluation is the brief meeting and grade at the end of a learning experience, but these are only a small part of the picture. An ongoing evaluation process—including setting clear expectations on the first day, continual observation and assessment of the learner’s performance, and behavior-specific feedback to the learner—forms the foundation for the information shared in your final evaluation session. We offer our GRADE strategy as an outline for this process (Table 2).

G—Get Ready
One of the most difficult and important parts of the evaluation process is getting ready. Before meeting the learner, review the course objectives and the evaluation form to orient yourself to a new course or help focus your teaching for a familiar course. Ask the school for the objectives and evaluation form in advance. Less ideally, you can ask the learner for a copy and review it before you sit down together for orientation the first day. If you have any concern or confusion regarding the materials or your role in evaluation, call the course director for clarification.

Decide what you expect from the learner. For example, when does the learner need to be in the office or hospital? What style of case presentation do you prefer? How much reading is expected?

R—Review Expectations With the Learner
Review the school’s and your expectations with the learner at your initial orientation. Describe how these expectations can be met in your setting or indicate which goals may be challenging (eg, male physical exam in an obstetrician’s office). The more specific you can be and the earlier you tell the learner, the more likely he or she will be able to implement your suggestions and meet your requirements.

Review the learner’s expectations. What does the learner hope to get from the experience? Are there specific procedures he or she expects to learn? Is he or she hoping to achieve a specific grade? Not all learners are up front with (or have even thought about) their own expectations, but this kind of questioning promotes an adult learning style. Knowing the learner’s expectations from the start can help you prevent conflicts that can come from unrealistic or unmet goals.

Determine the training level of the learner. Learners at the same stage of training may have varying skills based on the rotations they have completed so far. Your expectations and your criteria for evaluation will need to reflect the learners’ clinical experience.

Describe how you plan to evaluate the learner. How and when will you give ongoing feedback? How can he or she give you feedback? When will the evaluation sessions occur? Review the actual evaluation form and the criteria you will use.

A—Assess
Assessment should occur throughout the rotation. Observing learner-patient histories, physical exams, presentations, and reviewing their notes provides the basis for your evaluation. Compare the learners’ performances. A method for briefly recording observations, such as jotting short phrases or a patient name on a note card in your pocket, can help jog your memory. Use these notes to provide behavior-specific feedback at the end of the day and save the notes for reference at the final evaluation session.

Regular feedback is a key component of the evaluation process. Timely, specific feedback on weaknesses gives the learner an opportunity to improve and gives you a chance to observe the learner’s efforts and successes. Similarly, positive comments on a good performance should not wait. Few things reinforce good behavior and growth like timely, specific feedback.

As a practicing professional, much of the feedback and evaluation of your day-to-day performance must come from self-assessment. Encouraging learners to assess themselves (“How do you think your encounter with that patient went?”) before giving your own feedback promotes this important behavior and gives you valuable data on how learners view their own performance.

D—Discuss Evaluation at the Midpoint
A midpoint evaluation provides an excellent opportunity to reinforce the learner’s strengths, point out weaknesses, identify problems, clarify expectations, set new goals, and suggest strategies for improvement. A relatively small amount of time and effort during the middle of the rotation can dramatically improve the ultimate satisfaction of the learner and the preceptor.

It is important to schedule a specific time for this meeting and to set aside a minimum of 30 minutes to ensure adequate time for discussion. Both the preceptor and the learner should prepare for the mid-rotation evaluation. Make two extra copies of the evaluation form and have each of you complete it separately to avoid influencing each other’s assessment. Ask the learner to think about whether the rotation has met his or her needs so far and possible improvements.

At the meeting, compare forms and review the learner’s performance to date. Significant discrepancies between the learner’s assessment and yours should be discussed in detail and expectations revisited. Highlight areas of good performance and areas for improvement and give specific recommendations for improvement.Discuss areas where either the preceptor’s or the learner’s expectations are not being met and agree on a plan for improvement.

E—End with a Grade
The last step of this process is the final evaluation session. It is important to schedule sufficient time for a formal, private meeting. Consider setting aside an hour. A late start and an interruption or two can whittle a half hour down to nothing, and options for rescheduling are limited at the end of the rotation.

Complete the evaluation in advance so that you have time to carefully reflect on the learner’s knowledge, skills, attitudes, improvement during the rotation, and room for further growth. It is difficult to do this effectively when the learner is looking over your shoulder. Support your evaluation with specific examples (this is where your notes can be helpful). Make sure your evaluation is future oriented; although this particular learning experience is ending, the learner’s education and professional career will continue.

Include comments and specific examples of both positive attributes and areas for improvement on the written evaluation. Your comments help give the school a clear picture of the learner’s performance and are often incorporated into dean’s letters. Make sure the comments and other aspects of the form reflect the learner’s overall performance. Comments on an honors-level performance should include superlatives describing the learner and show where expectations were significantly exceeded.

Assigning a grade for the rotation can be particularly challenging. Some schools describe the level of performance expected for a particular grade. If you have had several learners for this rotation in the past, you have a useful standard for comparison. You may wish to discuss the learner’s performance with partners or colleagues who have had experience with this type of learner. You may also want to contact the course director at the school for his or her perspective and guidance, particularly if this is a new teaching experience for you. It is most important that you be able to justify your final decision based on the actual performance of the learner.

Complete any necessary paperwork as promptly as possible, while the experience is still fresh in your mind. Completing forms in advance of your final evaluation meeting gets most of the work out of the way before the learner leaves. You can also reserve the last 10 minutes of your final evaluation meeting to wrap up any paperwork after the learner has left the room.

Acknowledgment: The content of the article is based on materials developed as a part of the Preceptor Development Program, a comprehensive program of preceptor development materials supported by HRSA grant #1D15PE50119-01. Detailed information on this project can be obtained from our Web site at www.mtn.ncahec.org.

Corresponding Author: Address correspondence to Dr Langlois, MAHEC Division of Family Medicine and MAHEC Office of Regional Primary Care Education, 118 W.T. Weaver Boulevard, Asheville, NC 28804. 828-258-0670. Fax: 828-257-4739. johnl@mtn.ncahec.org.

Back to March Family Medicine Listings
Back to Office-based Teacher Table of Contents