STFM Podcast Bonus Conference Episode

2025 Conference on Medical Student Education Scott Fields Lecture

As a special episode of the STFM Podcast, listen to an audio recording of the 2025 STFM Conference on Medical Student Education Scott Fields Lecture: "In Pursuit of Fairness: Overcoming Bias in Assessment" by Karen Hauer, PhD, University of California, San Francisco.

SUBSCRIBE ON APPLE SUBSCRIBE ON SPOTIFY

Saturday, February 1, 2025

Bias in assessment of medical learners presents a critical, ongoing challenge to the quality of medical education. Experiences of bias may manifest in access to learning opportunities as well as in quantitative ratings and qualitative comments describing performance. This bias interferes with learners’ developmental progress through training and has consequences for their future careers and the patients they may serve. Solutions to address bias are needed for individual faculty and leaders designing and implementing education systems. This session reviews the literature on the causes and consequences of bias in assessment of learner performance in medical education. Dr Karen Hauer discusses recommendations to avoid bias in assessment drawn from the Josiah Macy Jr. Foundation Conference on Ensuring Fairness in Medical Education Assessment: Conference Recommendations Report. The speaker shares resources for implementing recommendations and using them in faculty development.

Learning Objectives

Upon completion of this session, participants should be able to:

Identify causes and consequences of bias in assessment of clinical learners.
Apply recommendations to avoid bias in assessment.
Describe the design and implementation of an equitable assessment system.

Listen to All Episodes

Karen Hauer, PhD

Karen Hauer, PhD, is vice dean for Education and Professor of Medicine at the University of California, San Francisco (UCSF). As vice dean, she is responsible for post-baccalaureate premedical, undergraduate, graduate, and continuing medical education across the multiple UCSF clinical training sites. In her prior position as associate dean for Competency Assessment and Professional Standards, she designed and led the program of assessment in the UCSF School of Medicine Bridges curriculum and developed and directed the School’s medical student coaching program. For this work, she led the team which received the ASPIRE international award for excellence in student assessment. She is an active researcher in medical education and a research mentor for fellows, residents, students, and faculty with a focus on competency-based medical education, learner assessment, equity in assessment, coaching, and remediation. She completed a PhD in Medical Education through a joint program with UCSF and the University of Utrecht in the Netherlands. She received the 2024 Hubbard Award from the NBME for excellence in medical education assessment. She has served on leadership committees with the National Board of Medical Examiners and Macy Foundation, served as deputy editor for the journal Medical Education, and is past president of the Clerkship Directors in Internal Medicine national organization.

11400 Tomahawk Creek Parkway

Leawood, KS 66211

(800) 274-7928

stfmoffice@stfm.org

Tips for Using STFM's AI Assistant

STFM's AI Assistant is designed to help you find information and answers about Family Medicine education. While it's a powerful tool, getting the best results depends on how you phrase your questions. Here's how to make the most of your interactions:

1. Avoid Ambiguous Language

Be Clear and Specific: Use precise terms and avoid vague words like "it" or "that" without clear references.

Example:

Instead of: "Can you help me with that?"

Try: "Can you help me update our Family Medicine clerkship curriculum?"

Why this is important: Ambiguous language can confuse the AI, leading to irrelevant or unclear responses. Clear references help the chatbot understand exactly what you're asking.

2. Use Specific Terms

Identify the Subject Clearly: Clearly state the subject or area you need information about.

Example:

Instead of: "What resources does STFM provide?"

Try: "I'm a new program coordinator for a Family Medicine clerkship. What STFM resources are available to help me design or update clerkship curricula?"

Why this is better: Providing details about your role ("program coordinator") and your goal ("design or update clerkship curricula") gives the chatbot enough context to offer more targeted information.

3. Don't Assume the AI Knows Everything

Provide Necessary Details:The STFM AI Assistant has been trained on STFM's business and resources. The AI can only use the information you provide or that it has been trained on.

Example:

Instead of: "How can I improve my program?"

Try: "As a program coordinator for a Family Medicine clerkship, what resources does STFM provide to help me improve student engagement and learning outcomes?"

Why this is important: Including relevant details helps the AI understand your specific situation, leading to more accurate and useful responses.

4. Reset if You Change Topics

Clear Chat History When Switching Topics:

If you move to a completely new topic and the chatbot doesn't recognize the change, click the Clear Chat History button and restate your question.

Note: Clearing your chat history removes all previous context from the chatbot's memory.

Why this is important: Resetting ensures the AI does not carry over irrelevant information, which could lead to confusion or inaccurate answers.

5. Provide Enough Context

Include Background Information: The more context you provide, the better the chatbot can understand and respond to your question.

Example:

Instead of: "What are the best practices?"

Try: "In the context of Family Medicine education, what are the best practices for integrating clinical simulations into the curriculum?"

Why this is important: Specific goals, constraints, or preferences allow the AI to tailor its responses to your unique needs.

6. Ask One Question at a Time

Break Down Complex Queries: If you have multiple questions, ask them separately.

Example:

Instead of: "What are the requirements for faculty development, how do I register for conferences, and what grants are available?"

Try: Start with "What are the faculty development requirements for Family Medicine educators?" Then follow up with your other questions after receiving the response.

Why this is important: This approach ensures each question gets full attention and a complete answer.

Examples of Good vs. Bad Prompts

Bad Prompt

"What type of membership is best for me?"

Why it's bad: The AI Chat Assistant has no information about your background or needs.

Good Prompt

"I'm the chair of the Department of Family Medicine at a major university, and I plan to retire next year. I'd like to stay involved with Family Medicine education. What type of membership is best for me?"

Why it's good: The AI Chat Assistant knows your role, your future plans, and your interest in staying involved, enabling it to provide more relevant advice.

Double Check Important Information

While the AI Chat Assistant is a helpful tool, it can still produce inaccurate or incomplete responses. Always verify critical information with reliable sources or colleagues before taking action.

Technical Limitations

The Chat Assistant:

Cannot access external websites or open links
Cannot process or view images
Cannot make changes to STFM systems or process transactions
Cannot access real-time information (like your STFM Member Profile information)

Membership

Community

About

Advocacy

Governance

Key Initiatives

Journals

Other Publications

CAFM Educational Research Alliance (CERA)

Research Resources

Annual Spring Conference

Conference on Medical Student Education

Conference on Practice & Quality Improvement

General Information

Certificate Programs

Webinars

Online Courses

Fellowships

Other Faculty Training

STFM Curriculum

More Curriculum Resources

Resources Fundamental to Family Medicine Education

Topic-Specific Aggregated Resources

Preceptor Recruitment and Retention Resources

Resources For

Awards

Scholarships

STFM Project Fund

2025 Conference on Medical Student Education Scott Fields Lecture

Saturday, February 1, 2025

Learning Objectives

Listen to All Episodes

Karen Hauer, PhD

Contact Us

Events

Deadlines

Useful Links

AI Chatbot Tips

Tips for Using STFM's AI Assistant

1. Avoid Ambiguous Language

Example:

2. Use Specific Terms

Example:

3. Don't Assume the AI Knows Everything

Example:

4. Reset if You Change Topics

5. Provide Enough Context

Example:

6. Ask One Question at a Time

Example:

Examples of Good vs. Bad Prompts

Bad Prompt

Good Prompt

Double Check Important Information

Technical Limitations

STFM AI Assistant