SHEE Journal No. 4 2023 (Full Version)
SHEE Journal
In this issue, the team has focused on the theme: "Workplace-Based Assessment". Workplace-based assessment (WBA) is an essential evaluation tool, particularly for clinical health science students who must develop the skills to care for patients in real-life situations. While WBA has been used in Thailand for many years across various curricula, there are still challenges related to misunderstandings that create unnecessary burdens for instructors and hinder the effectiveness of student learning. A deeper understanding of these tools is believed to benefit training programs, instructors, students, and ultimately, the patients that students will care for in the future.
This issue of SHEE Journal presents key insights into WBA, including:
- The significance of workplace-based assessment.
- Choosing appropriate assessment tools.
- The relationship between WBAs and Entrustable Professional Activities (EPAs).
- Detailed explanations of various assessment tools.
- The use of technology to support WBA implementation.
The content includes contributions from faculty members and physicians at the Siriraj Health science Excellence center, Siriraj Medical School, and other external experts. This comprehensive approach aims to provide valuable guidance for health science educators in designing and implementing effective WBA systems, enhancing both teaching and learning outcomes.
Author: SHEE
Downloads: 724
Issue4/2023-01 Executive talk
An assessment tool that has been increasingly used in health sciences education over the past 10 years is workplace-based assessment. Although there have been various training sessions on this topic, and these tools have been implemented both before and after graduation for some time, many educators and academic professionals still express concerns about using these tools and have misconceptions about them. In this issue of the SHEE Journal, the SHEE Center team presents a journal with the theme “Workplace-based Assessment,” with the hope that the content provided will help clarify the origin, necessity, and importance of using this method. Additionally, we aim to present examples of tools commonly used for assessing clinical-year students, which should help educators and those involved in workplace-based assessments feel more confident and comfortable in applying this method to evaluate learners.
Author: Assoc. Prof. Dr. Cherdsak Iramaneerat, M.D., Ph.D.
Downloads: 97
Issue4/2023-02 Why Should Workplace-Based Assessment Be Used?
The assessment of learning outcomes in medical education can be categorized into different levels. If we use Miller's framework as a criterion for classification, we find that the highest or most valuable level of assessment is the evaluation of students' ability to perform tasks independently (does) in a real workplace, or workplace-based assessment. However, assessing at this level is complex and has many limitations, making it a significant challenge for both assessment designers and evaluators. They must work to minimize factors that may affect the accuracy, reliability, and validity of the assessment outcomes. This article will discuss the definitions, benefits, limitations, and criteria for assessment, clarifying the assessment process. By understanding these aspects, educators will be able to better apply this knowledge to assess students in their own institutions, based on their specific context.
Author: Asst. Prof. Usapan Surabenjawong
Downloads: 89
Issue4/2023-03 From Workplace-Based Assessment (WPBA) to Entrustable Professional Activity (EPA)
Workplace-based assessment is considered one of the key tools for promoting learning through formative assessment. However, another important role of workplace-based assessment is to determine whether students have achieved the competencies outlined in the curriculum (summative assessment). In this context, Entrustable Professional Activities (EPAs) are a key concept that helps facilitate effective summative assessment. Understanding the relationship between workplace-based assessment and EPAs allows students who are ready to take on progressively challenging tasks according to their individual potential. However, if the principles are misunderstood, it may lead to overtrusting students who are not ready, which could result in patient safety issues. This article aims to present the concept of EPAs and the approach for utilizing data from workplace-based assessment to ensure accurate evaluation of EPAs.
Author: Thitiphan Srikulmontree, M.D.
Downloads: 352
Issue4/2023-04 How to Choose Among Various Workplace-Based Assessment Tools
A key factor that significantly impacts the quality of Workplace-Based Assessment (WPBA) is the selection of appropriate assessment tools that align with what is being evaluated. The author has outlined the essential characteristics and the required assessment skills for each WPBA tool, summarized in the table, to serve as a guide in the decision-making process for selecting the most suitable WPBA tools to evaluate the educational skills in health sciences students more effectively.
Author: Assoc. Prof. Thos Harnroongroj, M.D.
Downloads: 108
Issue4/2023-05 DOPS: Direct Observation of Procedural Skills
Clinical skills are a crucial learning outcome in health sciences curricula. Every health sciences school must pay attention to teaching and assessing students' clinical skills to ensure that all graduates can perform procedures according to professional standards, ensure patient safety, and lead to positive healthcare outcomes. In the traditional era, the logbook system was primarily used to track whether students had practiced and developed their clinical skills. Most curricula set a requirement for the number of procedures each student must perform before graduation, with some also assigning points for completing the required number of procedures. However, the logbook system has considerable limitations. It can only assess the quantity, i.e., how many times the student has performed a procedure, but it does not provide detailed information about the quality of each procedure performed. This does not align with the direction of competency-based curricula, where instructors must ensure that graduates possess the necessary skills and competencies according to professional standards. Therefore, current assessment methods for clinical skills must include measures that reflect the qualitative details of the procedures performed.
Author: Assoc. Prof. Dr. Cherdsak Iramaneerat, M.D., Ph.D.
Downloads: 173
Issue4/2023-06 Enhancing Clinical Competency Assessment: The Mini-Clinical Evaluation Exercise (mini-CEX)
In medical education, clinical skills assessment is a crucial tool to ensure that students are prepared for real-world medical practice. The Mini-Clinical Evaluation Exercise (mini-CEX) is a method of assessing the knowledge and clinical abilities of trainees in various dimensions. It focuses on evaluating clinical skills in real-world situations and provides timely feedback. Mini-CEX is a form of workplace-based assessment (WPBA) and can assess students at the "Does" level according to Miller's pyramid of learning, which represents the highest level. This article will discuss information about mini-CEX and how to use it to assess students and provide effective feedback.
Author: Lt. Col. (M.D.) Dollawat Sangpanit
Downloads: 927
Issue4/2023-07 Cased-based Discussion
Workplace-based assessment (WPBA) not only evaluates procedural skills or communication skills, but also allows for the assessment of clinical reasoning skills through a tool known as Case-based Discussion (CbD). In this article, the author presents the definition of CbD, strategies for effectively applying CbD, and the important roles of both the learner and the educator in facilitating learning through CbD.
Author: Thitiphan Srikulmontree, M.D.
Downloads: 254
Issue4/2023-08 5 Questions with 360-Degree Evaluation
360-Degree Evaluation, or multisource feedback (MSF), is an assessment of a student's behavior during their work period. The purpose is to provide students with feedback on their work-related behaviors to help develop their potential further. The process consists of four steps:
- Collecting behavioral data from each individual in the workplace through questionnaires.
- Gathering data anonymously while maintaining confidentiality.
- The student receives a report on the assessed information.
- The advisor receives the data, offering praise or collaborating on plans for areas that need development.
Author: Assist.Prof. Piengbulan Yapan, M.D.
Downloads: 296
Issue4/2023-09 Managing Workplace-Based Assessment with Technology
Currently, technology is being increasingly used to manage Workplace-Based Assessment (WPBA) in order to achieve optimal effectiveness, such as providing feedback to learners, tracking their development, and enabling systematic management. A key piece of technology that aids in WPBA is the smartphone. The advantage of smartphones today is that both assessors and those being assessed have them as personal devices, carry them everywhere, and have sufficient internet connectivity at health science schools. Smartphones are easy to use and familiar to everyone as personal devices, allowing users to access them at any time. This makes entering WPBA data in real-time feasible, providing learners with timely feedback. Additionally, if the data in the WPBA system is automatically integrated into a portfolio system, it further enhances learning and simplifies curriculum management.
Author: Assoc. Prof. Kasana Raksamani, M.D.
Downloads: 74
Issue4/2023-10 Students’ voice
Once again, we present the article "Students’ Voice," which in this issue focuses on workplace-based assessment (WPBA). In this article, we aim to share the understanding and perspectives of students regarding this form of assessment. To provide a comprehensive viewpoint that can be applied to students at various levels, the author has gathered data from representatives across multiple faculties, including both pre-graduates and post-graduates.
This article explores three key areas related to WPBA:
- The significance of assessment through WPBA
- Challenges in implementing WPBA
- Approaches to enhancing learning opportunities from WPBA assessments
Author: Pimmada Samukchan, M.D.
Downloads: 163
Issue4/2023-11 เชิด-ชู
In this issue of the SHEE Journal, the team is honored to have the privilege of receiving insights from Assoc. Prof. Tripop Lertbunnaphong, a distinguished medical educator who was awarded the "Chao Phraya Phra Sadej Surenthara Thibodi" prize for Outstanding Clinical Medical Teacher in the academic year 2020. In the article titled 'Honoring Excellence,' we are pleased to share his reflections and inspiration on being a good medical educator with all of you.
Author: SHEE
Downloads: 65
Issue4/2023-12 สับ สรรพ ศัพท์
Validity is one of the essential qualities of assessment. It refers to the accuracy of a test and its ability to produce meaningful results that can be applied. In other words, a highly valid assessment is one that effectively measures what it is intended to measure, according to the objectives.
Validity threats refer to any factors that interfere and prevent the assessment from measuring what it is supposed to. There are two main types of validity threats:
-
Construct under-representation: This occurs when the assessment does not adequately cover what it is supposed to measure, resulting in an inaccurate representation of the test taker's abilities. For example, if an exam covers only two out of five course objectives, the evaluator cannot conclude that a student who fails the exam lacks knowledge of the entire subject; they may only lack knowledge in the two objectives included in the test. Therefore, the results cannot be used effectively.
-
Construct-irrelevant variance: This refers to external factors that are unrelated to the abilities of the individual being assessed but still affect the assessment results. Examples of these external factors include the test itself, the examiner, the grading process, or the method of scoring.
Author: Kanin Dardtanim, M.D
Downloads: 129
Issue4/2023-13 Education movement: Implementing Workplace-based Assessment in Thailand
From the content presented in this article, it is clear that Workplace-Based Assessment (WPBA) is an evaluation method that should be promoted and supported more extensively in health sciences schools. If implemented effectively, WPBA can enhance learning and help students develop medical skills that align with the principles of Situated Cognition and Ecological Psychology. Additionally, it can reduce the pressure on students that typically arises from high-stakes standardized testing. In this article, the author proposes guidelines for health sciences schools to adopt effective strategies for implementing WPBA in the context of health sciences education in Thailand.
Author: Assoc. Prof. Dr. Cherdsak Iramaneerat, M.D., Ph.D.
Downloads: 83
Issue4/2023-14 SHEE Sharing: A randomized controlled trial of feedback to improve patient satisfaction and consultation skills in medical students
Performance assessment in the workplace can be complex for certain competencies, as they cannot be evaluated directly in a single instance or by just one evaluator. Multi-source feedback is an important tool that allows for a comprehensive assessment of competencies and promotes effective learning and development of students' skills (assessment for learning).
Author: Passawut Sirithongthawon, M.D.
Downloads: 139
Issue4/2023-15 SHEE research - Why is validity in research instruments important, and how can it be ensured?
Have you ever ordered food but received something different? I once ordered fried rice with pork and a fried egg, but I got fried rice with curry paste and catfish instead. In everyday language, this is called receiving something that doesn’t match what was ordered. Some people also order items from online stores, and when the delivery arrives, the product doesn't match the advertisement. This is also called receiving an item that isn’t as advertised. If it’s food, you might not want it and have to make it again. With online products, you might have to contact the seller or file a complaint with the Consumer Protection Board. Receiving something that doesn’t meet your expectations can have both minor and major negative impacts. In research, there’s a similar issue related to validity, and it can cause significant problems. Today, I’d like to invite all readers to understand the importance of validity and how to check for it.
Author: Dr. Kiattiyot Kuldetchaichan
Downloads: 142
Issue4/2023-16 Click & Go with technology: Creating a PDF Portfolio with Adobe Acrobat Professional
Workplace-based assessment (WPBA) is an evaluation method conducted in the workplace and can take various forms, such as clinical assessments in real-world settings and practice (mini-clinical evaluation exercise; Mini-CEX) or direct observation of procedural skills (DOPs), where assessments are usually conducted multiple times. Therefore, it is necessary to collect evaluation results for each learner in one central location for easy access. In this regard, the author proposes the use of a PDF Portfolio, a feature in Adobe Acrobat Professional. You can create a new document in the form of a PDF Portfolio and combine various learner assessment results, both in PDF format and non-PDF formats (such as audio files, videos, images, and other formats) into one location, without merging them into a single file. This enables instructors to open the different documents quickly and conveniently. Additionally, non-PDF files in the PDF Portfolio can be edited using the program that opens that document, and once edited, the changes will be saved within the PDF Portfolio as well. Once you have created the PDF Portfolio, you can share a link to the file stored in Adobe Cloud Storage with learners or other instructors to view it, which can be accessed through the Adobe Acrobat Reader application on a mobile phone or tablet. Now that we've covered this, let's begin creating a PDF Portfolio!
Author: Assist. Prof. Dr. Vorawan Vanicharoenchai
Downloads: 106
Issue4/2023-17 Podcast
The Siriraj Health science Education Excellence center (SHEE) has created the SHEE Podcast, which compiles interesting topics in health sciences and presents them through casual discussions to make the content more accessible. The podcast is released on the 2nd and 4th Friday of each month at 8:00 PM. To date, there have been six series published:
Series 1: Disruptive Education—If you don't move, you won't keep up. Series 2: Successful Teaching Strategies—Tips (not so secret) to Enhance Teaching Success. Series 3: Effective Assessment Methods—Educational assessments to develop learners effectively. Series 4: Enhancing Transferable Skills—Elevating versatile skills to develop expertise beyond knowledge. Series 5: Creating Active Learning Strategies—Foster active learning and ignite curiosity in learners. Series 6: Mastering Performance Assessment—Practical assessments to measure learner abilities effectively.
From September to November 2023, the following episodes were released:
Author: Passawut Sirithongthawon, M.D., Pimmada Samukchan, M.D.
Downloads: 66
