How role playing can give examiners better insights into the CanMEDS Roles
How and why Dr. Michelle Chiu champions simulation-based education
It’s not easy to assess the CanMEDS Roles — such as collaboration and health advocacy competencies — in a 13-minute Royal College exam question.
“We would try, but it was very difficult to get that sense,” says anesthesiologist Michelle Chiu, MD, FRCPC, a Royal College Fellow for the past 19 years and examiner since 2013. Her experience in simulation and medical education made her believe that role play was the answer so that’s what she pitched to the Royal College Examination Nucleus Board for Anesthesiology.
“I said, ‘If we really want to assess these aptitudes, we need to directly interact with the candidates,’” says Dr. Chiu, a physician at The Ottawa Hospital and associate professor at the University of Ottawa.
Dr. Michelle Chiu
Proving role playing is a sound model of assessment
“We decided to introduce role-playing in which the examiner would play the role of a patient, a family member, a colleague or another health care professional with whom the candidate would have to interact,” she explains.
After role-playing was launched in the exam process, research was needed to determine if the model was feasible and acceptable for both the candidate and the examiner. The findings were encouraging: an investigation led by Dr. Chiu and presented at the Royal College’s International Conference on Residency Education (ICRE) in September 2019 showed there was no loss in validity between the role play examiner scoring and an observing examiner scoring.
The addition of role-playing was a good fit with Dr. Chiu’s passion for simulation and medical education, ignited by her fellowship at Monash University in Melbourne, Australia in 2002-2003. In 2014, she became simulation director for the Department of Anesthesiology and Pain Medicine at the University of Ottawa, where she oversees simulation training for undergrads, residents, fellows and faculty.
“It’s something that I really, really enjoy,” she says. “In my role, we really work to advance the integration of simulation into education, both nationally and internationally.”
Launching the Canadian National Anesthesiology Simulation Curriculum
As a member of the Royal College Anesthesiology Specialty Committee, Dr. Chiu proposed a project to develop a national curriculum of simulation scenarios for assessing graduating anesthesiologists’ competency in managing critical events. The committee gave the plan a green light and in 2013 she started a committee to develop CanNASC — the Canadian National Anesthesiology Simulation Curriculum.
“We gathered a group of program directors and simulation educators from across the country representing all 17 Royal College accredited Anesthesiology programs and we did a needs assessment. We also came to Delphi-based consensus to narrow things down to scenarios that we felt were critical to competency as a practising anesthesiologist.”
The committee developed assessment rubrics, implementation guidelines and a rater calibration guide.
“We rolled out the following year, in 2014, with one scenario and we have subsequently rolled out all of the scenarios. As we went along, we’ve actually gone through and revised all the scenarios to make sure that they are as standardized and as authentic as possible in terms of being able to assess the things that we want to assess.”
Dr. Chiu is quick to point out that CanNASC is not just about medical management and medical knowledge. “It actually focuses a lot on how the anesthesiologist works in the environment to manage the crisis, including leadership, team work, situational awareness, resource allocation — all of those skills.”
So far, over 1,600 CanNASC assessments have been completed across the 17 Canadian residency programs in Anesthesiology.
“I really believe that simulation is an amazing technique for teaching and practising skills that we really can’t teach or practise any other way. And I think that there’s a lot that can be done that way,” says Dr. Chiu.
“So that’s why I do what I do. I think that that’s the best way that I can have an impact on patient care.”
Related: Teaching the teachers proves a learning experience for Dr. Michelle Chiu
