Using analytics to change the CBD game
Residents training within specialties that have implemented Competence by Design are assessed frequently. The increased amount of assessment data that we are collecting within these competency-based programs can be analyzed to provide data-informed insights on the performance of the people, programs and systems that make up Canadian specialty medical education. How might training change if we could track the impact of our educational interventions on patient care? Would we be able to support our trainees better if we could spot concerning patterns earlier? Could we identify gender- or race-based bias in our assessments and, with this knowledge, begin to correct inequalities?
The major question looming over this data is
how can we use it in a way that is ethical and fair?
Taking a page from Hollywood: Using analytics to change the game The 2011 movie Moneyball portrayed the true story of how baseball was revolutionized with the introduction of sports analytics. Formulas brought data to life to identify clear characteristics of high performers. Subsequent work investigated how training could be improved and injuries could be prevented.
The movie also depicted struggles with the leads working through skepticism as they compiled and interpreted the data, and gained acceptance of a new practice in an industry steeped in tradition.
At the January 19 Royal College Research Forum, Brent Thoma, MD, FRCPC, used the Moneyball analogy to provide medical education leaders with an exciting display of data he has collected from his CBD residents after two years’ worth of trial and error.
A Royal College clinician educator and associate professor of Emergency Medicine at the University of Saskatchewan, Dr. Thoma is openly sharing his CBD data dashboards, which provide valuable insights into the training characteristics of residents and their teachers.
“The systematic collection of CBD data holds immense potential to improve medical education and, ultimately, patient care.”
Dr. Brent Thoma, Emergency Medicine, University of Saskatchewan presenting at the Royal College Research Forum on January 19, 2021. View the presentation recording.
Working together to evolve resident learning
Drs. Susan Moffatt-Bruce, Jason Frank and Tanya Horsley collaborated on the research panel to inspire a national conversation on how we can work together to use sophisticated data visualization and analytical techniques to transform competency-based assessment data to support the next generation of resident learning.
Dr. Moffatt-Bruce underscored the importance of moving forward, in the spirit of inclusivity, by engaging learners, evaluators and patients to ensure that the data we use is “safe, secure, equitable and enabling.”
Dr. Frank agreed there are “great possibilities as we go forward” in leveraging the ground-breaking work out of Saskatchewan. He explained that it is “an exciting time to be involved in medical education, and yet another example of how by learning from each other within the MedEd community, we can continue to advance residency education.”
Medicine is changing at a rapid pace and CBD is a key initiative to ensure our residency education keeps up. Dr. Thoma’s research is an excellent example of the value of CBD data. While there are still many challenges to overcome, thanks to this emerging research it is now possible to imagine new data helping advance the resident learning experience.
