Competence by Design Innovators

Our hard-to-encounter and hard-to-achieve EPAs were readily identifiable through regular program-level progress reviews, Competence Committee meetings, and mapping EPAs to rotations. Certain EPAs are only encountered on one or two rotations all year, so we flagged these for both faculty and learners to ensure they focused on these EPAs during the respective rotations. For example, an Obstetrics and Gynecology resident will likely only get a chance to perform a trauma primary survey on General Surgery. This was flagged and emphasized to ensure optimal attempts during that rotation. On the flip side, "identifying critically ill patients" was flagged as being particularly hard to achieve, so specific faculty development exercises were created to address this.

We developed these initiatives to address two problems that were immediately identified when CBD launched in our program: faculty didn’t understand the EPA entrustment scale ratings and they needed to better understand the descriptions. For example, our EPA regarding "identifying a critically ill patient" was often assessed on a resident’s ability to fully manage these patients, which is well beyond the scope of the EPA and training level. We created video modules specific to each EPA, which encompassed multiple areas of early faculty development. The modules were presented at grand rounds, where participants could actively engage using their mobile phones to compare their assessment to the established criterion standard. The video modules were also made available online and distributed by email. This quick exercise was very well received. To enhance uptake, it was delivered in a variety of forums with different audiences.

There was confusion early on regarding who should initiate EPAs, when, where, and how often. An OSCE was developed and delivered during the resident orientation bootcamp. This consisted of 10 stations, each involving different aspects of care for the same patient (i.e., history, documentation, management of post-operative complications, etc.). Each station would end with the learner requesting the related EPA and receiving direct feedback from the faculty or senior resident instructor in the room. Through this activity, both learners and supervisors became more familiar with the process and, in turn, helped drive culture change in the program. Through these innovations, we observed considerable improvements in EPA attempts as well as improved qualitative and quantitative feedback on assessment forms.

EPAs are based on the traditional Royal College training standards. The bar hasn’t been raised, but now we are looking for proof of competence. This means we are checking to ensure they clear the bar as opposed to assuming they can clear it because they have finished their requisite months of training. Some standards require assessment of rarely observed skills, so simulation may be a necessity for achieving proof of competence in the new model. Our program instituted new trauma simulations to address this gap, and I would recommend working with your simulation experts to address identified needs in your own programs.

Firstly, you can't rush culture change, so don't get frustrated if the returns of your efforts are slow. Secondly, it's not all about the numbers. Try not to get lost in the endless quest for "five-out-of-fives" or which program is getting more EPA assessments per month. One of the primary goals of CBME is improved qualitative feedback, so try to use this as your primary barometer of success. And finally, the transition to CBD for new residents can be very challenging. I would encourage frequent check-ins with the PGY1 cohort in the first few months of training to ensure expectations and processes are well understood and that residents aren't falling behind for non-academic reasons.