Beginning January 2017, candidates attempting the National Registry of EMTs paramedic certification will see a new and improved psychomotor exam. In the first phase of implementation, candidates will test at six stations, including an integrated out-of-hospital scenario, instead of the current 12 discrete skills. Preparations for these changes have been carefully planned and piloted in consultation and support from a robust group of stakeholders.
Candidates will continue to perform the current trauma assessment, two oral stations (A and B), and cardiology (dynamic and static). These stations won’t change.
Subject matter experts representing EMS educators, medical directors, state officials, training officers, field clinicians and supervisors have all been involved in dozens of planning meetings, review sessions and focus groups since 2008 to update the exam.
One of the biggest changes is the addition of a paramedic psychomotor competency portfolio requirement. Students beginning a paramedic program on or after August 2016 are required to document their psychomotor practice and proficiency with a portfolio that’s designed to show progression in learning and competence.
Students start by documenting sufficient progress with individual skill repetitions on task trainers, progressing to skills performed during scenarios in classroom labs. The portfolio includes recommendations for hospital clinical rotations and a capstone field internship.
Thirty-three new detailed formative skill sheets are guides for novices to learn each step of a skill. Evaluation forms for scenarios and field clinical rotations are also available. More importantly, the portfolio has incorporated concepts of crew resource management (CRM) and EMS culture of safety.
Although completing a portfolio is required, both the NREMT and the Committee on Accreditation of Educational Programs for the EMS Professions (CoAEMSP) are allowing programs flexibility in the types of forms, tracking methods and outcome goals. Each program director will be required to provide attestation of the completed portfolio of their students.
An example of a portfolio and skills sheets are provided on the NREMT website and can be used or modified by programs; however, instructors should be mindful of how and how much they modify. After all, the portfolio was piloted and revised yearly since 2009 until being implemented by programs nationwide. Suggested minimum repetitions for individual skills and types of patient complaints are based on findings from more than 100 pilot schools with data collected for more than five years.
The process of review, consensus building and revision based on these pilots gives the NREMT-developed portfolio internal and external validity. Inventing different goals or editing steps in skills should ideally be re-validated and based on evidence, not just personal preferences. Programs are encouraged to set their own target goals in consultation with their advisory groups and medical director.
Formative vs. Summative
Many instructors and students use the current NREMT evaluation forms as both learning and final testing tools. The old forms were intended for summative, final evaluation of a single skill, not as a learning tool. When using these forms to study, candidates think—wrongly—that they have to memorize all of the steps listed, when the reality is that the exam expects candidates to care for the simulated patient by setting the right priorities, not blindly reciting a series of memorized steps.
The 33 new and/or updated individual skills and scenario evaluation forms have been created for learning.
To be successful at implementing this portfolio, programs will want to ramp up, and re-think formative and summative scenarios. One educator recently described the “formative” as a step-by-step recipe, in which you gradually add ingredients to a dish. Eventually, these dishes make a full meal.
Formative scenarios can be described in this context as the re-enactment of a patient-centered case, where students are gradually adding in newly acquired skills.
Formative scenarios can be described in this context as the reenactment of a patient-centered case, where students are gradually adding in newly acquired skills they learned on a task trainer, into increasingly realistic simulated environments.
The formative scenarios would therefore gradually increase in difficulty and quantity of skills as the student progresses through the program. This is similar to how programs should increase the cognitive level of written exams to improve critical thinking ability.
Building upon learned skills with additional skills and greater complexity improves performance. As skills are learned and repeated, they’re retained in long-term memory where they can easily be retrieved when necessary. However, this process should be progressive. (For more, read “Effective Sims: Applying cognitive load theory to design simulation training,” by David Page, MS, NRP & Bill Robertson, MS, NRP, in the March 2016 issue.)
Summative scenarios in this analogy are more like combining multiple dishes and courses into a full meal experience, complete with the affective (behavioral and subtle attitudinal ingredients needed). Toward the end of a program, the novice has practiced each of the segments or parts of the meal, including the soft skills required to set a healthy and positive environment.
Putting it all together, the multiple courses and dishes are now combined, sometimes with less-than-ideal ingredients. The successful learner isn’t necessarily following every step of the recipe in an exact order, but rather adapting the recipe, ensuring key ingredients and actions end with the desired result.
Similarly, the successful paramedic graduate can and should modify their assessment and care to manage a diverse range of patients effectively.
New Out-of-Hospital Scenario
The new practical exam will include a 20-minute scenario. The candidate must successfully perform as a team leader, caring for a simulated patient with a professional partner.
It’s expected that one or both team members will actually perform the skills required. These will be performed on a mixture of simulated patients with task trainers, where appropriate, or manikins.
The professional partner is an already graduated and registered paramedic. This partner will perform skills properly as directed.
The NREMT, in partnership with the National Association of EMTs, produced a video to help instructors and candidates, available on the NREMT website.
The current practical has been in use for 20 years. It’s a psychometrically valid and proven exam. Although hallway conversations with educators have sometimes generated concerns over inter-rater reliability and cost effectiveness, no one in the industry has proven there’s a better system to verify competency.
The industry experts have debated the need for a psychomotor practical for a long time. Why not just allow instructors to sign off on skills? In a way, the portfolio does just that. The bulk of skills will be evaluated by the program’s instructors. The NREMT exam is only a spot-check confirmatory step.
The debate will continue to be rhetorical until proper research can create an acceptable body of evidence to answer the question more scientifically.
In the absence of empirical evidence, the anecdotal prevailing opinion centers on the concept of verification of competency. Our educational system is just now fully embracing accreditation and peer review. Annual reports and a site visit every five years may not be sufficient to ensure the current level of accountability. The current use of a cognitive and psychomotor exam is an essential industry standard to protect the public.
In early pilots of the scenario exam, it was clear that while candidates were able to perform individual skills in isolation, when they were asked to do the same skills in the context of a scenario, they struggled.
With increasing focus on culture of safety, teamwork and the benefits of more realism in simulation, the concept of using a more realistic, summative, integrative and case-based approach was explored.
This change is really about “developing a thinking practitioner.” Pilot schools have overwhelmingly reported increased student practice and accountability in their own learning. As scenario practice increased, students have also reported satisfaction with a more realistic practice model of lab.
Preceptors have reported improved skill and critical thinking ability in the field. Just as importantly, the NREMT has reported seeing improved cognitive exam scores from students completing the portfolio.
Ideally, the EMS industry, providers and educators have been keeping up with the alarming reports of medical errors1 and need for a culture of safety, CRM and improved simulation-based education.
Curriculum materials for EMS to incorporate these teachings continues to evolve. Initial reports from educators involved in the pilots appear to indicate that incorporating these concepts on the national paramedic psychomotor exam means that the industry, educators and publishers will more rapidly adopt and teach these important concepts.
What About EMT & AEMT?
The National Registry hasn’t announced plans to make changes to the emergency medical responder (EMR), EMT and advanced EMT (AEMT) psychomotor exam requirements. The rigor of Paramedic Program Accreditation by the Commission on Accreditation of Allied Health Education Programs (CAAHEP) required of all paramedic programs set the foundation to investigate moving away from focusing on 12 isolated skills and moving the National EMS certification process for paramedics to a more scenario-based evaluation. However, EMR and EMT psychomotor exams are largely a function of industry and state regulatory agencies. Instructors and state officials don’t need to wait for the NREMT to institute an EMR, EMT or AEMT portfolio.
Introducing all EMS providers to increased scenario practice, teamwork and culture of safety can only improve patient care in the field. The paramedic portfolio includes many BLS skills and concepts that apply to any provider level.
Over the past five years, there has been a significant effort to notify all program directors and state agencies about these upcoming changes. Unfortunately, staff turnover and workloads have sometimes resulted in educators not knowing about the change, or the level of effort needed to prepare for it.
Paramedic instructors reading this article shouldn’t delay in downloading, reading and becoming experts in the NREMT portfolio and new scenario exam. Trying to implement the portfolio in the middle of a class would be like trying to rebuild an airplane mid-flight. It’s essential we have an industry-wide proactive approach.
1. Makary MA, Daniel M. Medical error-the third leading cause of death in the US. BMJ. 2016;353:i2139.
Where to find more information
NREMT portfolio information: www.nremt.org/nremt/about/ppcp_info.asp
EMS Culture of Safety: www.emscultureofsafety.org
EMS performance measures: www.emscompass.org
NREMT/NAEMT instructor and candidate video: http://bit.ly/nremtpractical
EMS checklists: www.emsreference.com/checklists
Free 25-min. lecture on concepts of CRM: http://bit.ly/EMSCRM
UCLA list of props for scenarios: http://bit.ly/emsproplist
Formative vs. Summative Assessment: www.azwestern.edu/learning_services/instruction/assessment/resources/downloads/formative%20and_summative_assessment.pdf
Cognitive load theory: http://bit.ly/cltjemscolumn
Building a safer health system: www.ihi.org/resources/Pages/Publications/ToErrIsHuman.aspx