There’s a growing body of research in education,1 behavior change2 and performance improvement3,4 that shows “timely” feedback measurably improves the effectiveness of that feedback as well as future performance.

Some feedback should be delivered in realtime5—as the event is occurring—so that the person doing the task can use that feedback to adjust their activity or technique to improve the outcome of that event.

Other feedback should be delivered as soon as practical after the event—near-real-time, either to avoid distraction during the event, or because the activity isn’t recorded or measurable until after the event is concluded. Until recently, we were severely limited in our view of on-scene performance. We were unable to know if the patient’s outcome was due to the crew’s performance, the patient’s condition, or a combination of the two. When assessing for quality measures, all we had to go on was the EMT and paramedic’s documentation.

Now when we look at an entire patient encounter, not only do we have the crew’s documentation in the electronic patient care report (ePCR) and data from the ECG monitor/defibrillator, but also the patient data from computer-aided dispatch (CAD), emergency medical dispatch and, in some systems, the hospital ED and discharge diagnosis information. Technology has made it possible to determine how well our system and crews are doing by comparing data collected across these sources.

Analysis of the data can tell us a lot: Are we doing the suggested 100–120 compressions per minute and compressing at least two inches? Do we know we’re actually ventilating the patient 10 times per minute? Are we giving our patients the right treatments at the right times, based on their condition? Does our medical director know we’re following established protocols based on evidence-based guidelines? Are we documenting our patient findings and care accurately and in a way that allows our service to improve, get reimbursed and reduce risk?

Real-time and near-real-time feedback devices and software that analyze data and performance are revealing that we may not have been as good as we thought. This new technology allows providers to see exactly how they’re performing during the call or shortly thereafter, ensuring the patient receives the best possible care. It also allows administrators and medical directors to review quality measures and see exactly how the crew performed in order to provide feedback for providers so they can learn from each call while it’s still fresh in their minds.

Real-Time Feedback

The ECG monitor/defibrillator is an invaluable tool that continues to evolve. New technologies are giving us ways to better evaluate and help our patients. The advent of real-time feedback has been around for years and whether you know it or not, you’ve been using it.

Monitoring oxygen saturation or SpO2 shows us in real-time if our efforts with oxygenation are effective. Before most of us knew the usefulness of capnography for CPR performance, we only used it to verify tube placement and then monitor ventilations to avoid hyperventilation. These are examples of real-time monitoring.

Newer versions of this technology in today’s monitor/defibrillators take care to the next level. The screen has an organized, audiovisual dashboard the user can see and use to guide care. Most of the feedback is used to assist CPR performance, but this too is evolving. Depending upon the manufacturer, there’s information on rate, depth, release/recoil, pauses, elapsed time, a countdown timer, SpO2 monitoring and EtCO2 monitoring.

Although many EMS agencies have real-time software on their defibrillators, few actually use it. Not using feedback devices is like driving on a highway at night without your headlights on; you can do it, but it’s more dangerous and less effective than driving with them on.

Using Real-Time Monitoring

The American Heart Association (AHA) Consensus Statement on CPR Quality, published in June 2013, sets out the most current evidencebased guidelines on CPR.6 But, how do we know we are complying with the AHA’s guidelines? You guessed it, by using real-time feedback technology.

When performing chest compressions, position one crewmember so they can see the display screen and hear the audio cues on the defibrillator. Information on the screen clearly displays what the compressor is doing regarding rate, depth and pauses. If performance isn’t within the AHA guidelines, there are cues that make corrections easy.

Figures 1 and 2 show a visual representation of two cardiac arrests created by the manufacturer’s software after uploading the code file from the monitor to a PC. In the depth box, each blue line is a compression. Yellow indicates a pause, and the green stripe across the top of this box is the ideal depth. In the rate box, each brown dot is the rate for each compression.

Figure 1 shows a cardiac arrest where the crew didn’t have the CPR dashboard visible. They were performing “blind,” relying only on their training and what they felt was the best care. You can see the rate (148 compressions per minute) and depth (1.39 inches) aren’t within the guidelines and there are excessive pauses. The CPR fraction is only 51%, meaning compressions were only being done half the time during the arrest.

Figure 1: Cardiac arrest resuscitation without feedback

Figure 2: Cardiac arrest resuscitation using CPR dashboard

Figure 2 shows a cardiac arrest where the crew used the CPR dashboard. You can see the rate, depth and pauses are all within the guidelines. The improvement is striking!

This technology makes it easy to hit 100–120 compressions per minute and achieve a depth of greater than 2″ every time. It also helps maximize the chest compression fraction by alerting the user when pauses occur and facilitates and organizes cardiac arrest management with the use of a countdown timer.

When the timer gets to zero and resets, the crew knows to check the rhythm, defibrillate if appropriate, and change compressors. Those running the scene can more effectively monitor the performance and coach those at the task level. Crews are able to see if they are complying with the AHA’s recommendations for high-performance CPR.

Does It Affect Outcomes?

EMS organizations that use devices that have realtime CPR feedback are showing improvement in cardiac arrest survivability. One study in Mesa, Ariz., showed significant improvement in cardiac arrest survival.7 By conducting scenario-based training and using real-time CPR feedback, both Guardian Medical Transport and the Mesa Fire and Medical Department (MFMD) significantly improved out-of-hospital cardiac arrest survival.

The MFMD’s compression fraction went from an average in the low 60% range to the mid 80% range. Before implementing this change, the MFMD’s survival to discharge for patients with a witnessed shockable rhythm was 26.3%. After these changes were applied, survival to discharge increased to 55.5%. These patients were 2.72 times more likely to survive.7 Although it’s not a double-blind study on the use of real-time feedback, it’s a great indicator of the significance of this technology.

Identifying Performance Deficiencies

Before the study, MFMD weren’t measuring key parameters. The initial phase of the study identified the average chest compression fraction was in the low 60% range. Average rate and depths weren’t hitting the mark and pre- and post-shock pauses totaled over one minute. Needless to say, MFMD officials were stunned by actual performance. If you’re not measuring performance, you don’t know how you’re doing.

Along with CPR performance metrics, other issues that decrease CPR performance were identified: compressor fatigue, transportation, advanced airway placement and EtCO2 monitoring.

We now have a visual representation of how fatigue affects the compressor’s ability to do high-quality chest compressions and know that, after about two minutes, even the fittest person starts to lose effectiveness. They may tell you they’re not tired, but you can see compression depth starts to suffer, and they unconsciously speed up the rate to compensate. To avoid this, use your countdown timer and change compressors every two minutes!

It makes sense, but now we’re able to quantify that our CPR quality goes down during transport. Not only are the members at great risk because they’re most likely not seat-belted in the back of the ambulance, but compression quality suffers. More and more evidence is finding that the best care for cardiac arrest is to work the patient on the scene until return of spontaneous circulation, field termination or the use of mechanical compression devices.

Compressions also suffer during advanced airway procedures. Unless you have a policy and train to intubate without interrupting compression depth or rate, chest compressions will suffer.

Dan Spaite, MD, a prominent researcher at the University of Arizona, has dubbed the phrase, “EtCO2 monitoring—that isn’t.” As part of his research with EMS agencies, he has found when EtCO2 is being applied, few providers are actually monitoring and correcting what the monitor is telling them. If we don’t use this information to guide our ventilations, it merely documents us hyperventilating the patient.

But just because you have the coolest, newest technology doesn’t mean that your front line crews will know what to do with it. Crews need to be educated on the components of high performance CPR and why they’re important. Small group, in-person training is the cornerstone for transitioning to this technology. Also remember, just because the information is in front of your face doesn’t mean you are looking at it. Your crews need to know the importance of the information being presented on the dashboard and taught not to get distracted from it.

This technology can also easily be used in a training mode on manikins to improve performance on the streets. You can quickly upload the data during the training session and show the crews exactly how they did. And, if you incorporate this type of training at the onset, doing it for quality improvement purposes is an easy transition.

 

Near-Real-Time Feedback

Near-real-time feedback in EMS can mean different things to different people, but many people consider it to mean feedback from any time after the end of the call to the end of the same shift, or in some cases early into the shift, or at least during the next shift worked.

The goal is to provide feedback while the call is fresh in the person’s mind, when they can recall the details of the call, including timing, patient condition, treatments, dosages, etc. This is important because it may vary from, or provide more context to, the documentation.

The most effective time for feedback can reasonably vary. It can be extended, for example, in smaller or rural systems where crews run only a few calls each shift, or it may be shorter in very busy systems where crews take a large number of calls per shift. Also to be considered are systems with shift schedules with gaps of several days between shifts, since it may be difficult to recall details across several days, or where the person works in another EMS system in between shifts, etc.

The research doesn’t explain why timely feedback is more effective, but we speculate that the more timely and concrete the feedback is, the easier it is for the EMT or paramedic to personalize the feedback and incorporate the lesson into their behavior, while more delayed feedback becomes more abstract and less personal—like a regular class that applies to everyone equally.

It’s important to note that some data isn’t easily available to monitor during the call, and can only be gathered and effectively analyzed after the call, so it’s not always feasible to give feedback during the call, but feedback should still be provided in a timely manner.

A challenge in providing timely feedback is when the data or the monitoring mechanism aren’t black and white and requires some human review, context and discussion. EMS systems are now overcoming this challenge by using technology and automation to analyze call information in near-real-time, and use the software to review all possible aspects of the data that can be done by technology, saving the human reviewer’s time so it can be used to review aspects of calls that only a human can handle. In other words, they let the computers do what they do best, and save the humans for the parts of the QI review that require judgment, experience, and often, a sense of the bigger picture.

Below are some examples where near-real-time feedback can be used to improve EMS:

Communications center improvement: It’s a primary goal of all communications centers to decrease the time it takes from when a 9-1-1 call is received until enough information has been gathered from the caller, so that it’s available in the CAD system and an EMS unit can be assigned. After months of trying to improve call-taker performance through traditional retrospective reporting, CenCom (New Jersey) Manager Gareth Williams began to display live gauges showing the percentage of compliance over the last 12-hour period to their call-taking performance goal on large screens in the communications center showing the overall performance of all call-takers in the center.

Without even having to review individual performance, provide additional training, or use discipline, CenCom’s call-takers improved their own performance from 77% to 92% over a few months.

Automatic near-real-time monitoring of dispatch data has another benefit: Sunstar (Pinellas County, Florida) uses automatic alerts to reduce the workload of communications center staff by automatically sending out management notifications for certain kinds of calls, such as first responder transporting, medical helicopter usage notification, multiple unit responses, calls with long response times, etc.

This is especially helpful, because these messages, although important, create a great deal of related work in the communications center, and management notifications are often a lower priority and can be delayed or even missed.

Operational improvement: The San Miguel Fire Department in San Diego County decided to improve their out of chute times, and used near-real-time feedback via desktop and mobile dashboards along with automatic alerts for each unit on each shift to provide same-shift feedback to crews. This near-real-time feedback improved their performance from 75% to over 90%, and gained buy-in for the value of real-time feedback from their leadership at all levels.

The Orange County (Florida) Fire Rescue Department has been working to reduce their hospital offload times, and uses automatic near-real-time alerts to notify their battalion chiefs (BCs) when an offload exceeds 30 minutes, so the BC can go to the hospital, determine the cause for the delay, and take over patient care if necessary, releasing the EMS crew to return to duty and be available for another call.

Clinical improvement: St. Charles County Ambulance District in Missouri uses near-real-time alerts to notify their EMS BCs when the scene time for STEMI and stroke calls are greater than 10 minutes. This allows the BCs to follow up with crews later in the same shift if their scene time for these time-critical patients was long.

Next to providing the highest level of patient care and ensuring safety for all involved, properly documenting the call, patient’s condition and care provided is one of the most important things EMS providers can do. One missing data field can be the difference in whether or not your organization gets paid for patient transport, or is liable if a legal question about the call arises later.

Williamson County (Texas) EMS recently used automatic monitoring of their ePCR data to determine that some paramedics were incorrectly documenting their administration of fentanyl in milligrams, rather than micrograms. The actual dose provided was correct in each case, but could have raised questions if any of those cases was called into court for other reasons, so early discovery, feedback and the resulting improvement was important.

If an EMT or paramedic doesn’t document complete patient information, it could take your billing office hours to track down the information needed to complete a single patient record. The Richmond (Virginia) Ambulance Authority (RAA) uses FirstWatch, which allows them to monitor in real-time from a variety of data sources: CAD systems, ePCRs, records management systems, public health, even emergency departments and hospitals. The software provides them with a real-time solution, allowing them to make interventions right away rather than waiting to run a report. Duty supervisors receive a near-real-time alert for every incomplete ePCR that they can then immediately direct to the appropriate field crew member. This allows field providers to correct the missing ePCR information before they’ve even ended their shift. Since implementation, missing ePCRs a day have gone from as many as 7–8 times a day to either once or none each day.

This illustrates how near-real-time feedback has the power to affect change. Other EMS agencies have adopted this approach, and had similar success, including the highly regarded North Shore-Long Island Jewish EMS system.

Richmond Ambulance Authority Chief Operating Officer Rob Lawrence trains his staff using FirstWatch’s real-time dashboards. Photo courtesy FirstWatch

Quality Improvement

The data collected in real-time and near-real-time is also invaluable for improving quality. “Review of the quality and performance of CPR by professional rescuers after cardiac arrest has been shown to be feasible and improves outcomes. Despite this evidence, few healthcare organizations apply these techniques to cardiac arrest by consistently monitoring CPR quality and outcomes.”7

This technology can also be used to conduct performance reviews and show crews exactly what went well and what didn’t go so well. Automated systems that evaluate performance in the communications center and in the field (operationally and clinically) can help dramatically reduce the amount of work required for a quality improvement (QI) review and consequently reduce the time from the call to the delivery of effective feedback.

Many systems are now using their data systems and hardware to reduce in-house staffing and workload. For example, Sedgwick County (Kansas) EMS has worked to have FirstWatch and FirstPass to take all possible review work off of the shoulders of their QI staff member, so they can focus their attention on aspects of the QI process that requires a human touch.

The previously mentioned RAA uses First-Watch and FirstPass to review 100% of their calls within minutes of the dispatch, ProQA or clinical data hitting their databases. The initial, automated review happens immediately, and patient care that doesn’t comply with RAA’s protocols, or is simply outstanding, are made available right away for human review.

RAA staff routinely provide complete QI feedback on acute calls to their crews within an hour of the call, and non-acute calls worthy of feedback during the same shift or by the next shift. Prior to implementing the automated near-real-time feedback system and approach, they reviewed 100% of the cardiac arrests, and about 25% of all other calls. Their goal was to review the cardiac arrests by the next day, and the 25% within several days.

In addition to this near-real-time feedback on a per call basis when appropriate, RAA is able to use past calls and overall system protocol compliance to identify which issues are really system issues, rather than issues with individual medics, and incorporate that information into their system’s continuing education, and into individual preceptor activities.

This automated, near-real-time review of information about all calls, from multiple data sources, provides visibility into system and crew performance which helps provide context about the system, the crew’s past performance and other crews’ performance in similar circumstances to give as complete a picture as possible, and avoid knee-jerk reactions. And, EMS systems that use statistical process control-based approaches (e.g., Six Sigma) in their QI programs can base their analysis, alerting and feedback on only those protocols or measures where it’s appropriate.

Summary

Progressive EMS organizations need to be monitoring, capturing and measuring data continuously, in real-time and near-real-time to ensure quality patient care and optimum clinical and operational performance. Previously, this required exhaustive staff time and efforts, cobbling data together manually from various sources. We can now use EMS technologies to make useful, actionable decisions in near-real-time based on the data we collect—all as close to the event as possible.

While technology is fantastic, it’s still very important to keep an overall focus on the patient.

“Treat the patient, not the monitor” is a very common sentiment heard in paramedic training programs around the world. The availability of technology doesn’t preclude this statement. While crew members performing tasks on scene should concentrate on the job at hand, when possible, there should be a team leader that watches over the entire scene to help provide real-time direction and feedback. Crews can see what they are doing while they are doing it and it improves outcomes.

Real-time and near-real-time information and feedback shows leaders where their system is headed. It gives our patients the best chance at the best outcomes, and gives our EMS systems the best chance to improve and provide measurable outcomes.

References

  1. Dean CB, Hubbell ER, Pitler H, et al: Classroom instruction that works: Research-based strategies for increasing student achievement, 2nd edition. Association for Supervision & Curriculum Development: Alexandria, Va., 2012.
  2. Shuger SL, Barry VW, Sui X, et al. Electronic feedback in a dietand physical activity-based lifestyle intervention for weight loss: A randomized controlled trial. Int J Behav Nutr Phys Act. 2011;8:4.
  3. Stockford P. Good to great: Rapid results with real-time performance management [white paper]. SaddleTree Research: Scottsdale, Ariz., 2012.
  4. Robinson DG, Robinson JC: Performance consulting: Moving beyond training. Berrett-Koehler Publishers: Oakland, Calif., 1996.
  5. Shute VJ. Focus on formative feedback. Review of Educational Research. 2008;78(1):153–189.
  6. Bobrow BJ, Vadeboncoeur TF, Stolz U et al. The influence of scenario-based training and real-time audiovisual feedback on out-of-hospital cardiopulmonary resuscitation quality and survival from out-of-hospital cardiac arrest. Ann Emerg Med. 2013;62(1):47–56.
  7. Meaney PA, Bobrow BJ, Mancini ME, et al. Cardiopulmonary resuscitation quality: Improving cardiac resuscitation outcomes both inside and outside the hospital: A consensus statement from the American Heart Association. Circulation. 2013;128(4):417–435.