In May 2015, an Amtrak commuter train near Philadelphia took a curve at twice the speed required to navigate the turn safely. The train fell off the track and rolled on its side. Eight people died in that incident and over 200 people were injured. The black box provided data after the fact that described the event, but didn’t prevent the event.
Just days after the crash, Allan Zarembski, PhD, PE, FASME, an expert on railway transport and track safety, talked about the role of Positive Train Control (PTC) in preventing exactly these kinds of disasters. Working with global positioning system (GPS) coordinates, the device correlates current location with data about the track and interacts with the train by automatically controlling the capacity to accelerate, acting as a “governor.” It also acts as a black box for the train, recording data about performance forces during transit.1
In 2008, Congress mandated PTC installation on passenger railway systems across the country. At the time of the incident, the system had not yet been installed because the cost was too high.
Investing in Safer Driving
EMS faces a similar crisis of safety. Aggressive driving in an industry pressured by response times in competitive markets as well as a mission mentality to save lives is a pervasive problem regardless of agency size, call type or service geography.
An analysis of Bureau of Labor Statistics looking at fatal occupational injuries revealed that EMTs and paramedics have the highest fatal injury rates compared with all workers. The most likely source of fatal injury is an on-duty vehicle crash.2 In 2015, the National Institute for Occupational Safety and Health recommended four approaches to reducing work-related crashes:
1. Safe and well-maintained vehicles;
2. Public policies to guide actions promoting road safety;
3. Company leaders to commit to road safety; and
4. Driver selection, training and evaluation to maximize road safety.3
Technologies targeting ways to improve this safety crisis have evolved over the last two decades and three types of safety systems have emerged:
1. Surveillance: These systems provide retrospective insight into driving behavior through data that captures violations of G-forces, speed, turn signals, seat belt utilization and other factors. Notification of an event can be marked by an audio tone, but audio, video and/or performance information is retrieved separately at a later time to review the event.
2. Coaching: These systems give real-time feedback, through warning lights, audible tones or even vibrating steering wheel, to inform the driver when driving is becoming unsafe, such as turning to sharply or lane departures.
3. Automatic drive controls: These systems interact with the vehicle to limit speed or even to apply brakes for a full stop in the event of imminent collision. They operate through GPS-linked information or use radar, cameras or lasers to “read” driving situations.
Research has provided significant evidence that the use of vehicle monitoring systems in several EMS fleets across the United States has made a significant improvement in driver performance and reduction in crash-related injuries and mortality. (See sidebar, “Evaluating the Impact of a Vehicle Safety System on Driver Performance,” below.) These published results recommend that the installation of these technologies should be mandatory as a way of preventing harm in the EMS industry. They also recognize that the biggest hurdle to jump would be the cost, which can run into the thousands of dollars for each vehicle’s initial installation, in addition to the costs in monitoring and maintaining that follow. 4—7
Evaluating ROI
Any true evaluation of the costs of EMS vehicle safety monitoring and feedback systems must be balanced with the return on investment (ROI). In keeping with the logic of any risk management system, the cost of mitigation must match the magnitude of potential loss.
The model for evaluating ROI builds on the logic that a more carefully driven vehicle should place less wear and tear on tires and wheels, transmissions, suspension systems and brakes. Fuel cost reduction should also be expected as quick stops and starts would be avoided and vehicles would supposedly maintain more moderate speeds. Savings should also be realized on insurance premiums for the fleet and through the reduction of collisions. We would also hypothesize that the utilization of safety systems would also influence the culture of safety. (See Figure 1.)
In 2013, a study was undertaken to understand how the real cost of investing in safety returned benefit to investing companies. The study included evaluations of ROI from an ambulance company using the surveillance method and ROI for an ambulance company using the combination of surveillance and coaching.
Impact of Surveillance
Superior Ambulance Service has operated in and around Chicago for over 50 years. With over 150 ambulances in the region, it’s the largest independently owned EMS provider in Illinois.
In 2013, Superior elected to begin rollout of a vigorous safety management system across the fleet. Their plan was to initially install a surveillance system and to then progress to a driver monitoring and feedback system. The surveillance system consisted of multiple cameras in each truck that were focused on the cab to observe driver and passenger behaviors as well as a recording activity on the road in front of and behind the vehicle. The surveillance system used was the in-car video system by Digital Ally, which allowed data for this study to be captured immediately after the installation and activation.
Vehicle crashes and incidents across several months of surveillance showed no significant decrease in the number, type or cost of crashes that occurred. Maintenance data was available for just a limited period after the installation of the surveillance equipment. There was a decline in maintenance costs about six months after the installation, but a change in maintenance recording processes meant that the data wasn’t available to identify if this was a trend over time or just a 2—3 month period of lower maintenance costs. (See Figures 2 and 3, below.)
Figure 2: Ambulance crashes after installation of survellance-only vehicle safety system
Superior Ambulance’s safety officer, Ken Sink, reported a savings of $100,000 in their annual insurance premium for the fleet in the year immediately after the installation of the surveillance system.
The system also provided visual documentation of their parked emergency vehicle being hit by a passing car. This evidence was used in a civil court case that contributed to the judgment assigning responsibility
of compensation to the driver of the passing car. The use of the system in a fleet of this size, however, required almost all the time of a single employee to review tagged incidences.
Figure 3: Crash and maintenance costs after installation of surveillance-only vehicle safety system
Surveillance & Audible Feedback
Butler Medical Transport operates ambulances throughout Maryland, Pennsylvania and the District of Columbia. Safety systems were installed in the second quarter of 2012 with feedback to the drivers made immediately operational. Butler used a combination of Road Safety from ZOLL Data Systems and the in-car video system by Digital Ally.
The rollout included a two-pronged incentive system for the drivers. Conditions of employment as a driver were amended to include consistent performance at level 5 (out of 10 levels). Those drivers achieving level 5 were entered into an annual drawing that awarded a family vacation to Walt Disney World.
Because of the quick rollout and immediate switch to feedback mode, pre- and post-comparison of the effect of the audible feedback tones wasn’t available. But average driver performance across the organization was at a level 7 over the two-year period of analyzed data.
Data for gallons of fuel and information about mechanical expenses were available from 53 ambulances for six quarters after the initial installation and activation of the safety system’s driver feedback tones. Fuel cost calculations were based on a constant price per gallon, so that fluctuating gas prices didn’t influence the change in expense. Fuel costs remained fairly constant despite a slight increase in miles driven. Repair expenses showed a slight increase over the time period.
There was a significant decrease in crashes and incident expenses for the two years measured after the safety system installation in Q1 of 2012. The single catastrophic event in 2011 was a freeway crash that resulted in property loss as well as personal injury. Butler Medical Transportation Operations Director Will Rosenberg cites this incident as a key factor in choosing to go to a combination of surveillance and driver monitoring and feedback system.
After the installation was completed and activated, costs of crashes or vehicle damage dropped from an average annual cost of approximately $20,000 to less than $5,000 over the next 10 quarters.
Butler maintained the view that the investment also returned a decrease in overall maintenance, however, the study was unable to track those savings through the documentation system.
Discussion
Very little has been published about measuring the return on investment. ZOLL often uses an ROI calculator based on data from Virginia’s Richmond Ambulance Authority. The data tracked vehicle “fails” as well as safety incidents over a three-year period and the agency reported a decrease in crash-related repairs and claims of 28% per year.8
The review of ROI on the two ambulance services provides some evidence that implementing vehicle safety and monitoring technology into an EMS system does return value in the decrease of crashes, vehicle damage and insurance/legal fees.
As with any retrospective study, the data has its limitations. None of the information was collected for the purpose of quality improvement, so much of the data needed to replicate the work of the Richmond Ambulance Authority was unavailable. Neither repairs nor fuel expenses were tied to miles traveled, and in many cases couldn’t even be traced back to the particular vehicle. Computerized maintenance tracking systems easily generated reports, but in a format that required cumbersome translation to usable data fields. Each piece of data–crash damage reports, legal fee records, maintenance expenses, fuel costs, mileage traveled, driver performance–was stored in separate records and databases, making retrieval and correlation difficult.
Future efforts to measure ROI will benefit greatly from a prospective approach with ongoing collaboration throughout the data collection period between the investigator and the agencies studied.
Figure 4: Claims data showing losses before and after installation of multiple vehicle safety devices
Conclusion
The cost of safety in EMS–much like the cost of safety in other areas of public transportation–must be evaluated by the return of employee well-being, extended careers, culture of safety as well as the dollar return on the hard-wired cost of maintenance, fuel and general operations. Culture, policies, procedures and management styles all influence safety performance.
Certainly in this ROI project, the targeted returns were evaluated only as those financial factors typically tracked. Both companies wondered what the return was in terms of employees who improved their driving skills both at work and at home: What was the effect of investing in safety systems on the morale and/or safety culture of the company? What impact did this have from the patient perspective? It’s not difficult to imagine a different kind of ride in the back from more cautious driving behaviors. And how does the modification of aggressive, fast driving affect the response times or run capacity of an organization? There are a lot of questions to take forward into further studies.
Extending operational safety beyond the black box, to identify what went wrong after the fact, has a benefit, theoretically, to the operations and culture of any organization. Quantifying that effect is difficult and requires a complex model to measure all the components that matter. Certainly a decision to make the investment requires the chance to evaluate data about the magnitude of the risk in comparison to the cost of mitigation.
Ken Sink from Superior Ambulance Service, summed it up best: “We don’t usually measure how great it is that everyone goes home from work safely at the end of the day.”
References
1. Zarembski AM. (May 14, 2015.) Safety technology could have prevented amtrak crash, says NTSB. National Public Radio. Retrieved Aug. 16, 2016, from www.sdpb.org/blogs/news-and-information/safety-technology-could-have-prevented-amtrak-crash-says-ntsb/.
2. Reichard AA, Marsh SM, Moore PH. Fatal and nonfatal injuries among emergency medical technicians and paramedics. Prehosp Emerg Care. 2011;15(4):511—517.
3. National Institute for Occupational Safety and Health. (March 2015.) Preventing work-related motor vehicle crashes. Centers for Disease Control and Prevention. Retrieved Aug. 16, 2016, from www.cdc.gov/niosh/docs/2015-111/.
4. Levick NR, Swanson J. An optimal solution for enhancing ambulance safety: Implementing a driver performance feedback and monitoring device in ground emergency medical service vehicles. Annu Proc Assoc Adv Automot Med. 2005;49:35—50.
5. Levick NR. (January 2006.) Hazard analysis and vehicle safety issues for emergency medical service vehicles: Where is the state of the art? Objective Safety. Retrieved Aug. 16, 2016, from www.objectivesafety.net/LevickASSEPDC2006.pdf.
6. Levick NR, Wiersch L, Nagel ME. (2007.) Real world application of an aftermarket driver human factors real time auditory monitoring and feedback device: An emergency service perspective. Objective Safety. Retrieved Aug. 16, 2016, from www.objectivesafety.net/LevickESVDriverFeedbackpaper07-0254-O.pdf.
7. Myers LA, Russi CS, Will MD, et al. Effect of an onboard event recorder and a formal review process on ambulance driving behavior. Emerg Med J. 2012;29(2):133—135.
8. Overton J, Blake C, Register T. Maximizing fleet performance. JEMS. 1998;23(10):36—44.
Evaluating the Impact of a Vehicle Safety System on Driver Performance1
This case study evaluated the impact of coaching with audible feedback provided by the ZOLL Road Safety system. Data from the devices were extracted from 14 vehicles in 11 ambulance services.
Drivers were given a driver grade ranging from 1—10, a composite score for driver performance based on the distance between aggressive driving events. Frequent acceleration, deceleration, and elevated side-to-side G-forces from high-speed turns would score the driver a grade of 1. Other unsafe driving practices such as speeding or not wearing a seat belt would also negatively impact the driver grade. A 10 is the highest score and reflects hundreds of miles between triggering events.
Figure 5: Driver performance across 11 EMS agencies pre- and post-activation of vehicle safety system audible tones feedback
For the first three weeks of the six-week study, drivers weren’t coached by the system. For the second half of the study, the audible feedback tones were enabled. As the driver approached and exceeded preset safety thresholds, the audible tone would intensify, coaching the driver to modify their driving behavior.
The data showed that driver behavior improved significantly as the drivers began to receive real-time feedback about their performance. (See Figure 5.)
Reference
1. Trial study: Changing driver behavior in EMS. (2015.) ZOLL. Retrieved Aug. 17, 2016, from www.zolldata.com/uploadedFiles/ZOLL_Data_Management/Success_Stories/Road%20Safety%20–%20Trial%20Study%20MCN%20DP%201509%200001.pdf.
