The implementation of an EMS data system is a challenge to any agency. It not only requires a strong commitment to electronic data collection but also an understanding of its impact on resources within the daily workflow. If you’ve already implemented an electronic patient care reporting (ePCR) system, the goal of this article is to provide you with insight and suggestions on how to effectively use your data. If you have not yet implemented ePCR but are considering it, this article will provide you with some insight into how to successfully choose and implement software.
Clean Data Is Good Data
One of the most important reasons to implement ePCR software is to use and apply information learned through the data that is collected. This sounds easy enough, and the best information to use and apply is typically the most current information available, often referred to as real-time data. In the world of healthcare, this information can become a form of vital signs to your EMS agency. However, just like vital signs can be incorrect or misinterpreted, so can data. Therefore, the first step to the application of EMS data is to make sure the data is clean, complete and appropriate, and that it correctly describes the issue or event.
Clean data is also called valid data, and the process of cleaning or evaluating the quality of the information in a data system is called data validation. This constitutes an ongoing circular process in which each data element to be used in a report is evaluated to determine its completion rate for each event. If the data element isn’t being collected on all of the EMS events as expected, one of the following processes should be put into place to improve the data collection process:
Staff education: Make sure your personnel understand the importance of collecting all the required information, which will point out areas where improvement is needed or the completing person’s identification number.
Business rules: If your ePCR software is capable, set business rules—data element completion requirements—so the data element in question must be completed for the type of events you’re evaluating. An example would be a rule where no ePCR could be finalized without entering the patient’s age or date of birth.
Checks and balances: Each ePCR is audited by staff (preferably an EMS supervisor) to assure that the required information is collected. Depending on the missing information, this could be corrected by the audit staff or returned to the EMS caregiver to complete the information.
Good Reports Are Choosy Reports
Once the ePCR data has been evaluated and determined to be complete and valid for use in a report, the next task is to identify the types of EMS events—or ePCR records—that should be chosen for each specific report. This is the most critical step in the design of any EMS report because the selected reports will provide an answer to a question regardless of its appropriateness. Therefore, the record selection process must ensure the answer is appropriate to the question.
Response time reports are some of the most common and most miscalculated in EMS. All agencies should be monitoring their response times, but how these are calculated can make a significant difference in the actual number. There are three important record selection decisions to make for many report types, including EMS response times. They include:
- What type of events, or responses, should be included (9-1-1, medical transports, stand-by, rescue, etc.)?
- What level of response was used by the EMS unit (lights and sirens or no lights and sirens)?
- Which data elements should be used to calculate the time (9-1-1 call time, EMS notification by dispatch time, EMS en-route time, etc.)?
All time-related reports are typically generated to measure how fast an EMS agency functions for a specific service delivery time interval. Since the goal is to respond or move as fast as possible, these reports typically should be calculated using only responses dispatched by 9-1-1 that included an emergency lights and sirens response. No other event types and response levels should be included in this type of time-sensitive report.
When Is Enough Enough?
Now that the data has been cleaned and appropriately selected for a report, there’s one last issue to consider. Are there enough events in the ePCR data for the report to reliably give you insight and information? How many events are required to accurately represent an agency’s response time or a provider’s success rate? If there aren’t enough events, the information in the report may lead to bad decisions or actions that don’t truly represent the agency’s normal operations.
When using data for performance improvement, the number of events used in the analysis is important. The reports should contain enough information to evaluate the topic over a set date range. But also keep in mind that if the selected date range is too long, the information may not reflect current performance. For response-time evaluation, a minimum of 500 events should be used.
It’s a Date
All reports use information either collected as an event through the ePCR program or entered into your data system to describe or define your agency. When a report is generated, it’s important to consider the date range of the events used. It’s also important for the date range to represent the period of time (or at least a similar period of time) the information is intended to describe.
As an example, if an agency creates a report to identify the number of EMS events in the data system for the past year, they should set a beginning and end date for the report. If a date range isn’t selected, the result will focus all events in the system instead of just the past year. Make sure your ePCR software reports allow this date range to be changed by your staff as needed.
Tell the Story You Want to Tell
Reports can also be displayed in many shapes and sizes. How you tell your story will depend on how much information is involved and how much detail is needed, as well as the knowledge and understanding of the audience it’s presented to. Reports should be able to be displayed either in tables or with graphics.
Tables provide numbers and calculations in their text and numeric form, and are similar to spreadsheets in their ability to display detailed numeric information. Graphs and charts represent the information in the report using a picture. Tables provide more detail, while graphs and charts provide a less complex visual presentation.
For large audiences and especially audiences that may not understand EMS, charts and graphs are a must. A well-designed graph or chart will be understood visually with little to no explanation. There are several types of charts and graphs, such as pie, bar, line, area and scatter. Each type has been designed to present information in a unique way. Using graphs and charts well requires practice and experience, but they are critical to communicating the information from any data system.
Can We Finally Generate a Report?
You survived! I’m sure you were beginning to think we would never discuss any specific reports. But now that we’ve ensured your reports are complete and accurate, let’s look at some specific reports every agency should be generating.
Most reports can be grouped into one of three types: service delivery, provider performance and patient care. Depending on how the information is being used, these reports can be general or detailed. Much of the information discussed in this section and the tables is based on information contained in the North Carolina EMS Performance Improvement Guidelines. These guidelines can be found at www.ncems.org/nccep.html.
Service delivery reports: These describe the resources and time parameters associated with the service component of an EMS Agencies operations. This could include anything from vehicle maintenance, supply use, scheduling and staffing, response and other time reports, and resource utilization. Performance improvement reports on EMS service delivery should include information on resource and equipment, time parameters, and complaints and investigations. See Table 1 for a breakdown what information should be included in performance improvement reports for this category.
Table 1: EMS Service Delivery Reports
Resource and Equipment Reports
- Vehicle failures
- Vehicle crashes
- Patient care equipment missing from daily inspections
- Patient care device failures
- First responder on scene percentage
- Dispatch center: time of call receipt
- Turn-out (wheels rolling time)
- Response time to scene
- Response time to the patient
- Scene time
- Transport time
- Back-in-service time
- Dispatch center delays
- Response time delays
- Scene time delays
- Transport time delays
- Turn-around time delays
- Frequency of ED off-load delays
- Internal service delivery
- Patient care complaints
- External service delivery
- Patient care complaints
EMS professional (patient caregiver) performance reports: These describe the performance of each EMS professional across the spectrum of job duties and responsibilities. The focus should be on performance measurements which impact service delivery or patient care. See Table 2 for a breakdown what information should be included in performance improvement reports for this category.
Table 2: Provider Performance Reports
- ePCR documentation completeness
- Protocol used documentation
- Vital sign documentation
- Skills performed documentation
- Skill proficiency
- Protocol compliance
- Patient contact as an EMS unit crew member
- Patient contact as a primary patient caregiver
- Number of ePCRs completed
- Continuing education hours
- Individual dispatch times (if emergency medical dispatcher)
- EMD protocol compliance (if emergency medical dispatcher)
- Controlled substance log ;compliance
Patient care reports: These describe the treatment provided to individual patients as well as the overall care provided to groups of patients. The patient care provided during high-risk events should also be monitored and evaluated. Performance improvement reports on EMS service delivery should include information on care of individual patients as well as those in high-risk groups and at high-risk events. See Table 3 (p. 14) for a breakdown of what information should be included in performance improvement reports for this category.
Table 3: EMS Patient Care Reports
Individual Patient Care
- Patients with no documented protocol
- Patients with no documented category (Defined by NEMSIS E09-11, 12, 13, and 15)
- Patients with medication complications
- Patients with skill complications
- Patient triage and destination plan compliance
- Patient pain control
High-Risk Groups Patient Care
- Frequent flyers (EMS use greater than four times per month)
- Repeat patients (within 48 hours)
- Deaths while in EMS care
- Patients with restraint use
- Patient refusals
- EMS patients where EMS response was canceled by first responder
- Patients with obstetrical deliveries by EMS
- Patients with assisted ventilation or invasive airway use
- Patients who underwent drug-assisted intubation or rapid sequence intubation
- Patients who underwent chest decompression
- Patients who underwent cardioversion
- Patients who met trauma center criteria
- Patients with acute STEMI
- Patients with acute stroke
- Pediatric patients with acute serious illness or injury
- Patients with cardiac arrest
- Patients with a Glasgow Coma Score of <9
- Patients with abnormal vital signs
High-Risk EMS Events Care
- Physician on scene
- Multiple patients
- Mass gathering
- Police custody
- Tactical EMS
- Wilderness EMS rescue
Are My Results Good or Bad?
EMS is the youngest healthcare industry. That means we’re still learning, developing, improving and standardizing our service delivery and patient care approach. EMS is also very diverse in how service is provided across different community sizes and geographic areas. As a result, there are very few performance standards that can be used for measurement and comparison.
The best way for EMS agencies to learn from each other and evaluate service delivery and patient care performance is through benchmarking, a method by which agencies compare themselves with similar agencies. This is not done by receiving a grade, but by comparing report results that are defined and measured the same way. Two sources of information, the National EMS Database and state EMS data systems, can be used to benchmark your agency against others.
The National EMS Database provides standardized reports using the data provided by participating states. Many national EMS reports are grouped by agency size (urban, suburban, rural and wilderness). Each can determine their group using information provided on the NEMSIS Web site and then benchmark their local results with the national report.
Some states provide reports with benchmarking through their state EMS data system. For example, North Carolina provides an extensive benchmarking and performance improvement solution with the EMS toolkits through their state EMS data system and the EMS Performance Improvement Center ( www.EMSPIC.org). Check with your state to determine if benchmarking reports are available.
This article only begins to describe how to use and apply data from an ePCR system. Having a data system is not the same as using a data system. The success of any data system depends on how the information is used. Don’t accept poor-quality data when you make operational decisions. Make sure the reports you generate answer the specific question you have. Remember, you can manage what you can measure.
I encourage you to seek out other agencies in your area. Everyone can benefit as best practices are identified, and as each develops reports, the design can be shared. This also sets up a great opportunity for benchmarking.
The ability to generate reports will ultimately be based on the capabilities and reporting tools associated with either the software you purchase and update or the state EMS data system. As a baseline, any software considered should be certified as NEMSIS Gold Compliant. A list of NEMSIS Gold Compliant Software can be found at www.NEMSIS.org.
Additional information on EMS data and performance improvement, including the EMS Performance Improvement Toolkits, can be found at www.EMSPIC.org.
Disclosure: The author has reported no conflicts of interest with the sponsors of this supplement.