Administration and Leadership, Documentation & Patient Care Reporting

Pinellas County (Fla.) EMS Implements Truly Effective Quality Improvement

Issue 2 and Volume 43.

Pinellas County (Fla.) EMS implements an effective QI process

Photo James Zabon/courtesy RedFlash Group

Has your quality improvement function actually made anything better? The honest answer for many EMS systems is, “No.” For those that can say yes, better documentation is often cited as their evidence—not the kind of improvement that inspires thank you letters and cookies from grateful patients.

The EMS system in Pinellas County, Fla., is one that’s made measurable clinical improvements to the care that’s provided to their patients and their community.

Its success can be credited to deep collaboration between more than 1,800 frontline EMTs and paramedics working in 19 different organizations across this western Florida coastal community: 18 municipal fire departments providing ALS first response and Sunstar Paramedics, the contracted ALS ambulance service.

These agencies, along with the Pinellas County EMS and Fire Administration and the EMS medical director, work together in a unique way that delivers real benefit to patients.

Getting nearly 2,000 EMS clinicians on the same page about providing consistent, high-quality care is remarkable and doesn’t happen by accident.

Since 2008, Pinellas County EMS providers had been using electronic patient care reports (PCRs) and manually tracking performance indicators. A few years later, they looked for ways to automate and improve the system.

In 2014, the clinical leaders from each of the 19 organizations, and Angus Jameson, MD, their new EMS medical director, decided that they wanted to transform their quality improvement (QI) system into one that’s truly patient-centered, inclusive, comprehensive, respectful of all providers and free from fear.

They began by building trusting relationships through transparency, open sharing of data, and implementation of Just Culture, defined by the Center for Patient Safety as an organizational culture that “supports open communication of errors in a non-punitive environment.”1 (See Table 1)

To help facilitate the process, the Pinellas team turned to FirstPass, software that was created by data analytics organization FirstWatch. The software allows them to monitor the care delivered and documented on more than 200,000 EMS calls a year, provide feedback and coaching to each EMT and paramedic in near real-time, and improve overall performance.

Management of individual quality assurance reviews or medical case reviews is based upon this framework. A flowchart for Just Culture implementation and decision making is contained in the EMS medical director’s Medical Quality Management Plan to ensure the proper application of the framework on a case-by-case basis.

Selecting a QI Project

One of the challenges in the Pinellas system is having a large number of organizations, each with its own leadership structures and policies.

An important piece of the county’s EMS QI system is a written Medical Quality Management Plan that clarifies the roles of the county medical director and the roles of the individual departments.

In addition, prior to implementation, each organization made sure to communicate clearly the purpose of FirstPass and what it was being used to look for—and why.

For example, one of Pinellas County EMS’s first improvement projects focused on patient interactions that didn’t result in transport to the hospital. This was chosen after Jameson and others noticed that the county had a relatively high refusal rate and the documentation of this potentially risky decision was often minimal.

Many of the patient care reports simply stated, “No ambulance needed,” or, “Cancelled on scene,” even when crews had been with the patient for more than 15 minutes.

A protocol existed for managing patient refusals, but providers frequently didn’t follow it and there was wide variation across the county in how these patients were handled.

To address the issue, Jameson collaborated with all clinical leaders on an update to the Pinellas County EMS refusal protocol that included current legal perspectives. The performance improvement team also began working to build trust with each provider organization, their clinical leaders and their frontline paramedics.

With a strong commitment to a non-punitive approach, they collaborated and worked hard to demonstrate a focus on improving care for patients, rather than punishing providers. They made their expectations crystal clear and were very transparent with systemwide and department-level performance data.

Before they started providing performance feedback, they reviewed all of the quality measures and baseline data in detail. Paramedics were also provided detailed examples of how to manage a variety of different situations involving patient refusal of transport.

In addition, the team defined performance measures for patient refusals and designed a communication plan to share information with the entire system.

The measures were built into their FirstPass QI system so that PCRs from every non-transport would be checked for compliance with their guidelines within seconds of submission of the report. The automated system checked each report using these criteria:

>>Was the patient’s final Glasgow coma score 15?

>>Were two sets of vital signs recorded?

>>Was a chief complaint documented?

>>Was the patient’s history, medications, and allergies documented?

>>Was a witness signature obtained?

>>Was the narrative greater than 150 characters?

>>Was their “decisional capacity” described in the narrative?

The 150-character test was designed to address the providers’ tendency to write a three-word narrative as the only description on calls where crews had spent significant time with patients. “Decisional capacity” describes the ability of someone to consent to or refuse care.

Paramedics in Pinellas County were also trained to conduct a comprehensive EMS Cognitive Evaluation that could be used to help determine if a patient has the decisional capacity to refuse transport.

Reviewing Past Performance

At the beginning of the improvement project, Pinellas County performed a baseline analysis of the system’s past performance based on these criteria and found that overall adherence to the guidelines was low, hovering around 10%.

On some individual criterion within the bundle of care—narratives longer than 150 characters, acquiring witness signatures, a final Glasgow score of 15, and documenting chief complaint, history, meds, allergies—performance exceeded 80%.

But field providers were obtaining two sets of vital signs less than 10% of the time and “decisional capacity” was described in the PCR less than 20% of the time.

Analyzing this earlier data allowed the medical director and 19 organizations to focus improvement efforts on the two parts of their refusal bundle that needed the most attention.

The FirstPass system automatically reviews all PCRs for compliance to the appropriate protocols, including refusals.

Figure 1: PCR flagged by QI software

The patient care report in this case was flagged because it was missing a second set of vital signs. You can also see that this paramedic documents two sets of vital signs and at least one Glasgow coma scale 81.48% of the time, while the entire system meets this criterion 86.43% of the time.

The Pinellas County EMS system produces thousands of patient refusal PCRs each month. When one or more of the seven criteria listed previously aren’t met, the report is flagged for review by the appropriate individual within the PCR author’s department—often the EMS supervisor or quality improvement manager, depending on the organization’s policies.

Because the PCRs are reviewed by the software immediately, feedback and coaching can be provided while the call is still fresh in the minds of the EMTs and paramedics.

In addition, the compliance with the entire bundle and with each of the criteria can be measured and tracked at the individual, department and systemwide levels.

Communication & Feedback

Pinellas County also implemented a campaign to ensure that each EMT and paramedic in the system understood the clinical and legal risks associated with patient refusals, how to conduct and evaluate a proper EMS cognitive evaluation, the rationale for obtaining two sets of vital signs, the importance of good documentation and the seven tests included in the refusal bundle of care.

The campaign was conducted via email, Facebook and newsletters, as well as in the classroom. Conversations about patient refusals were built into new employee orientation and field training officer programs.

“We didn’t just turn this system on full throttle overnight,” one EMS captain said. “We rolled it out over 18 months of planning and lots of beta testing. Full implementation felt for some people like we went from 0 to 1,000 mph, but they have been impressed by how much improvement we’ve made so quickly.”

Once the campaign was well underway, the clinical leadership teams began providing feedback to crews whenever a call was flagged in FirstPass. Each department handles the coaching and feedback process in a way that works for them. One department prefers group emails and communication over individual feedback, while another department schedules weekly one-on-one meetings with every medic on its team.

Even though they customize the feedback process, all 19 departments provide feedback quickly and carry the same patient-centered non-punitive message; they also coordinate efforts with the county medical director and follow the same policies to determine when it is necessary to notify Jameson.

Dramatic Results

The results of their efforts to improve care for patients who refuse transport have been dramatic.

Figure 2: Pinellas County EMS compliance with atient refusal bundle of care

Pinellas County EMS saw its compliance with the patient refusal bundle of care improve dramatically after implementing a focused quality improvement program.

“For the longest time, QA was viewed as the, ‘What have you done wrong today club,'” an EMS captain said. “With our change to Just Culture combined with the nearly instantaneous feedback facilitated by FirstPass, we can concentrate on what [we] are doing right. I send out a thank-you letter signed by the chief for every medic that has 100% performance on our clinical protocols. I used to send out 12 to 20 a month, now it’s between 170 and 180 a month.”

Transitioning to a true quality management system, rather than a quality assurance system that only looks for mistakes and slaps providers on the wrist, has also allowed for evaluation not just of adherence to clinical protocols but also whether those protocols are, in fact, the best for the system.

That goes not only for refusals, but for other patient care bundles being continuously assessed in the system, including cardiac arrest, major trauma, acute coronary syndrome and others.

“Once we have the system reliably providing care in alignment with the protocols and documenting it properly, it’s possible to evaluate trends and really see how well the protocol is serving our patients,” Jameson said. “We fine-tuned our chest pain protocol to optimize recognition of STEMI faster and to better manage pain as early as possible.”

By tracking performance over time, building a non-punitive just culture, using near real-time analysis tools, and having a truly patient-centered, collaborative approach, the departments delivering EMS in Pinellas

County have the data to show some of the biggest improvement their system has ever seen—and they’re just getting started.


1. Just/Accountable Culture. (n.d.) Center for Patient Safety. Retrieved Dec. 11, 2017, from