Eight Years Ago, Federal Government Experts Noted that Vaccine Tracking Systems Couldn’t Share Information

Capt. Lana Clouser, from Minot, N.D., records patient information prior to surgery in an operating room aboard the hospital ship USNS Mercy. (U.S. Navy photo by Mass Communication Specialist 2nd Class Ryan M. Breeden)

In the early 1990s, John Grisham wrote The Pelican Brief, a novel about government knowledge to which few paid attention until it was too late; and those in the know went to extraordinary lengths to forestall critical changes. 

Two decades later, facing 400,000-ish deaths (as of this writing), America is struggling to wrap its arms around efforts to track COVID-19 administration data — to sift real from fantasy, medicine from snake oil. To figure out where the virus is and how it is changing. To identify who needs a second shot but has yet to get it (or lacks access to it). Even where vaccine supplies need to be shipped and where they are failing to reach their destination. 

More from the Author

Instead, our multifaceted vaccine delivery machinery is groaning: local jurisdictions are relying on white pieces of cardstock that just beg to be lost amid so many papers in the kitchen junk drawer. Meanwhile, clinicians in states that were slammed by COVID-19, like Texas and California, are noticing — to the collective dismay of political leaders and bureaucrats whose jobs focus on public health and interoperability — that years of cannibalizing the public health budget, even letting consultants and contractors run the show, have yielded obsolete database manuals (that are still downloadable from official state websites), confusion among emergency responders and public health professionals, and supply chain holes so gaping that good money is being dumped after bad, as databases are unceremoniously ripped out and replaced spontaneously with new “solutions.” Even legislators’ offices have confessed they cannot get answers from their own state executive agencies. 

Want to be really frustrated? Forget Hollywood horrors like Contagion and I Am Legend. Like the West Wing episode where the White House realizes that no one is reading the studies it has spent millions commissioning, the federal government’s compliance leads flagged these issues eight years ago. 

The public and those who care for it are frustrated by the lack of consistent public health messaging, even about explanations for the rise and fall in COVID-19 case counts, that the feeling of mystery and arbitrary regulation has led to a strengthening cacophony pushing for the recall of California Governor Gavin Newsom. Want to know why COVID-19 infection data are so spurious, and care providers so furious? Even healthcare providers cannot engage America’s immunization databases:

  1. The Texas Governor’s EMS and Trauma Advisory Council (GETAC) spent a substantial part of its February 2021 meeting discussing ways to share data with the state’s immunization registry — even as some agencies have resorted to hiring hospitals’ electronic health record scribes to manually enter thousands of vaccination records into a state system that resists seamless uploads. 
  2. One of California’s largest fire-based EMS agencies has “hired an army of secretaries” to recopy paper-based white paper forms into the state public health data, even after CalRedie was deemed unreliable, corrupted, and simply “broken,” according to the Los Angeles Times. The cost to rip-and-replace the broken database cost California taxpayers $15.3 million — but a new solution that was put in place so fast, following discovery of the system failure, that it is inconceivable how sufficient time was allotted to run a fair competitive bid and test the supposed replacement.
  3. Similarly at the federal level, in February 2021, the New York Times reported that “the Trump administration awarded the first of two no-bid contracts worth up to $44 million to a national consulting firm to help patients register to be immunized and states collect detailed data on vaccine recipients.”
  4. In January 2021, a separate problem impacted the “vaccine management tool that California has used to coordinate waitlists and inventory as well as send email proof of vaccinations to patients.” 
The above two screenshots from the Los Angeles Times shows reporting on how California was hit by software problems in its vaccine rollout.

Efforts to contact the California Department of Health & Human Services (CDHHS), in an effort to supply the department with missing information generated by one of the state’s largest public health and safety agencies, were referred to Michele Cunningham, Special Assistant to the Agency Secretary and Undersecretary for the CDHHS. On September 3, 2020, Ms. Cunningham wrote: “I am in receipt of your request and, I appreciate your patience in my response time regarding your inquiry.” Follow-up inquires received no reply.


Registry is the technical name given to database that contains personally identifiable information, especially regarding matters like health. Their purpose is to aggregate data so as to (for instance) inform patients and care providers when someone who received a first COVID-19 vaccine failed to show up for a second. Data being collected in municipalities, hospitals, and pharmacies nationwide should be ingestible by registries in order to ensure that our country — and at large scale, our world — does not watch our public health efforts crumble in the face of a massive GIGO problem (or “Garbage In, Garbage Out,” the bane of syndromic surveillance and accurate statistics). 

After all, failure to follow-up is how outbreaks flare up, according to Kayleigh Rogers of FiveThirtyEight. But the reality is not so simple.

“How do you make sure everyone is coming back for their second shot? Our ability to stop the spread of COVID-19 depends on the answer. If people only get one dose, they will not be fully protected, but may behave as if they are. And even worse, there’s a fear that if enough people get only the first injection, the virus could develop resistance to the vaccine.

Simple strategies like booking follow-up appointments at the time of the first dose, or sending reminders via text, phone or email, have been found to be effective in increasing the likelihood that patients complete the HPV vaccine course, and may be useful tools for making sure patients get both doses of the COVID-19 vaccine…Given these are new vaccines, and the country is facing one of the biggest widespread public immunization efforts in half a century, there are a lot of unknowns. This will be a massive undertaking, but the stakes couldn’t be higher.”

Unlike HPV, anyone infected with COVID-19 can spark a full-blown outbreak, so how do you keep a rising number of homeless people living in makeshift camps from dying en masse? How do you contact people without phones or internet connections, or who are afraid to provide real contacts for fear of government intrusion?

Bloomberg and others threw laurels at Apple and Google when they announced a collaboration around contact tracing. It was an effort destined to fail, as both this author and VentureBeat called at the time. It did succeed at adding to the noise and mismanaged expectations of what is currently possible within the bounds of consumer technology and privacy laws.

Ever wonder why, even as Elon Musk and Richard Branson take civilians into space, we are still documenting immunizations on white paper cards that are likely to be lost or torn and come with a nefarious bonus gift: identity theft?

From the West Coast to New England, widespread guidance from public health agencies seeks to submit data to the Center for Disease Control & Prevention’s Vaccine Administration Management System (VAMS), “an optional, web-based application…for temporary, mobile, or satellite COVID-19 vaccination clinics…that do not have existing IT systems for vaccination clinic management.” One public health executive in Vermont said, “I think we are going with a system that is compatible with the federal system.” 

But doing so is a manual process, with little help from software-based automation. The New York Times says VAMS has been “spurned by most states and become an object of scorn.” Little wonder why: a physician at a large hospital in the Southeastern U.S. attempted to upload into VAMS patient data that meets the system’s technical requirements. But the feeder into the federal CDC’s central immunization database does not even have a field in which to enter the vaccine’s name or lot number. (Figure 1) How does one track vaccinations without capturing the name of the vaccination that was used? 

Figure 1

South Carolina’s vaccination registry is even more challenged: while the Department of Health and Environmental Control (DHEC) says its vendor meets federal “Meaningful Use” (MU) Stage 3 rules (Figure 2)— that is, it complies with standards governing the “meaningful” implementation of clinical documentation software — there is no means by which to upload data to the state vaccination database. At all. Every vaccination record must be entered by hand, recopied one-by-one from the white paper card — and South Carolina uses the same system as eleven other states and territories (Figure 3): Arkansas, Connecticut, Delaware, Guam, Kansas, Kentucky, Missouri, Philadelphia, Nevada, New Mexico, and the Mariana Islands. The published literature references “HL7 messages” and compliance with MU Stage 3.

They say the Devil is in the details. Each of the states listed above, and more, says their vendor uses an immunization system leveraging HL7 version 2.5.1. Meaningful Use Stage 1, however, introduced a data sharing model based on HL7 version 3. MU State 1 hearkens back to the American Reinvestment & Recovery Act (ARRA) passed in response to the 2009 Great Recession. It built massive electronic health record companies like Epic, Cerner, and Allscripts. (Figure 5)

Figure 2
Figure 3
Figure 4
Figure 5

While the federal government is muddling through no-bid contracts, why are the states using software that was supposedly evolved to a modern iteration over a decade ago? (Figure 4) Which version is correct?

Turns out that they both are. 


America’s federal government uses *at least* three data sharing standards that conflict with one another:

  1. HL7 v3 for personal health-related data; 
  2. HL7 v2.X for immunization and other sorts of data;
  3. NEMSIS for EMS data; and
  4. Fire-based EMS agencies face a fourth non-interoperable data system (NFIRS)

(When a Texas EMS agency recently told GETAC that its patient records vendor could export data using “HL7,” it confused these two formats: hospitals share data using HL7v3 documents, but the state registry requires HL7v2. Failure to understand the difference can mean paying for an export model that ultimately cannot send data in the format required by the destination database.)

In December 2020, the Congress of Mobile Medical Professionals (CoMMP) published a guidance document that asked , among many other question, “How will Mobile Medical agencies be able to identify patients who have already had one vaccine, and require the second dose?” An executive at a national EMS trade associations responded simply: “Ask Tiberius.”

According to the Wall Street Journal: Tiberius is a product of “data-mining company Palantir Technologies Inc. [which is] is helping the federal government set up a system that will track the manufacture, distribution and administration of COVID-19 vaccines.” WSJ wrote: “Federal health officials would use to manage the various vaccine data and identify any issues that could prevent Americans from getting the shots…an attempt to use cutting-edge data science to help the federal government manage the work of protecting Americans against COVID-19.” 

By January 17, 2021, however, the same EMS trade association executive that previously suggested relying on Tiberius for answers now said “the Palantir tracker has not lived up to its expectation.” An executive at Palantir told me: “We are not actually the data broker or supplier for Tiberius.” Who is ensuring that the systems our tax dollars are paying for do what they say they will do?

Poor data upload structure is not as especially unusual, especially among government systems — some nuclear silos are still powered by 5.25″ floppies. Billions of tax dollars were invested in building vaccination and other clinical registries, as well as the data sharing standards that underpin electronic health record systems. How would the public feel if it turned out that all of the current challenges faced by mobile medical and public health agencies could have been prevented…if only the government had listed to its own experts?

Concerns about holes in the data sharing process, however, published by the federal Department of Commerce in 2013, were ignored. That year, National Institute of Standards and Technology (NIST) 2020 HL7 Fellow Robert Snelick; and Sheryl Taylor, then of the consulting firm Booz Allen Hamilton, published the following on the NIST website (Figure 6):

“For example, the transmission to immunization registries certification criterion and associated tests assesses whether the EHR product can create HL7 V2 messages in accordance with the requirements given in the referenced standard. “It does not, however, place any requirements on the receiving system, in this case the immunization information systems (IIS) used by the registries.

“In order to achieve interoperability the receiving systems must implement the corresponding requirements. The Meaningful Use program envisions that these systems will, over time, acquire such capabilities due to market forces and the benefits gained (e.g., the public health agencies will want to take advantage of the standardized interfaces).

In short, an award-winning member of America’s technical standards agency said the government incentivized producing data — but not consuming it.

The government basically had mandated the use of red or green ink without ensuring that the reader could see color. Or it required the consumption of pizza without ensuring that the eater had a mouth. NIST said another federal agency had failed to require that systems be capable of using the data that it paid companies billions to build using a framework ironically called “meaningful use.” 

Figure 6

As the pandemic rages and Americans line up to get shots — especially if we are going to have to get COVID-19 vaccinations with some sort of chronic frequency — we should ask why our state and federal governments incentivized software firms to implement a standard data architecture but not to consume the data they created. Why did they wait for “market forces” to kick in if doing so meant the risk of waiting too long — when in-house experts themselves highlighted such a risk?

After spending billions to spur the build technology systems, the government then decided to rely on “market forces” to close the gap? The market forces arrived in the form of a pandemic. We’re not ready. In June 2017, the Office of the Inspector General within the federal Department of Health & Human Services identified that “millions in electronic health record incentives that did not comply with federal requirements” (Figure 7).

Figure 7

Four year later, in January 2021, the office that awarded incentive funding to electronic health record companies— and which oversees both health information exchanges and meaningful use — said it would seek to strengthen interoperability between electronic health records and vaccination registries. 

But a month after these latest round of awards from the Office of the National Coordinator (ONC), an emergency physician at one of the largest hospitals in the Southeastern U.S. wrote:

“Unfortunately, the time of administration, location of administration (right or left deltoid muscle), and lot number have to be entered individually on each patient in the [Vaccine Administration Management System, or VAMS] system. These cannot be mass uploaded from any other worksheet or system. Our CNO [Chief Nursing Officer] has sent a request to [VAMS] to allow automatic upload of this information with the rest of the patient information, but that has not been completed at this point and it is unclear if it will ever be adjusted to accommodate this request. The best option that I can come up with seems to be getting the patients registered and scheduled in VAMS prior to inoculation and having a person at the vaccination clinic document these specific criteria as each patient goes through and gets their shots. Tedious to say the least and a significant waste of time and resources, but I have not been able to come up with any other solution.”

One state immunization registry lead, when asked about uploading data using a standard structured spreadsheet, as published in the state vendor’s instruction manual, said: “I am not too familiar with this method, as I haven’t worked with an organization who has done this before. I will speak to my team Lead for further guidance and follow up with you all.” 

A week later it was revealed that her state had officially removed the technical ability to upload data in the manner that was instructed in the public guide. The state said that the guide book was itself several years old. However, this author saw that the function the state thinks was removed from the registry is still present as of this writing. 

In the midst of a pandemic, agencies are now spending precious time and millions (billions?) more hand-entering data for millions of patients into websites run by government contractors…that are still getting paid. Should we be demanding our money back? The thorny question was raised during a Texas GETAC meeting: What value are taxpayers — whether citizens or public health and safety agencies — actually getting from the investment into multi-million dollar state and federal registries? 

The systems being used today do not even have safeguards to protect against unusable or unreliable data: for example, they allow free-text and arbitrary data entry, like fields titled “any other relevant medical information?” One Texan hospital executive told me about a populous county that was so desperate to finish the Sisyphean task of retyping all its white paper cards that it had outsourced to a data entry team — and now it was finding inaccuracies in gender and race information. Pity the policymakers: How can they make wise decisions on behalf of the rest of us when the systems they rely upon are sieves lacking basic quality controls?

No posts to display