Sitting across your desk is the regional representative from Pelagic Systems, who s demonstrating his new cervical spine monitoring device. The representative is a friendly fellow, well-dressed in a blue suit with a Star of Life pin on the lapel.
He reports that his company has developed a device that s highly effective in detecting cervical fractures in the prehospital setting. The device is a mesh that fits around the neck of the injured patient. Within it are electromagnetic and acoustic sensors adjusted to detect the slightest amount of crepitus in the cervical spine. These signals are sent to a series of lights that identify the likelihood of a cervical fracture.
The idea, he explains, came from research on electrical sensors found in sharks. He says, Dr. Matt Rothenstein, who wrote The EMS Bible, is using the device in Seattle with great results. He elaborates, The Tahitian Defense Forces have equipped all of its medical units with the device.
You ask him for supporting research, and he provides a glossy brochure that his company has prepared. It has plenty of graphs, pictures and multiple quotes. He says, The $4,500 unit cost is nothing compared to a lawsuit for missing a spinal fracture and not immobilizing the patient.
EMS personnel love gadgets and technology. Who could argue that detecting cervical spine fractures in the prehospital setting might be good thing? But is it? EMS personnel are facing an increasing number of claims and standards of care that are not based on any valid scientific evidence. So, how can you tell the science from the pseudoscience in EMS? Use my EMS Baloney Detection Kit.
1. If it sounds too good to be true, it probably is. How does the claim fit with the way that we know the world works? Most of us are fairly intelligent people, and we have gut feelings when things don t seem quite right. Always question everything, and take nothing at face value.
2. Is there a reliance on anecdote? An anecdote is a short story of personal experience or opinion; it s subject to various biases and carries no weight. It s not uncommon for a salesperson to report that their device or practice is being used in such-and-such system with tremendous results. Anecdotes and rumor carry the same scientific weight none.
3. Ask for independent confirmation of the facts. Have the claims been evaluated by another source? Proponents of devices and practices will almost always provide some form of research supporting the product. But not all research is the same. Research funded or performed by the company making the device should be suspect. The company is likely to report only the positive findings and not publish the negative findings the so-called confirmation bias. Ask for controlled experiments especially double blind studies in which the person measuring results is not aware of the test and control subjects.
4. Does the preponderance of evidence point to the proponent s conclusion or to a different one? You should always review the evidence for yourself. Does the proponent misrepresent the science (which is common)? Does the proponent ignore or discount negative studies? Scientific studies should be reproducible. If several studies point to the same conclusion, it s most likely correct.
5. Does the proponent of the device support substantive debate on the evidence from knowledgeable people with differing points of view? If the proponents of a product or practice have quality research, they won t object to scientific scrutiny of the product. Beware of people who immediately become defensive when their product or practice is questioned. If they don t have quality research to support their position, they ll quickly circle their wagons and resort to beating plough shares into swords to defend it.
6. Is there an appeal to authority figures? Each profession has authority figures and these people are important. However, many times people will drop such names in reference to proponents or users of the device or practice. Remember, arguments and opinions from authority figures carry little weight. In science, there are no authorities. Beware of organizations and certifications that promote a proprietary product. These are often misrepresented as objective organizations when, in effect, they are not. Some actually have some of the characteristics of cults.
7. Is there a reversal of the burden of proof? Proponents of a practice or device have the responsibility of providing supporting scientific evidence before it s put into practice. Often times, they ll put the device on the market without scientific support. In the history of EMS, such devices and practices as MAST, CISM and System Status Management were widely adopted without any scientific evidence. Later, when studied, these practices failed to stand the test of science. However, when it came time to do away with these unscientific practices, there was (and is) considerable resistance.
8. Is there an appeal to ignorance? Along the same line as reversing the burden of proof, proponents will often appeal to ignorance. For example, a proponent might say, If you can t prove that it doesn t work, it must work. This is nonsense. Remember, the absence of evidence is not the evidence of absence.
9. Are failures rationalized? Problems or failures with a device or practice are often rationalized. Proponents will often downplay failures by pointing out how the people who were involved in the failure did not properly use the device or practice or deviated from some protocol.
10. Does the proponent suggest dire consequences from an unfavorable decision? This is a common tactic. They suggest that you and your system may not be up to the standard of care if you don t adopt their device or practice. Or, they may suggest that you re exposing your organization to possible legal problems or liability if you fail to use their product or practice.
11. Do they resort to ad hominem attacks? The term ad hominem means to the man. Often, proponents will attack those with a different message or opinion in hopes of discrediting the claim by discrediting the claimant.
12. Do they resort to the use of emotive words? Some words in our language are full of emotion and get our attention. Beware of proponents who often use such words as liability, standard of care and litigation. These are used solely for their emotional value in an attempt to shame you into purchasing the product.
I hope you find my Baloney Detection Kit of value. For additional information on science and pseudoscience, I highly recommend the following texts:
- Shermer, Michael: Why People Believe Weird Things. New York, New York: Henry Holt and Company, 2002.
- Sagan, Carl and Ann Druyan: The Demon Haunted World: Science as a Candle in the Dark. New York, New York: Ballantine Books, 1996.