It is commonly accepted that a false alarm is an indication of a built-in or embedded test failing, when in fact no such failure exists. The problem I had with this definition when I wrote “The True Nature of False Alarms” in 2014 was that there really is no way to determine whether the failure existed at the time it was reported. More importantly, what we really want to know is what we can do about such an indication.
My revised definition was that “false alarm (FA) is a call for maintenance action, when none was needed.” This definition could be measured by learning that the actions taken in response to such alarms were a total waste of time and resources as they produced no tangible improvements. Under this definition, we can include insignificant or “nuisance” alarms that may be informative but should not alarm anyone. An indication of the percentage of battery power life remaining may be welcome information, but an alarm going off at 50% power remaining and at each 10% step after that is not so welcome, especially by hospital caretakers who monitor medical monitors.
In a 2012 Biomedical Instrumentation & Technology article entitled, “Monitor Alarm Fatigue: An Integrative Review,” Maria Cvach, assistant director of nursing at Johns Hopkins Hospital in Baltimore, MD, thoroughly studied alarms triggered in medical monitoring equipment and its impact on caretakers. In one study by the Food and Drug Administration (FDA), 566 reports of deaths related to monitoring device alarms occurred during the period of 2005 and 2008. A four-month review of the same database between March 1, 2010 and June 30, 2010 revealed 73 alarm-related deaths with 33 attributed to physiologic monitors. Physiologic monitor alarms are purposefully designed for high sensitivity to not miss a true monitor event. To achieve this high coverage, normal anomalies and marginal conditions where the condition is indeterminate or inconsistent are directed to trigger alarms.
The motivation behind this is that it is better to have cried wolf 10 times than to miss only a single breach. Or is it?
Cvach cites studies that indicate that the presence of false and/or clinically insignificant alarms ranges from 80%–99%: “Frequent alarms are distracting and interfere with clinicians performing critical tasks and may lead to staff disabling alarm systems. Excessive false positive alarms result in caregiver apathy and desensitization such that real events are less likely to be acted upon.”
What happens eventually is that alarm fatigue sets in. Dev Raheja, a reliability engineer, and Maria C. Escano, M.D., a physician, co-authored “Hazards in Patient Monitoring Alarm Systems” in the May-June 2012 issue of the Journal of System Safety, in which they further investigate the phenomenon of alarm fatigue. They
point out that “Alarm fatigue is so common that it makes the ability to distinguish one alarm from another more challenging.” They collected several suggestions on how alarm fatigue can be overcome. All of them seek information beyond just a binary “alarm vs. no alarm” choice. The information they seek would support a “smart alarm” capability. Instead of having the caretaker guess whether an alarm is real or not, circumstantial evidence should be collected and presented to the caretaker as (s)he responds to the alarm or multiple simultaneous alarms. For example, if the patient just returned from surgery his/her monitor readings are more critical than a patient who freely moves around and creates monitor disconnect conditions that nonetheless trigger an alarm.
There are many other factors to consider and it is too much of a burden to force the caretaker to decide on his/her own whether and what action is needed in response to the alarm. In addition to the almost continuous alarm triggering in a hospital with hundreds of monitors, a condition is likely to occur when multiple alarms are sounding simultaneously and a decision needs to be made which monitor to attend to. Making the wrong choice could be fatal to a patient.
This is a serious problem. In 2016, an estimated 251,000 patients died due to “hospital errors” in the United States, making it the third-leading cause of death nationally according to Martin Makary, MD, of Johns Hopkins University, reporting in the BMJ (formerly the British Medical Journal). We have to wonder how many of these were directly or indirectly caused by alarms that were false or just a nuisance, especially considering that there are far more monitors and monitor alarms in a hospital today than there were during the 2010 FDA studies I cited earlier.
In avionics, false alarms can also reach 80% as measured by the No Fault Found (NFF) event when the removed line replaceable unit (LRU) is tested in a repair depot. According to his AUTOTESTCON Keynote address in 2015, Deputy Assistant Secretary of Defense for Maintenance, John Johns, estimated that false alarms and NFFs cost about $20 Billion for the U.S. Department of Defense each year. A similar estimate for commercial avionics places the cost at about $100,000 per aircraft per year. Consumer electronics, smartphones, and automobile electronics can also suffer from these ailments. With the Internet of Things (IoT) approaching 50 billion units by the end of this decade, we will likely all suffer from alarm fatigue (Some of us already do!). Perhaps for electronics, the loss is “just money,” while medical electronics causes the loss of life. Nonetheless, the problems stem from the same source. Alarms are false more often than they are true and yet we cannot ignore them. We need to overcome them by smart systems that can guide us to discriminate between the real and the false indications.
Bayesian analysis, as shown in my 2014 article and in several others since then, can be used to calculate the “valid faulty result” and therefore the “invalid faulty result” (or % of false alarms) as 100% minus valid faulty result. Typically, these tend to yield false alarms that are greater than 70%. Manufacturers typically specify no more than 2% false alarms. This is confusing, until you consider that the manufacturers’ specification is for “fractions of false alarms” and is primarily a measure of signal-to-noise ratio. It does not take into account many other false alarm causes, such as user or operator error, test errors, built-in test errors, environmental conditions, intermittent glitches, software errors, temporary disconnects, and design or tester design errors. These conditions happen much more often than 2% of the time while the product or the patient is doing just fine. That is the reason most alarms are false. Additionally, some alarms should not be alarms at all.
If you want to identify with a hospital caretaker affected by false alarms, consider your home smoke detector. When one detector sounds the alarm, you can probably deal with it, that is, after you can find which smoke detector sounded the alarm and after you have ascertained that it is not reacting to an actual fire or smoke. (In fact, if you wouldn't be busy tracking down the cause of what you consider to be a false alarm, you would be getting out of the house as soon as possible!) If your smoke detector alarm goes off in reaction to the steam emanating from your bathroom after you take a shower, you can also appreciate the “nuisance” alarm as well.
Imagine what you would do if you heard an alarm emanating from your infant's room on one side of the house and at the same time an alarm sounding from the room housing your invalid elder relative on the opposite side of the house. You have seconds in which to make a decision—just as the caretakers in a hospital—and the wrong decision can cost a life. It will be of little comfort to your nerves if you find out that it was "just" a false alarm that produced this event. If this happens often, there may come a time when you will not react at all, and the well-intentioned false alarm saturation will kill someone. This is what happened in a January 2000 fire at Seton Hall University dormitory that came to be known as the Bolland Hall Tragedy. The fire was the proximate cause that killed three students and injured 58 others. Another cause of the deaths and injuries was the fact that “the residence hall went through several years of false fire alarms causing students to ignore the alarms, including the one warning of this fire."
Shouldn’t we be solving the false alarm paradox now? Shouldn’t we equip hospitals with intelligent alarm discriminating equipment that will advise caretakers how to react to the hundreds of alarms they encounter each day? Any suggestions how?
Louis Ungar is the Test Engineering, DFT and Management Consultant/Instructor and President at A.T.E. Solutions Inc. and BestTest Group