When Hospital Software Is Meant to Warn of Sepsis
The Epic Sepsis Model uses patient data to estimate who might develop sepsis. A retrospective study at one hospital found many missed cases and few eventual sepsis cases among the hospital stays it would have flagged. This article examines the implications for clinical work without treating those findings as evidence of an effect on care.

The Epic Sepsis Model is intended to draw clinicians’ attention to patients who might develop sepsis. It calculates a risk score from electronic health records every fifteen minutes. At Michigan Medicine, an operations committee had selected 6 as the threshold for notifying clinicians. A 2021 external study calculated retrospectively which hospitalizations the then-current model would have flagged at that threshold.1
Of 38,455 hospitalizations, 6,971 would have been flagged, or 18 percent. Only 12 percent of those later met the study’s sepsis definition, while 67 percent of later sepsis cases remained below the threshold. The study observed neither real alerts nor clinical responses.
These results allow an estimate of the cases that would need review and those the warning system would miss. They leave open whether alerts improve care. The article therefore examines how a hospital needs to coordinate thresholds, responsibilities, and the time available for clinical review.
What the numbers say about the review queue
The then-current model was a proprietary penalized logistic regression embedded in the electronic health record. The study covered 27,697 adults and 38,455 hospitalizations; its composite definition identified sepsis in 2,552 of them. An AUC of 0.63 (95 percent confidence interval, 0.62–0.64) was a precise estimate of weak discrimination between hospitalizations with and without the later outcome.
The threshold made the work implications concrete. Of 100 hospitalizations with sepsis, 33 would have been flagged; of 100 without it, 83 would have stayed below the line. Roughly eight flagged hospitalizations would have required review for one eventual sepsis case. These measures cannot be collapsed into one “accuracy” figure: they describe both a demand on scarce clinical attention and a large group of unflagged cases.
Sixty percent of the sepsis hospitalizations missed by the model still received timely antibiotics, so many reached treatment without a model flag. That figure alone does not show whether or how sepsis was recognized, but it does show why an evaluation must include the clinical paths below the threshold.
How a score became a page
The Epic Sepsis Model recalculated a patient’s risk score every fifteen minutes. An operations committee chose 6, within the developer’s recommended range, as the threshold. At 6 or above, the system would page a clinician.
The threshold first selected which hospitalizations received additional attention. The interface then shaped how the clinician encountered the score: it could present the number as one observation among many or make it appear to be an instruction to examine this patient first. The validation study did not observe how clinicians read the display, so it provides no evidence that they deferred to it.
The page added a concrete action. It interrupted someone and requested a review, although the clinician still decided whether any intervention was warranted. Review consumes time even when it ends with no treatment. The operational effect therefore began before the treatment decision.
Governance determines whether that sequence can be changed. Someone needs the authority and monitoring data to move the cutoff, suspend alerts, or restore a non-model process when local evidence no longer supports the deployment. Merely naming an owner does not create that capacity.
What a live pathway still has to answer
Hospitalizations exposed to possible real ESM alerts were excluded from the analyzed cohort to reduce bias. The paper therefore cannot establish effects on treatment or mortality, and it provides no evidence of automation bias. Its authors also note the limits of one academic center and a composite sepsis definition that remained contested. Later model versions and deployments elsewhere require their own evidence.
The same boundary applies to alert fatigue. The calculations indicate a large potential workload; they do not show that staff in the excluded live units actually became fatigued. With a positive predictive value of 12 percent, the design must account for review capacity. Human oversight is meaningful only when clinicians have enough time and information to assess the case.
Wong and colleagues do not describe whether Michigan Medicine could suspend the alerts or how missed and unnecessary alerts informed governance. Evidence of benefit would need to compare the full clinical pathway with usual practice, measure outcomes on both sides of the threshold, and retain another way to identify patients whose scores remain below 6. Until such evidence exists, the retrospective score performance and the live clinical intervention remain different objects of evaluation.
Footnotes
-
Andrew Wong et al., “External Validation of a Widely Implemented Proprietary Sepsis Prediction Model in Hospitalized Patients”, JAMA Internal Medicine 181(8), 2021, pp. 1065–1070, corrected 2 August 2021. This was a retrospective external cohort validation at one academic health system, covering December 2018 through October 2019. It evaluated the then-current model and hypothetical alerting strategies, not the causal effects of a live rollout. ↩

