Impact of nonphysician, technology-guided alert level selection on rates of appropriate trauma triage in the United States: a before and after study
Article information
Abstract
Purpose
Overtriage and undertriage rates are critical metrics in trauma, influenced by both trauma team activation (TTA) criteria and compliance with these criteria. Analysis of undertriaged patients at a level I trauma center revealed suboptimal compliance with existing criteria. This study assessed triage patterns after implementing compliance-focused process interventions.
Methods
A physician-driven, free-text alert system was modified to a nonphysician, hospital dispatcher-guided system. The latter employed dropdown menus to maximize compliance with criteria. The preintervention period included patients who presented between May 12, 2020, and December 31, 2020. The postintervention period incorporated patients who presented from May 12, 2021, through December 31, 2021. We evaluated appropriate triage, overtriage, and undertriage using the Standardized Trauma Assessment Tool. Statistical analyses were conducted with an α level of 0.05.
Results
The new system was associated with improved compliance with existing TTA criteria (from 70.3% to 79.3%, P=0.023) and decreased undertriage (from 6.0% to 3.2%, P=0.002) at the expense of increasing overtriage (from 46.6% to 57.4%, P<0.001), ultimately decreasing the appropriate triage rate (from 78.4% to 74.6%, P=0.007).
Conclusions
This study assessed a workflow change designed to improve compliance with TTA criteria. Improved compliance decreased undertriage to below the target threshold of 5%, albeit at the expense of increased overtriage. The decrease in appropriate triage despite compliance improvements suggests that the current criteria at this institution are not adequately tailored to optimally balance the minimization of undertriage and overtriage. This finding underscores the importance of improved compliance in evaluating the efficacy of TTA criteria.
INTRODUCTION
Background
In the United States, trauma injuries exert an extensive toll on the population and the healthcare system [1]. The efficient and effective care of trauma patients relies heavily on consistent and accurate triage prior to their arrival at the trauma center. Unfortunately, for a variety of reasons, some patients are mistriaged, meaning that they did not receive the appropriate trauma team activation (TTA) based on the severity of their injuries. Patients who are undertriaged face an increased risk of mortality and adverse outcomes [2,3], whereas overtriage leads to inefficient use of time and resources and can also contribute to provider dissatisfaction [4,5].
Of the two forms of mistriage, undertriage is considered more detrimental. In fact, the American College of Surgeons Committee on Trauma (ACS-COT) defines optimal rates of undertriage as less than 5% and overtriage as less than 35% [6]. However, hospitals worldwide find it challenging to meet these standards, with some reporting undertriage rates nearing 30% and others noting overtriage rates as high as 71% [2,7–9]. Regular assessments of appropriate triage and mistriage rates are conducted by trauma centers. The Standardized Triage Assessment Tool (STAT) is a commonly used method for evaluating triage patterns [10–12]. With the STAT, each patient is assigned a triage designation based on a combination of Injury Severity Scores (ISSs) and the requirement for specific trauma interventions (Fig. 1).
Workflow of the Standardized Triage Assessment Tool (STAT), used to determine triage rates. The STAT is a combination of the Cribari Matrix Method (CMM) and the Need for Trauma Intervention (NFTI) tool. AT, appropriate triage; OT, overtriage; UT, undertriage; ISS, Injury Severity Score.
Designing the ideal trauma triage system remains an elusive goal. Much of the previous research has concentrated on identifying patient populations that are often misclassified, with the aim of adjusting the TTA criteria to better serve these patients [13–15]. More recently, research has shifted towards evaluating and improving compliance with triage criteria [3,16,17]. However, no consensus yet exists on the best way to minimize rates of mistriage. Given this lack of clarity and the fact that trauma impacts a large proportion of the population, it is crucial to explore potential solutions.
Compliance with a specific institution’s clinical guidelines for TTA is shaped by a complex interplay of numerous factors. These include the quantity and complexity of the guidelines [18,19], the ease of referencing criteria within the workspace [20], the number of staff members involved in implementing the criteria [21], staff education about the criteria [22], and feedback loops for staff regarding clinical performance [20,23]. The influence of human factors, educational infrastructure, and the institutional team culture cannot be overstated. Furthermore, the behavioral tendency of physicians to disregard protocols to avoid so-called cookbook medicine can also affect compliance [24]. Thus, efforts to improve compliance with TTA criteria must include assessment of each of these elements and more.
Objectives
This study explores the execution of a compliance-focused intervention at a level I trauma center. A physician-oriented free-text alert system was transformed into a nonphysician, hospital dispatcher–driven dropdown menu alert system through a series of workflow interventions (Fig. 2). The first objective of the study was to evaluate the compliance rates with TTA criteria before and after the intervention. The second objective was to examine changes in triage patterns preintervention and postintervention. The primary outcome measured was the rate of appropriate triage determined using the STAT. Secondary outcomes included overtriage and undertriage rates, as well as clinical outcomes. We hypothesized that appropriate triage rates would rise in correlation with improved compliance with the existing TTA criteria. Furthermore, we anticipated a decrease in undertriage and overtriage rates in relation to increased compliance with these criteria.
Preintervention and postintervention processes for designating trauma alert levels. Hospital dispatchers are first responders or emergency nurses. (A) Preintervention physician-driven, free-text entry alert system. (B) Postintervention hospital dispatcher–driven, dropdown menu alert system. The hexagon represents the process intervention. EMS, emergency medical services; CPR, cardiopulmonary resuscitation; GCS, Glasgow Coma Scale; SBP, systolic blood pressure; HR, heart rate.
METHODS
Ethics statement
This study was approved by the Institutional Review Board of Carilion Clinic (No. 21-1320). The requirement for informed consent was waived due to the retrospective nature of the study.
Study design and population
This before-and-after study involved a retrospective review of trauma patients who presented at a level I trauma center between May 2020 and December 2021. The data selected for extraction were guided by the current TTA guidelines at the trauma center (Fig. 3), as well as previous analyses identifying mistriaged populations and those examining non-compliance with triage guidelines [3,14,15,17]. Direct admissions, patients without TTA, and patients lacking a recorded date of arrival were excluded from the analysis (Fig. 4). We assessed patient demographics, injury patterns, prehospital vital signs, and outcomes to demonstrate equivalence between groups. All data referenced in this study were sourced from the hospital trauma registry and electronic medical records.
Trauma team activation criteria for this institution. CPR, cardiopulmonary resuscitation; GCS, Glasgow Coma Scale; LOC, loss of consciousness; TBSA, total body surface area; TBI, traumatic brain injury.
Inclusion and exclusion criteria for this study.
Process intervention
Two workflow changes were implemented to transform a physician-driven, free-text alert system into a nonphysician, hospital dispatcher–driven alert system. The latter system utilizes dropdown menus to ensure maximum compliance (Fig. 2). The first change involved altering who was responsible for assigning the TTA level. In the original system, first responders arriving on the scene would relay a patient report to the attending physician in the emergency department. This physician would then assign the TTA level for the incoming patient. In the revised system, first responders provide their report to trained hospital dispatchers, who may be registered nurses, paramedics, or emergency medical technicians. These dispatchers are then responsible for assigning the TTA level. The attending physician is only contacted if the dispatcher has a question or concern. The second major workflow intervention was the introduction of a dropdown menu within the computer-based interface used for TTA alerts. Previously, the reason for TTA was entered into a free-text box. In the updated system, hospital dispatchers select the TTA indication from a dropdown menu. This provides just-in-time reminders of the TTA criteria and encourages adherence to these criteria. These changes were implemented sequentially, with the first change taking effect on March 10, 2021, and the second on May 12, 2021. We defined the preintervention period as May 12, 2020, through December 31, 2020, and the postintervention period as May 12, 2021, through December 31, 2021. We chose these time periods so that the months would align between cohorts, considering the seasonal variations in trauma patient presentations.
Compliance
We measured compliance with trauma triage guidelines by examining trauma patients who met at least one objective predetermined prehospital criterion for full TTA, as recorded by emergency medical services (EMS). For our trauma center, these criteria included a heart rate of ≤60 or ≥120 beats/min, a systolic blood pressure of <100 mmHg, a respiratory rate of <10 or >29 breaths/min, and a Glasgow Coma Scale score of 8 or lower. We classified each patient as either compliant or noncompliant, based on whether the correct TTA was initiated before the patient arrived. If a patient’s TTA level was upgraded or downgraded during transport, we grouped them with their final classification.
Defining appropriate triage and mistriage
All trauma patients were evaluated using the STAT [10–12]. This tool is a combination of the widely used Cribari Matrix Method (CMM) and the Need for Trauma Intervention (NFTI) method (Fig. 1). With the CMM, patients are evaluated based on their ISSs, a measure that reflects both the number and severity of injuries across different body regions. According to the CMM, any patient with an ISS greater than 15 should be given the highest level of TTA, hereafter referred to as a full alert. In the NFTI method, in contrast, patients are classified based on their need for specific emergency hospital interventions. Patients who require these interventions (NFTI+ patients) should also be assigned a full alert. NFTI criteria include receiving a blood transfusion within 4 hours of arrival, being discharged to the operating room within 90 minutes of arrival, being discharged to interventional radiology, being discharged to the intensive care unit, having an intensive care unit stay of at least 3 days, requiring mechanical ventilation during the first 3 days (excluding anesthesia), or dying within 60 hours of arrival [10]. Consequently, at the institution in the present study, undertriaged patients were defined as those with an ISS greater than 15 and a positive NFTI designation who were not assigned a full alert. Conversely, overtriaged patients were defined as those assigned a full alert despite having an ISS less than 15 and a negative NFTI designation.
Statistical analysis
The Fisher exact test was used for all categorical variables, while the Welch t-test was employed for continuous variables, with an α value of 0.05. All statistical analysis was conducted using R ver. 4.1.3 (R Foundation for Statistical Computing).
RESULTS
Patient characteristics
A total of 4,953 patients presented to our trauma center during the study period. After application of the exclusion criteria, the number was reduced to 3,693. These patients were divided into two groups, with 1,647 patients in the preintervention group and 2,046 in the postintervention group. We characterized patient demographics, injury patterns, and physiological parameters both before and after the process interventions (Table 1). Generally, the characteristics of patients in both groups were similar, with the exception of age and average ISS.
Patient characteristics in the preintervention and postintervention groups (n=3,693)
Compliance
The overall compliance rates with the objective vital sign criteria for a full alert significantly improved, rising from 70.3% in the preintervention group to 79.3% in the postintervention group (P=0.023). When examining compliance with individual objective vital sign criteria, we noted a significant increase in compliance for bradycardia, from 44.8% to 79.2% (P=0.005). Trends also indicated improved compliance in all categories except for tachypnea (Fig. 5). Furthermore, we observed a trend suggesting a decrease in trauma consultations that met the isolated objective vital sign criteria for a full alert, falling from 4.5% to 3.3% (P=0.274) (Table 2).
Compliance rates among trauma patients with objective, isolated trauma team activation (TTA) criteria in the preintervention (May 12, 2020–December 31, 2020) and postintervention (May 12, 2021–December 31, 2021) groups. The Fisher exact test was used. Includes all three TTA tiers (full, partial, and consult). EMS, emergency medical services; SBP, systolic blood pressure; HR, heart rate; RR, respiratory rate; GCS, Glasgow Coma Scale.
Assessment of trauma consults in the preintervention and postintervention groups
Triage patterns
Rates of appropriate triage decreased in association with the process intervention (Fig. 6). Following the implementation of this intervention, undertriage rates dropped by almost 50% (from 6.0% to 3.2%, P=0.002), crossing an important threshold to reach the optimal undertriage rate as outlined by the ACS-COT guidelines (<5%) [6]. However, rates of overtriage increased from 46.6% to 57.4% (P<0.001). This substantial increase in overtriage ultimately led to a decrease in the rate of appropriate triage, from 78.4% to 74.6% (P=0.007). Alongside the increased rates of overtriage, we noted a decrease in the number of consults (the lowest level of TTA), which was accompanied by an increase in partial and full alerts (Table 3).
Appropriate triage, overtriage, and undertriage in the preintervention (May 12, 2020–December 31, 2020) and postintervention (May 12, 2021–December 31, 2021) groups. The American College of Surgeons Committee on Trauma recommends upper limits for overtriage and undertriage of 35% and 5%, respectively. The Fisher exact test was used.
Alert types in the preintervention and postintervention groups (n=3,693)
Clinical outcomes
Virtually no significant differences in clinical outcomes were observed between the cohorts (Table 4). The exception was an increase in the average length of stay in the emergency department in the postintervention group.
Clinical outcomes in the preintervention and postintervention groups (n=3,693)
DISCUSSION
This study examined the relationship between a new compliance-focused process intervention and the rates of appropriate triage at a level I trauma center. We observed significant improvements in compliance with TTA criteria and undertriage rate. The decrease in the undertriage rate was especially noteworthy, as it brought this metric to an optimal level according to ACS-COT guidelines [6]. However, the rate of appropriate triage ultimately fell due to a substantial increase in the overtriage rate. Despite these unanticipated results, this study offers valuable insights into an easy-to-implement workflow modification that can improve compliance with TTA criteria. Furthermore, the dramatic shifts in triage rates underscore the importance of improving compliance as a crucial step in accurately evaluating an institution’s TTA criteria and guiding future adjustments.
The workflow modifications in this study were designed to increase compliance by simplifying the identification of criteria for a TTA, as well as by reducing the number of designated staff members responsible for assigning the TTA level (from 56 attending physicians preintervention to 10 hospital dispatchers postintervention). Before the intervention, TTA criteria were displayed on the badges of attending physicians and on the desks of hospital dispatchers. However, presenting the criteria in a dropdown menu format offers consistent visual repetition, which enhances memory and retention of the criteria. Previous research has shown that compliance with guidelines in various clinical settings can be improved by making the guidelines more accessible, simplifying them, and optimizing team efficiency and education [18–22]. Therefore, it is likely that both the dropdown menu modification and the dispatcher–driven system contributed to the overall improvement in compliance.
The present study also provides the valuable insight that doctors may not need to shoulder the responsibility of assigning each TTA level. The 2022 Physician Burnout and Depression Report highlights that emergency department physicians reported the highest burnout rates among specialties, at 60%, a rise from 43% in the previous year [25]. Given this escalating burnout rate, the importance of eliminating an unnecessary task for emergency department physicians is paramount, especially as hospitals nationwide continue to deal with the burden of the COVID-19 pandemic.
Interestingly, only compliance with bradycardia showed significant improvement in isolation despite a significant improvement in overall compliance. Additionally, compliance with some criteria (bradypnea, tachypnea) remained low, even in the postintervention period. Bradypnea and tachypnea had the smallest postintervention sample sizes across all isolated, objective criteria, with only four and 20 patients, respectively. Additionally, the dropdown menu was limited to 10 options, including “other,” which were selected based on the frequency of patients presenting to this institution with the alert criteria. Respiratory rates were not included as a criterion on the dropdown menu due to their low frequency, although dispatchers were trained to identify these criteria and could trigger a corresponding alert via the “other” category. Consequently, it is plausible that dispatchers were less likely to identify these criteria as being met, given their lower frequency and absence from the dropdown menus.
Another important observation is that despite a significant increase in overall compliance, the rate of compliance with isolated, objective criteria reached only 79.3%. One possible explanation is that patients who present with a traumatic injury mechanism but meet no other criteria aside from an abnormal vital sign, may seem less severely injured and more susceptible to a downgraded alert. Furthermore, the actual overall compliance rate at this institution for all patients could potentially be higher, as many patients fulfill multiple criteria and are therefore more likely to trigger TTA.
Previous research has pinpointed TTA criteria and compliance as potential areas for improvement in order to increase appropriate triage rates and decrease mistriage rates. However, changes in compliance with TTA criteria are seldom reported alongside mistriage rates. Tignanelli and colleagues [3] investigated these factors together and discovered that improved adherence to the ACS-COT minimum criteria led to a decrease in undertriage and an increase in overtriage. This study, however, analyzed only the six mandatory criteria. Many institutions heavily augment these criteria to better cater to their specific patient populations. Alternatively, many studies have individually tackled the question of which criteria, if any, should be added to the ACS-COT minimum criteria. The overarching goal of these studies is often to reduce undertriage. For instance, both Benjamin et al. [13] and Bardes et al. [26] advocated for the inclusion of age as a TTA criterion, while Schellenberg et al. [27] proposed a higher cutoff for the Glasgow Coma Scale TTA criteria in patients with a head injury. However, these studies had a limitation: each trauma center caters to a unique patient population, which may not all benefit equally from such changes in criteria.
Other studies have focused on enhancing compliance in order to improve the accuracy of triage rates. Stonko et al. [16] suggested a question-based system to boost adherence to existing alert criteria. In their single-center study involving 520 patients, the trauma activation protocol was modified from a PDF-based flow chart to an automated web tool. This tool guided dispatchers through a series of questions based on prehospital EMS data and automatically assigned a TTA level, resulting in a reduction of mistriage rates by over 50%. Notably, in that study, mistriage was defined as incorrect TTA leveling based on EMS data, a definition resembling that of non-compliance in the present study. While the intervention of Stonko et al. [16] shares mechanistic similarities with the workflow interventions discussed in our study, our research examines both compliance and triage patterns within a larger sample size and over a longer period.
In addition to triage patterns and compliance rates, clinical outcomes are a common metric among studies seeking to optimize trauma triage. Tignanelli et al. [3] described an association between increased compliance with the ACS-COT minimum criteria and decreased mortality. However, in the present study, no significant decrease in mortality was observed in the postintervention group. Moreover, we found an increase in the average length of stay in the emergency department for the postintervention group. This finding could potentially be attributed to the burden placed on this hospital by COVID-19. Throughout much of the latter half of 2021, the trauma center, the emergency department, and the hospital as a whole were functioning at an unusually high capacity.
At our institution, we have identified patterns of patients who are frequently overtriaged, and we have adjusted the TTA criteria to reflect these patterns. We are currently analyzing the impact of these changes. While undertriage is generally viewed as less desirable than overtriage, it is well-known that overtriage is linked to higher costs [28]. As a result of the observed changes in triage patterns, we anticipated and observed an increase in the percentage of full alerts (Table 3). In 2021, the cost of a full alert at this institution was $8,605. Therefore, the 10.8% increase in overtriage rates in the postintervention group would have led to an estimated additional healthcare cost of approximately $1,901,430 over 7.5 months. By nature, triage involves a balancing act between overtriage and undertriage. Each institution must evaluate and establish its own ideal balance to effectively cater to its unique patient population. In the end, we expect that a dual approach of improved compliance and adjusted alert criteria will help achieve optimal levels of appropriate triage and mistriage.
Limitations
This study had several limitations. First, we lacked a reliable method to determine which patients had their TTA level assigned through physician discretion within the trauma database or electronic health record. Physician discretion is a mandatory component of the minimum criteria for a full TTA, as per the ACS-COT minimum criteria [6]. In the newly implemented TTA system, hospital dispatchers contact a physician in cases of uncertainty about an incoming trauma patient. The physician can then manually assign the alert level. Likewise, a physician can upgrade or downgrade an alert after activation based on their clinical judgment. Future research may focus on developing a reliable method to examine the frequency of TTA due to physician discretion and evaluate its impact on triage patterns. Second, this study did not analyze nonobjective criteria, so we did not assess compliance with other criteria such as penetrating trauma proximal to the knee or mangled extremity. Given the complexity and subjective nature of many of these criteria, it was not feasible to assess compliance retrospectively within the trauma registry. Third, the data presented here were recorded during the COVID-19 pandemic, with potential implications on the study population or the functioning of hospital processes. Fourth, the decision to activate a trauma team was based on information from the prehospital EMS environment, as is standard practice at trauma centers. Currently, no measure is available of how accurately this data corresponds to a patient’s clinical presentation upon arrival. However, barring EMS equipment failure, the objective alert criteria used in this study should have provided an accurate representation of the patient’s clinical condition at the time of the alert. Finally, all of the data were collected retrospectively, making the analysis potentially subject to errors in data entry or confounding variables.
Conclusions
The implementation of a technology-based alert system, driven by hospital dispatchers, was linked to a significant increase in compliance with existing TTA criteria. However, it also led to an unexpected decrease in appropriate triage due to a substantial rise in overtriage. Interestingly, the increased compliance with TTA criteria was associated with a decrease in undertriage rates, shifting our system from exceeding to falling below the target for optimal undertriage rates. The decline in appropriate triage, despite improvements in compliance, suggests that the current TTA criteria at this institution may not be sufficiently tailored to our patient population to achieve an optimal balance between minimizing both undertriage and overtriage. This observation underscores the importance of improved compliance as a crucial step in assessing the effectiveness of existing TTA criteria and will inform revisions to the current TTA criteria at this institution. Furthermore, future research may explore the relationship between compliance with TTA criteria and the provision of regular feedback to hospital dispatchers about their performance in assigning TTA levels using objective criteria.
Notes
Author contributions
Conceptualization: all authors; Data curation: PAB; Formal analysis: MEH, ANT; Investigation: all authors; Methodology: all authors; Project administration: all authors; Supervision: JRG; Validation: PAB; Visualization: MEH; Writing–original draft: MEH, JRG; Writing–review & editing: all authors. All authors read and approved the final manuscript.
Conflicts of interest
The authors have no conflicts of interest to declare.
Funding
The authors did not receive any financial support for this study.
Data availability
Data of this study are available from the corresponding author upon reasonable request.
Additional information
The preliminary data from this project was presented at the 2022 American College of Surgeons Annual Meeting, Virginia Chapter on April 29-30, 2022, at the Kingsmill Resort in Williamsburg, Virginia.
Acknowledgements
The authors thank Cara Spivey, MS (Virginia Tech Carilion School of Medicine; Carilion Clinic Research and Development, Roanoke, VA, USA), for her support on this project.