Quality improvement in the ICU: treat first what kills first

A characteristic of professionals is that they possess an internal drive to produce high quality work. This is also true for professionals working in health care like nurses and physicians. They always seek for ways to improve the quality of their work, i.e., patient care. In fact, they automatically strive for continuous quality improvement because of their internal drive being a professional. Though medicine and health care delivery indeed reaches a higher standard year after year, sometimes habits are difficult to let loose and to be replaced by another way of working that appears to produce better results. A lot of research has been performed to understand why it is so difficult to implement new protocols and guidelines with the aim to improve the level of care for a group of patients as a whole. Several theoretical models give insight in the mechanisms behind these implementation barriers (1,2). The next step, after getting acquainted with working with guidelines, was the development of indicators. These indicators can give insight in the extent that a guideline is actually implemented. In addition, indicators can be developed to obtain insight in the quality of care, without a direct link to a guideline. These quality indicators are usually evidence based, which means that solid evidence from well-performed research has made clear that a certain way of treatment is effective. An indicator can give a signal whether or not this evidence is implemented in clinical practice. Usually randomized controlled trials (RCT) or meta-analyses from RCTs are seen as the highest degree of evidence (3). Some research aims to study whether the evidence from RCTs is actually leading to improved outcome when applied in routine daily care. Surprisingly, it appears that not always these studies can confirm what was found in the RCT(s). For instance, the first sepsis guideline published in 1992, was based on several RCTs but in future sepsis research several recommendations could not be confirmed and newer versions of the sepsis guideline now lack these recommendations or the evidence has been downgraded (4,5). The reasons for the gap between RCTs and daily practice are numerous. Here I mention two of the most important reasons. First, patients for RCTs are selected. When we focus on the ICU, usually only a subset of patients admitted to an ICU will be included in a RCT that is performed in that unit. After publication of the results, however, the new or adapted treatment modality is applied to a much broader group than the one that was described in the inclusion criteria of the original study. For instance, many elderly patients will not be recruited for RCTs but will receive the treatment when it is implemented in daily practice. The effect, however, may be smaller or even absent in elderly patients. A second reason is that selected and dedicated doctors perform RCTs. Their dedicated way of delivering the treatment during the trial may cause different effects compared to the performance of their colleagues after broad implementation when probably less focus on this treatment will be given. These two issues play a role when a divergence in the results between the RCT and studies concerning evaluation of daily practice is seen.

For Quality Improvement (QI) studies, the same is going on. In studies concerning QI the implementation of several interventions derived from RCTs is often studied including their effect when applied in daily practice. An example is given in the study performed by the BRICnet in 118 Brazilian ICUs (6). A bundle of interventions was implemented in half of these ICUs and the other half did not implement these interventions using a cluster-randomized design. The interventions were defined and monitored by quality indicators and the implementation was facilitated by the use of daily checklists. All interventions were proven beneficial in previous RCTs. This multifaceted study had three tools (checklists, goal setting, and clinician prompting) to reach adherence to the interventions. With these tools the study team was able to overcome the implementation barriers, which was shown by reasonably good numbers concerning the actual application of the interventions. However, the study did not show significant results in primary outcome (mortality) and most of the secondary outcomes. The six items with significant improvement (use of low tidal volumes, avoidance of heavy sedation, use of central venous catheters, use of urinary catheters, perception of team work, and perception of patient safety climate) could, apparently, not induce a lower mortality rate. In general, a multifaceted approach, which was chosen in this study, is the best way to success but still no guarantee (7,8). Several effective strategies in multifaceted quality improvement programs (9) were not implemented in the study design of Cavalcanti et al. The question is why these six improvements failed to reduce mortality. The use of low tidal volume was defined as 8 mL/kg maximum, which is rather high compared to the 4-6 mL/kg that is currently advised (10). In addition, perceptions of teamwork and safety climate are relatively abstract items when the primary outcome is mortality and may thus not have direct effects. The baseline characteristics of the ICUs show that the control group consisted of more academic hospitals with a greater number of hospital beds and larger ICUs. In addition, they had more medical patients. At first sight these imbalances might have played a role in the outcome but post-hoc analyses did not reveal a significant effect. It also appears that the standardized mortality of these ICUs is 1.4 in the control group and 1.6 in the intervention group. These SMRs are very high compared to European ICUs as the prediction model was built on ICUs with a SMR of 1.0. Apparently, other more critical interventions that the six significant ones in this study are underdeveloped in Brazilian ICUs and these probably explain both the negative study results and the high SMRs. In addition, the issue that I raised before concerning QI studies hampers this study too: a broader inclusion of patients compared to the original RCTs. For example, less than 50% of the patients were mechanically ventilated. The successful reduction of tidal volume and the prevention of ventilator associated pneumonia can be reached in these patients only and cannot effect mortality rates of those without mechanical ventilation. The 50% ventilated patients is a relatively low rate compared to other ICUs. Another flaw where this study is suffering from is the relatively short follow up time. Organizational changes need 1–3 years to obtain maximum results. The 6 months follow up time is too short to obtain measurable effects.

In conclusion, this large QI study failed to reduce mortality due to methodological issues. Some of them are fairly common issues in QI studies such as: (I) a broader inclusion of patients in the study population compared to mono-intervention RCTs; (II) the choice of the intervention leaving some other critical processes unchanged; (III) the QI feedback method is suboptimal as it lacks several proven intervention strategies and (IV) the study had a follow-up time that was too short to obtain measurable effects. In addition, the high baseline mortality rate suggests that the most important reasons for dying in Brazilian ICUs were not part of the study intervention. It was shown before that QI project are most successful when they are specifically targeted to the problems that need to be solved (8,9). As a result, it is understandable that the interventions that were performed in this study did not result in a lower mortality rate. The saying “Treat first what kills first” can be applied to this study. Nevertheless, the study of Cavalcanti and co-workers has learnt us a lot about how to design and perform QI studies: target your interventions to what really matters and use effective strategies.

Acknowledgements

None.

Footnote

Conflicts of Interest: The author has no conflicts of interest to declare.

References

1. Cabana MD, Rand CS, Powe NR, et al. Why don't physicians follow clinical practice guidelines? A framework for improvement. JAMA 1999;282:1458-65. [Crossref] [PubMed]
2. Cane J, O'Connor D, Michie S. Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci 2012;7:37. [Crossref] [PubMed]
3. Sackett DL, Rosenberg WM, Gray JA, et al. Evidence based medicine: what it is and what it isn't. BMJ 1996;312:71-2. [Crossref] [PubMed]
4. Bone RC, Balk RA, Cerra FB, et al. Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine. Chest 1992;101:1644-55. [Crossref] [PubMed]
5. Singer M, Deutschman CS, Seymour CW, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA 2016;315:801-10. [Crossref] [PubMed]
6. Writing Group for the CHECKLIST-ICU Investigators and the Brazilian Research in Intensive Care Network (BRICNet), Cavalcanti AB, Bozza FA, et al. Effect of a Quality Improvement Intervention With Daily Round Checklists, Goal Setting, and Clinician Prompting on Mortality of Critically Ill Patients: A Randomized Clinical Trial. JAMA 2016;315:1480-90. [Crossref] [PubMed]
7. de Vos M, Graafmans W, Kooistra M, et al. Using quality indicators to improve hospital care: a review of the literature. Int J Qual Health Care 2009;21:119-29. [Crossref] [PubMed]
8. van der Veer SN, de Keizer NF, Ravelli AC, et al. Improving quality of care. A systematic review on how medical registries provide information feedback to health care providers. Int J Med Inform 2010;79:305-23. [Crossref] [PubMed]
9. Prior M, Guerin M, Grimmer-Somers K. The effectiveness of clinical guideline implementation strategies--a synthesis of systematic review findings. J Eval Clin Pract 2008;14:888-97. [Crossref] [PubMed]
10. Dellinger RP, Carlet JM, Masur H, et al. Surviving Sepsis Campaign guidelines for management of severe sepsis and septic shock. Crit Care Med 2004;32:858-73. [Crossref] [PubMed]