Skip to main content

Suspected bias in selection criteria of target subpopulation and its validation

A Correspondence to this article was published on 25 August 2023

The Original Article was published on 21 June 2023

Dear Editor,

We read with great interest the recent publication by Osawa et al. regarding heterogeneity in the treatment effects of polymyxin B hemadsorption (PMX-HA) [1]. We consider it to be intriguing and express our respect to the authors for conducting such a meticulous analysis.

However, we have concerns that this study may contain biases. We would like to discuss the two biases of concern.

Survivor treatment selection bias

The authors used data from a retrospective observational study (JSPETIC-DIC study) to analyze the heterogeneity treatment effects of PMX-HA and suggested that prothrombin time–international normalized ratio (PT-INR) > 1.4 or lactate > 3 mmol/L could serve as criteria for a “targeted subpopulation” that may have greater treatment efficacy with PMX-HA. However, the analytical approach of the authors may give rise to survivor treatment selection bias, and consequently, these criteria may not correctly identify such patients.

Survivor treatment selection bias is where the effect of an intervention is overestimated because of not considering the time difference between the start of observation and the time of intervention [2]. Patients who die early after the start of observation cannot receive the intervention owing to their death and are therefore allocated to the control group, making the intervention group appear to have a higher survival rate. The effect of this bias is brought about by patients who died during the period in which they could have received the intervention, and the impact is greater when there are many such patients in the analyzed population. Moreover, this bias occurs owing to reverse causation between the outcome and the intervention, so adjusting for patient characteristics cannot eliminate it.

If survivor treatment selection bias is present, there will be a tendency to overestimate the effect of the intervention in patients who die early after the start of observation. Consequently, there will appear to be a large difference in mortality rates between the intervention and control groups in a population that is likely to experience early mortality after observation. This creates the illusion that the intervention is significantly more effective in such a population. As a result, the selection criteria for patients who are expected to have a large treatment effect could be influenced by this bias.

In their analysis, the authors appear to have used the variable PMX-HA in the JSEPTIC-DIC study data to create a PMX-HA group and control group. According to the definition of JSEPTIC-DIC study data, the variable PMX-HA was “Yes” if the patient received PMX-HA during the first 7 days after ICU admission, and the start of observation was ICU admission [3]. Although the day of receiving PMX-HA was absent in the data, it could be inferred that not all patients in the PMX-HA group received PMX-HA simultaneously with their ICU admission because of the retrospective nature of that study. Therefore, the results of the study could be affected by survivor treatment selection bias and these criteria may not capture patients who may benefit most from PMX-HA.

Immortal time bias

The authors applied the aforementioned criteria to a validation cohort derived from the EUPHRATES trial [4] data and reported that PMX-HA significantly improved mortality in the “targeted subpopulation.” However, these results could be influenced by immortal time bias.

Immortal time bias arises when there is a delay between cohort entry and completion of the intervention [5]. In this analysis, the exposure was defined as “the standard regimen of two PMX-HA treatments,” that is, two PMX-HA treatments with a 24-h interval [4]. Consequently, the time frame from cohort entry to the first PMX-HA treatment and the 24-h interval between the first and second PMX-HA treatments constitute immortal time. Essentially, the individuals in the PMX-HA group must survive during this time frame. An analysis that neglects to account for this immortal time could lead to overestimation of the effect of PMX-HA.

The authors simply divided the population according to the presence or absence of the exposure. The Kaplan–Meier curve in their Fig. 3 exhibits no fatalities in the PMX-HA group until day 3, which suggests the presence of immortal time. Additionally, the authors used the Cox proportional hazards model, which does not consider immortal time bias. Therefore, the observed significant improvement in mortality among the PMX-HA group in the targeted subpopulation might be overestimated by immortal time bias. It would be more desirable to analyze the treatment allocation in the EUPHRATES trial as an exposure (intention-to-treat analysis) or to perform an analysis considering immortal time.

The above study addresses an important issue using a novel methodology, but biases may influence the results. Nevertheless, it should be noted that even if these biases are present, they may not significantly impact the conclusions. If the authors provide reanalysis results that address these concerns, it would greatly benefit Critical Care readers.

Availability of data and materials

The original data were derived from the manuscript provided by the journal, and the data from the JSEPTIC-DIC study are publicly available (Sci Data. 2018;5:180243).

References

  1. Osawa I, Goto T, Kudo D, Hayakawa M, Yamakawa K, Kushimoto S, et al. Targeted therapy using polymyxin B hemadsorption in patients with sepsis: a post-hoc analysis of the JSEPTIC-DIC study and the EUPHRATES trial. Crit Care. 2023. https://0-doi-org.brum.beds.ac.uk/10.1186/s13054-023-04533-3.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Sy RW, Bannon PG, Bayfield MS, Brown C, Kritharides L. Survivor treatment selection bias and outcomes research: a case study of surgery in infective endocarditis. Circ Cardiovasc Qual Outcomes. 2009;2:469–74.

    Article  PubMed  Google Scholar 

  3. Hayakawa M, Yamakawa K, Saito S, Uchino S, Kudo D, Iizuka Y, et al. Nationwide registry of sepsis patients in Japan focused on disseminated intravascular coagulation 2011–2013. Sci Data. 2018;5:180243.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Dellinger RP, Bagshaw SM, Antonelli M, Foster DM, Klein DJ, Marshall JC, et al. Effect of targeted polymyxin B hemoperfusion on 28-day mortality in patients with septic shock and elevated endotoxin level: the EUPHRATES randomized clinical trial. JAMA. 2018;320:1455–63.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  5. Mi X, Hammill BG, Curtis LH, Greiner MA, Setoguchi S. Impact of immortal person-time and time scale in comparative effectiveness research for medical devices: a case for implantable cardioverter-defibrillators. J Clin Epidemiol. 2013;66:S138–44.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

We thank Analisa Avila, MPH, ELS, of Edanz (https://jp.edanz.com/ac) for editing a draft of this manuscript.

Funding

The authors did not receive support from any organization for the submitted work.

Author information

Authors and Affiliations

Authors

Contributions

MU and KH wrote the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Masatoshi Uchida.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing of interests associated with this manuscript.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Uchida, M., Hayashi, K. Suspected bias in selection criteria of target subpopulation and its validation. Crit Care 27, 284 (2023). https://0-doi-org.brum.beds.ac.uk/10.1186/s13054-023-04566-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://0-doi-org.brum.beds.ac.uk/10.1186/s13054-023-04566-8