Post-study therapy as a source of confounding in survival analysis of first-line studies in patients with advanced non-small-cell lung cancer
1Institute for Stroke and Dementia Research, Ludwig-Maximilians-Universität München, Munich, Germany; 2Institut für Medizinische Statistik und Epidemiologie, Klinikum rechts der Isar, Technische Universität München, Germany; 3Oncology Department, Asklepios Lungenfachkliniken München-Gauting, Germany
Clinical trials exploring the long-term effects of first-line therapy in patients with advanced non-small-cell lung cancer generally disregard subsequent treatment although most patients receive second and third-line therapies. The choice of further therapy depends on critical intermediate events such as disease progression and it is usually left at the physician’s discretion. Time-dependent confounding may then arise with standard survival analyses producing biased effect estimates, even in randomized trials. Herein we describe the concept of time-dependent confounding in detail and discuss whether the response to first-line treatment may be a potential time-dependent confounding factor for survival in the context of subsequent therapy. A prospective observational study of 406 patients with advanced non-small-cell lung cancer served as an example base. There is evidence that time-dependent confounding may occur in multivariate survival analysis after first-line therapy when disregarding subsequent treatment. In the light of this important but underestimated aspect some of the large and meaningful recent clinical first-line lung cancer studies are discussed, focussing on subsequent treatment and its potential impact on the survival of the study patients. No recently performed lung cancer trial applied adequate statistical analyses despite the frequent use of subsequent therapies. In conclusion, effect estimates from standard survival analysis may be biased even in randomized controlled trials because of time-dependent confounding. To adequately assess treatment effects on long-term outcomes appropriate statistical analyses need to take subsequent treatment into account.
Key wordsnon-small-cell lung cancer; first-line therapy; survival analysis; post-study therapy; time-dependent confounding
J Thorac Dis 2011;3:88-98. DOI: 10.3978/j.issn.2072-1439.2010.12.07
Advanced non-small-cell lung cancer (NSCLC) is the leading cause of cancer-related death (1). Currently there is no universally accepted standard regimen for the first-line treatment of advanced NSCLC. Still platinum-based combination chemotherapy is recommended as first choice. Here the question whether carboplatin is as effective as cisplatin is controversially discussed (2). With the availability of second- and third-line anti-cancer agents such as docetaxel, pemetrexed and erlotinib, and a greater acceptance for more aggressive therapy the majority of patients receive therapy beyond first-line (3). Especially many participants of clinical first-line trials as good risk patients are offered additional therapy.
In this paper we describe the concept of time-dependent confounding which may contribute to bias in the outcome measures of oncology trials. Therefore, we used the patient cohort from the oncology department of the Asklepios Lungenfachkliniken Muenchen-Gauting to detect whether response to first-line therapy may be a potential confounding factor in survival analysis. The most recent large and pivotal first-line NSCLC studies published from 2008 to 2010 were reviewed for the strategies used by the authors to account for post-study therapy and the way they discussed the resulting potential impact on the observed results.
The problem of endpoints in oncology trials
In view of the growing number of possible drugs, combinations, sequences, and settings to be tested for various diseases, the choice of endpoints in oncology trials is becoming a critical issue. There is increasing controversy about valid outcome measures in oncology trials, especially in the first-line setting. Overall survival (OS) is accepted as the most reliable and relevant endpoint. Its drawback is that - depending on the natural course of the disease – it may take a long time until the expected event is observed. Furthermore it is subject to all therapeutic measures applied in the course of an individual patient’s disease. Thus, patient OS may well be influenced by the use of post-study therapy (4). As a consequence, Itaya et al (5) proposed to use the surrogate end point progression-free survival (PFS) as the primary outcome measure in first-line trials in order to overcome potential confounding by subsequent treatment. But reliable evidence of relevant clinical benefits or advantages is not given by using PFS, as extensively reviewed recently (6). A weakness rather than strength of PFS compared to OS is that it does not reveal insight into the real long-term impact and/or benefit of the investigational treatment (6).
The concept of time-dependent confounding
The estimation of unbiased effect estimates should definitively be the goal in clinical trials. However, standard methods for survival analysis, such as time-dependent Cox proportional hazards model, may produce biased effect estimates, regardless of whether one further adjusts for covariate history. In this context there is a potential bias caused by “time-dependent confounding”, and this concept may apply, whenever: (a) there exists a time-dependent covariate for mortality that also predicts subsequent treatment and (b) this covariate is not independent of previous treatment history (7-10).
Condition (a) implies that the measured covariate (for example response to or performance status after therapy) may be a confounder for the following treatment that must be adjusted for. On the other hand condition (b) implies that the covariate may also be affected by the previous treatment and thus, being an intermediate variable (i.e. a step between treatment and mortality), it should not be adjusted for by standard methods (9).
This complex problem is illustrated for the example of a general cancer therapy study by the directed acyclic graph in Figure 1. Superiority studies investigating the effectiveness of first-line therapies hypothesize a significant difference in the outcome (for example response) between the treatment arms (arrow 1 in Figure 1). As indicated by meta-analyses (11,12), there is strong evidence for a higher efficacy of cisplatin over carboplatin with regard to tumor response. Therefore condition (b) was met for these drugs in a hypothetical study context.
Response to first-line therapy has been shown to be an independent predictor of mortality (as stated by 13-16 and shown as arrow 3 in Figure 1). If response additionally predicts, i.e. influences the choice of subsequent treatment (arrow 2 in Figure 1) then condition (a) is also met. It is to be assumed that the choice of second-line treatment will differ in dependence of the quality of the response achieved to the previous treatment regimen. However, information about this association is scarce. The decision on how therapy is being continued after firstline treatment is usually made on an individual basis, and it is thought to be influenced by the kind of drug used at first-line, the response to first-line treatment, adverse reactions, early discontinuation of first-line treatment, actual performance status and other individual patient characteristics (17-20). In a recent study (20), physicians were requested to assess the primary reason for selecting a specific chemotherapy by completing multiple choice forms. Main motivation was perception of efficacy in this study.
The potential consequences of time-dependent confounding for clinical trials
What are the possible consequences of these considerations? If, for example, an ineffective first-line treatment in a given patient may guide the decision on an effective second-line regimen, then this decision may lead to a longer OS which is then falsely attributed to the in fact less effective first-line regimen.
At least in large studies randomization prevents confounding by a comparable distribution of baseline characteristics and therefore prognostic factors between treatment groups (Figure 1, dashed arrows from baseline confounder to first-line treatment, 21). However, the choice of further therapy after disease progression is usually at the physician’s discretion, also in randomized trials. Time-dependent confounding may then become a problem, especially in first-line studies because of the high probability of subsequent lines of therapy.
In our own patient cohort about 30% of the patients actively treated were clinical trial participants within different firstline studies for advanced NSCLC (3,16). More than 60% of these participants in clinical first-line trials received a secondline therapy, about 35% a third-line therapy, and about 20% radiotherapy after first-line systemic therapy. An association between first-line treatment (platinum-based compared to not platinum-based therapy) and disease control (DC, defined as any response and disease stabilization) and between DC and OS has already been shown and is published for this study population (16). To fulfill the criteria of a potential time-dependent confounder DC in addition would have to be associated with subsequent treatment. In order to reveal such an association we exemplarily investigated if the use of subsequent treatment is indeed associated with the quality of response achieved to firstline treatment in an own cohort of patients.
Analysis of an own cohort of patients with advanced NSCLC regarding the association between response to first-line treatment and subsequent therapy
Patients and methods
Patients with histologically confirmed NSCLC with stage IIIB wet or stage IV were included in this prospective exploratory observational study. Between January 2003 and July 2007, 519 patients with untreated advanced NSCLC were admitted to a single ward of the Asklepios Lungenfachkliniken Gauting. Of these patients, 406 were treated at our center systemically. Patients were followed-up until August 2010. For further description of the study population and data collection see Zietemann and Duell (3,16).
Tumor evaluation was performed according to our internal standards. CT scans of the chest covering the upper abdomen including liver and adrenal glands and of the brain were carried out every 6 weeks during therapy and every 12 weeks in therapyfree intervals unless indicated by the worsening or development of clinical symptoms. Tumor response was evaluated semiquantitatively (categories: partial response, stable disease, progressive disease), a practice routinely applied in everyday clinical practice. Disease progression was defined as an appearance of new lesions or a clinically relevant growth or deterioration of known lesions and/or symptoms.
To investigate time-dependent confounding with respect to further-line treatments, we have chosen the variable DC after first-line treatment as a possible time-dependent confounder because DC rate was found to be a good predictor of OS in our and in other studies (13,15,16).
To assess the association between response and the initiation of second-line treatment (radiotherapy and chemotherapy, arrow 2 in Figure 1), the χ2-test was applied (22). Baseline variables (sex, age, stage of disease, histology, Karnofsky performance score, weight loss as symptom, smoking habit, and metastasis location) were included as adjustment covariates and factors within multiple logistic regression analyses considering initiation of radiotherapy or chemotherapy as dependent variable (23). Cox proportional hazard models were used to consider the time between end of first-line and initiation of second-line chemotherapy in the model 24. Odds ratios (OR) and hazard ratios (HR) were reported with 95% confidence intervals. To adjust for the continuous confounder “age” we used the SAS macro %RCS_Reg (restricted cubic spline functions) in all multiple analyses 25. All analyses were two-sided, conducted at a 0.05 level of significance and carried out using SAS version 9.1 (SAS Institute Inc, Cary, NC).
Results – Analyses of time-dependent confounding
Association between response to first-line and the use of systemic second-line therapy
Initiation of second-line therapy was less frequent in patients not achieving DC (Table 1; with DC: 60%, without DC: 47%; χ2- test: p-value=0.014). However, this difference was not significant after adjustment for covariates (ORadjusted=0.91; 95%CI: 0.54- 1.52).
Median time between stop of first-line and start of second-line was 117 days for patients with DC (inter quartile range (IQR): 71 to 188) and 23 days for patients without DC (IQR: 16 to 48). Achieving no DC was associated with a higher probability of an early initiation of second-line chemotherapy (HRadjusted=4.07; 95%CI: 2.71-6.13).
Association between response to first-line therapy and subsequent radiotherapy
Initiation of radiotherapy differed significantly depending on best response after first-line treatment (Table 1; χ2-test: p-value <0.001). Initiation rate was lower for patients not achieving DC (with DC: 30%, without DC: 10%; ORadjusted=0.26; 95%CI: 0.11-0.55).
Evaluation of our results
The analyses from our prospective observational study support the hypothesis that time-dependent confounding may bias the results of standard survival analyses in the first-line treatment of NSCLC. We found relevant associations between first-line treatment and response (16), between response and survival (16), and, as shown here, between response and the initiation of therapy after first-line systemic therapy.
Our observational study has limitations, but also strengths: because there was no patient selection for inclusion, the results represent the treatment situation in every-day clinical practice. Since data derive from only one department of a single institution the results cannot be generalized as self selection cannot be ruled out. Because confounding is a main problem in observational studies unmeasured confounding may have biased our results as well. Not using exact tumor measurement, such as RECIST, reflects our every-day clinical practice where we focus mainly on clinical criteria of response and clinical benefit rather than on tumor shrinkage. Because we had detailed information about radio- as well as systemic therapy after first-line treatment we could analyse the impact of response to initiation of both, including a large amount of possible confounding factors.
Review on recently published clinical first-line trials regarding post-study therapy in the light of timedependent confounding
As shown above, biased effect estimates may be obtained also in randomized first-line trials, whenever the choice of subsequent therapy may be influenced by the outcome of the previous line of treatment. Although some studies provided information on subsequent treatment separately for the treatment groups (for example for cisplatin- compared to carboplatin-based firstline therapy (26-30)), the potential impact of further-line treatments on survival has rarely substantially been considered. Fossella et al (26) reported that second-line treatment did not confound survival results in favour of therapy with docetaxel plus platinum, but they did not explain how they came to this conclusion. Belani et al (27) mentioned possible confounding of survival as a result of a different use of taxanes second-line. To our knowledge studies comparing cisplatin and carboplatin at first-line adjusting for different subsequent treatments are not available and therefore bias may have influenced the results of studies comparing these two drugs.
In 2000, docetaxel was the first cytotoxic agent to be registered for second-line treatment, pemetrexed at the end of 2004, and erlotinib followed soon after. Because of the increasing treatment options for second- (and even further-) line and the growing number of patients receiving more than one line of therapy it becomes more and more important to take the influence of subsequent lines of therapy on survival into consideration (31,32). Especially first-line trials initiated after 2004 are therefore potentially subject to time-dependent confounding. In this light we reviewed recently performed or published larger phase III first-line studies. We found some recent studies who did not mention post-study therapy at all (33- 35), including one study investigating the effect of early versus late second-line docetaxel (36).
Table 2 summarizes seven large studies with inclusion period starting 2004 who at least gave some information about poststudy treatment (1,37-42). Information about radiotherapy after first-line treatment was given only by three studies (1,38,41). Three studies did not discuss a possible impact on OS (table 1; 37-39) and one stated that there was no impact on the final results because of similar proportions of patients in both arms having received second-line therapy, although the groups differed regarding the post-study use of gemcitabine and docetaxel (40).
regarding the post-study use of gemcitabine and docetaxel (40). In the large cetuximab (FLEX) study it was stated that in the experimental arm post-study use of tyrosine kinase inhibitors was less frequent than in the chemotherapy-alone arm (38), which indicates that in unblinded studies the drugs used in prior lines influence the choice on those used later on.
One study which discussed the possible impact of post-study medication more in detail is the recently published AVAiL study (1). An exploratory OS analysis for patients who did not receive post-protocol therapy was performed. However, restriction to those patients without post-study-treatment may introduce bias because adjusting for a confounded intermediate will induce confounding even if the exposure is randomized (10,21,43).
The problem of time-dependent confounding is not only limited to the first-line setting but also to “maintenance” studies (44-46). In general it is an issue whenever “treatment by indication” is given (47). The data needed to correctly adjust for time-dependent confounding are notoriously difficult to collect, and many studies collect information on the class of drug administered upon progression, only.
Table 1. Subsequent therapies depending on response to first-line therapy (Patients may have received more than one drug; cells are not mutually exclusive).
EGFR: epidermal-growth-factor-receptor; VEGF: vascular-endothelial-growth-factor
Table 2. Characteristics of recently published large randomized phase III first-line trials in patients with advanced NSCLC
Table 2. Characteristics of recently published large randomized phase III first-line trials in patients with advanced NSCLC(continued)
TKI: epidermal-growth-factor-receptor kinase inhibitors (erlotinib, gefitinib); MAB: monoclonal antibody (bevacizumab; cetuximab); chemotherapy: (cisplatin, carboplatin, docetaxel, paclitaxel, gemcitabine, vinorelbine, mitomycin, pemetrexed); DC: disease control rate; ORR: overall response rate; OS: overall survival; HR: hazard ratio; n.spec: not specified; n.p.: not presented; mth: months; bev: bevacizumab; car: carboplatin; cet: cetuximab; cis: cisplatin; doc: docetaxel; erl: erlotinib; gef: gefitinib; gem: gemcitabine; pem: pemetrexed; tax: taxane; VP: cisplatin + vinorelbine; CG: cisplatin + gemcitabine; CD: cisplatin + docetaxel; CP: carboplatin + paclitaxel; GC: carboplatin + gemcitabine; TC: carboplatin + paclitaxel or docetaxel
Table 3. . Given information about poststudy therapy and discussion about the possible impact of poststudy therapy on OS
Table 3. . Given information about poststudy therapy and discussion about the possible impact of poststudy therapy on OS (continued)
OS: overall survival; bev: bevacizumab; car: carboplatin; cet: cetuximab; cis: cisplatin; doc: docetaxel; erl: erlotinib; gef: gefitinib; gem: gemcitabine; pem: pemetrexed; tax: taxane; VP: cisplatin + vinorelbine; CG: cisplatin + gemcitabine; CD: cisplatin + docetaxel; CP: carboplatin + paclitaxel; GC: carboplatin + gemcitabine; TC: carboplatin + paclitaxel or docetaxel
The importance of respecting information about anticancer systemic therapy, radiotherapy and surgical intervention during the post-study period is slowly entering the world of cancer trials. But in which extent this information is used in the statistical analysis is usually not revealed (45). In many published studies it is discussed that the impact of post-study therapy on survival was difficult to evaluate because the choice of subsequent treatment is left to the discretion of the investigators (46). Statements like “the selection of post-study treatment did not appear to influence the overall survival conclusion” (46) or “the fact that a small equal number of patients in each arm had second-line treatment and no response was observed shows that secondline treatment did not influence the survival data” (48) can be found, but they are usually made without adequate scientific and statistical evidence.
One way to overcome this problem was to predefine the post-study treatment at study entry/randomization in order to avoid bias introduced by such imbalances discussed above. Alternatively, new statistical methods are available to estimate the causal effect of time-dependent exposure in the presence of time-dependent confounders, i.e. marginal structural models and structural nested models (7,8,10). However, only one study could be detected using causal models to adjust for differential proportions of second-line treatment measures (radiotherapy and chemotherapy) in cancer clinical trials comparing cisplatin plus irinotecan with cisplatin plus vindesine (49,50). Unfortunately sample size was small and the results may therefore be unstable. Furthermore, exact information on each patient’s treatment history was not presented and it was assumed in the statistical model that the effect of second-line radiotherapy was maintained up to time of death once it was initiated (50). We could show, alike Yamagucchi and Ohashi (50), that the decision of the physician for the initiation of radiotherapy after first-line chemotherapy is associated with the response to first-line treatment. We furthermore identified sex, histology and brain metastases at baseline as relevant factors for initiation of radiotherapy, and Karnofsky performance score and bone metastases at baseline as relevant factors for initiation of chemotherapy (data not shown).
Conclusion and future perspectives
Conclusion and future perspectives between response and outcome, but we want to emphasize the consequences resulting from these associations.
At present our own ongoing study is underpowered for complex analyses like marginal structural models and structural nested models. Our future aim is to analyse the data using standard methods and causal models and to investigate the impact of time-dependent confounding. Results obtained from our own analysis and the literature indicate that response may be one of many potential time-dependent confounders in survival analysis following first-line therapy. The treatment flow of patients with advanced cancer is being determined by a complex combination of dynamic and static influence factors. Effect estimates may be biased especially if dynamic variables are not explicitly accounted for in the analyses (10). Future trials should take subsequent treatments and adjustment for t imed ependent confounding into consideration. Detailed documentation of subsequent treatment as well as other covariates influencing the respective treatment decision process is a must in future studies, otherwise even the best statistical approaches will fail to reveal the complex interdependence between the patients individual characteristics, the biology of the disease, and the therapeutic measures applied. To draw conclusions about optimal treatment strategies further analyses including all relevant time-independent and timedependent confounding factors are necessary.
We thank Prof. Loems Ziegler-Heitbrock for critical comments on the manuscript.
Cite this article as: Zietemann VD, Schuster T, Duell THG. Post-study therapy as a source of confounding in survival analysis of first-line studies in patients with advanced non-small-cell lung cancer. J Thorac Dis 2011;3(2):88-98. doi: 10.3978/j.issn.2072-1439.2010.12.07