Randomised controlled trials provide the best scientific evidence for the efficacy of biological drugs in inflammatory bowel disease (IBD). However, findings obtained from these trials might not be reproducible in clinical practice. This study aimed to estimate the percentage of patients with IBD treated with biologics who would have been eligible for randomised controlled trials, and to compare the theoretical efficacy of biological drugs with their effectiveness in clinical practice.
MethodsWe performed a retrospective multicentre study in 375 patients with IBD treated with anti-TNF agents and followed-up for 1 year. The eligibility criteria for the trial were taken from the ACCENT, SONIC, ACT, CLASSIC and CHARM trials. Eligible patients were included in a second analysis to compare results in clinical practice versus those hypothetically obtained if the patient had been included in a trial.
ResultsOnly 45.6% of 375 patients would have been eligible for pivotal trials. One-year clinical benefit (remission or response) was similar for eligible and non-eligible cohorts (68.4% vs. 68.6%, p=0.608). The clinical benefit was greater for current clinical practice than for a hypothetical trial situation (68.4% vs. 44.4%, p<0.001) in eligible patients.
ConclusionMore than half of patients with IBD treated with biologic drugs would not be represented in pivotal trials. The effectiveness of anti-TNF drugs in clinical practice exceeds their theoretical efficacy.
Los ensayos clínicos aleatorizados proporcionan la mejor evidencia científica de la eficacia de los fármacos biológicos en la enfermedad inflamatoria intestinal (EII). Sin embargo, los resultados pueden no ser reproducibles en la práctica clínica. Los objetivos de este estudio son analizar el porcentaje de pacientes con EII tratados con fármacos biológicos que habrían podido ser elegidos para un ensayo clínico aleatorizado y comparar la eficacia teórica de los fármacos biológicos con su efectividad en la práctica clínica.
MétodosRealizamos un estudio retrospectivo multicéntrico en 375 pacientes con EII tratados con anti-TNF con un seguimiento de un año. Los criterios de elegibilidad para la condición de ensayo clínico fueron extraídos de los estudios pivotales ACCENT, SONIC, ACT, CLASSIC y CHARM. Los pacientes elegibles fueron incluidos en un segundo análisis para comparar los resultados en la práctica clínica con los obtenidos tras realizar una estimación teórica si el paciente hubiese sido incluido en un estudio pivotal.
ResultadosSolo el 45,6% de los 375 pacientes cumplían los criterios de selección para un estudio pivotal. El beneficio clínico al año fue similar entre los pacientes elegibles y no elegibles (68,4% vs 68,6%). El beneficio clínico en los pacientes elegibles fue mayor en la práctica clínica que en la condición hipotética de un ensayo clínico (68,4% vs 44,4%, p<0,001).
ConclusiónMás de la mitad de los pacientes con EII tratados con fármacos biológicos no estarían representados en los ensayos pivotales. La efectividad de los fármacos anti-TNF en la práctica clínica es superior a su eficacia teórica.
Crohn disease (CD) and ulcerative colitis (UC) are chronic autoimmune inflammatory diseases of the gastrointestinal (GI) tract.1,2 Conventional treatment includes the use of corticosteroids and immunosuppressants. The incorporation in the last decade of anti-tumour necrosis alpha (anti-TNF) antibodies, with their good safety profile, has transformed the treatment and prognosis of these patients.3 Their efficacy is supported in several randomised and placebo-controlled clinical trials,4–6 which has led to the approval of these drugs by regulatory agencies.
It is generally agreed that randomised clinical trials (RCT) are the best scientific approach to study the effect of a drug or therapeutic strategy. RCTs use strict selection criteria and a defined study protocol to maximise their internal validity.7 Furthermore, most of the variables that could affect or interfere in the outcome are removed in order to evaluate the effect attributable to the study drug and remove most potential bias. In this way, RCTs evaluate the efficacy of a drug in a selected population and under controlled conditions. However, patient selection can limit the external validity or representativeness of the outcomes in the entire patient population eligible for treatment, and the findings must be carefully extrapolated to clinical practice (CP).
In contrast, observational studies evaluate the efficacy of a drug in real CP conditions, and variables and biases that affect the final outcome may appear during follow-up. The efficacy rates of observational studies in inflammatory bowel disease (IBD) using biologics are superior to the efficacy demonstrated in RCTs. For example, the sustained response rates at 1 year in primary responders with infliximab and adalimumab were 39% and 43%, respectively, in the ACCENT 14 and CHARM5 studies. These results were lower than those obtained in observational studies, which reported rates of over 60% for both anti-TNF.8,9
This paper (EFIFECT study) is a theoretical exercise aimed at evaluating the degree of representativeness of patients with IBD treated with anti-TNF in CP and RCTs, and to identify potential discrepancies between the effectiveness and theoretical efficacy of these treatments.
Materials and methodsStudy population and data collectionWe conducted an observational, retrospective study of patients with IBD treated with anti-TNF drugs in CP in 5 Spanish tertiary hospitals. Cases were selected randomly from electronic databases (1 case for every 4 adult patients with IBD treated with anti-TNF drugs) in order to obtain a representative sample over time. All patients selected were followed-up for at least 1 year after commencing anti-TNF therapy.
Demographic data, phenotype, disease location, current or past medication and history of surgery were recorded. The indication for the biologic, activity at the start of treatment and therapeutic response at 1 year were also included. The data were obtained from medical records.
The study was approved by the Balearic Islands Research Ethics Committee in October 2012 (IB 1947/12 EPA).
EndpointsPart 1. To analyse the percentage of patients with IBD treated with a biologic in CP who would have been eligible for a pivotal RCT, and to investigate the reasons for ineligibility.
Part 2. To evaluate the percentage of eligible patients who achieved remission or response in CP after 1 year of treatment, and to compare it with the outcomes under the hypothetical conditions of an RCT (theoretical estimation). To describe the factors that explain the discrepancy between the efficacy and effectiveness.
Definitions and criteria used in randomised clinical trialsRCT criteria: we selected the inclusion and exclusion criteria from the published papers of the ACCENT 1,4 ACCENT 210 and SONIC11 studies for infliximab in CD, CLASSIC 112 and CHARM5 for adalimumab in CD, and ACT 1 and ACT 26 for infliximab in UC. Patients who met all the inclusion and none of the exclusion criteria for at least 1 of the 6 pivotal trials were considered eligible for an RCT.
Remission, response and failure were evaluated at the discretion of the clinician responsible, although whenever the data could be extracted from the medical record, clinicians based their decision on the Harvey–Bradshaw score13 for CD and the partial Mayo Clinic score6 for UC. The clinical response was evaluated at week 52 in patients who continued with anti-TNF, or at the last visit if treatment was discontinued.
Eligible patients were re-evaluated under hypothetical RCT conditions. Evaluation criteria for the RCT were extracted from the different studies.4–6,10–12 Any protocol violation, such as a new steroid cycle, initiation of an immunosuppressant or topical therapy, escalation of an anti-TNF, pregnancy or serious infection was considered as a failure.
Escalation of the biologic was defined as a clinically indicated increase in the dose or frequency of administration.
Statistical analysisA descriptive analysis of the results was performed. Qualitative variables were described as frequencies and percentages with their 95% confidence intervals. Continuous variables were described as means or medians with the corresponding standard deviation or interquartile range. The chi-square test and Fisher's test was used to compare categorical variables, while the Student's t and Mann–Whitney tests were used for continuous variables with values that did not follow a normal distribution. A difference was considered statistically significant when the p value was less than 0.05. The sample size was calculated at 350 patients to be able to detect a 15% difference between the clinical outcome at 1 year with anti-TNF treatment in CP conditions and in the hypothetical RCT setting.
ResultsPart 1. Eligibility for a randomised clinical trialThe study included 365 patients with IBD (276 CD and 99 UC) who had received clinically indicated treatment with infliximab or adalimumab in real CP, and who had follow-up of at least 1 year.
A total of 171 patients (45.6%) met the selection criteria for at least 1 of the 6 RCTs. The percentage of eligible patients with CD and UC in the RCTs was 47.1% and 41.4%, respectively.
In CD, age, sex, smoking status, duration of disease and use of immunosuppressants were comparable in the eligible and ineligible patient populations (Table 1). However, ileocolonic location, non-obstructive non-fistulising pattern, perianal disease and percentage of biologic-naive patients were more common in the group of eligible patients. In UC, differences were only found in disease extension and use of immunosuppressants (Table 2).
Baseline characteristics of patients with Crohn disease.
| Eligible | Ineligible | p | |
|---|---|---|---|
| (n=130) | (n=146) | ||
| Age (years), mean±SD | 35.7±13.0 | 38.5±13.5 | 0.072 |
| Sex (female), n (%) | 63 (54.8) | 78 (56.5) | 0.782 |
| Active smoker, n (%) | 39 (30.7) | 44 (30.1) | 0.980 |
| Duration of disease, M (IQR) | 6 (1–29) | 6 (1–37) | 0.265 |
| Location of disease, n (%) | |||
| L1 (Ileal) | 29 (22.5) | 59 (41.0) | 0.002 |
| L2 (colic) | 22 (17.1) | 22 (15.3) | |
| L3 (ileocolic) | 78 (60.5) | 60 (41.7) | |
| L4 (upper gastrointestinal) | 0 (0.0) | 3 (2.1) | |
| Perianal | 62 (48.1) | 39 (26.9) | <0.001 |
| Evolutionary pattern, n (%) | |||
| B1 (inflammatory) | 82 (63.6) | 51 (35.4) | <0.001 |
| B2+B3 (stenosing or fistulising) | 47 (36.4) | 93 (64.6) | |
| Baseline HBI, M (IQR) | 8 (0–16) | 5 (0–24) | 0.002 |
| Use of immunosuppressants, n (%) | 122 (93.8) | 132 (90.4) | 0.293 |
| Anti-TNF naive, n (%) | 110 (85.9) | 106 (72.6) | 0.009 |
| Infliximab/adalimumab (%) | 65/34 | 48/51 | |
HBI: Harvey–Bradshaw index; IQR: interquartile range; M: median; SD: standard deviation.
Baseline characteristics of patients with ulcerative colitis.
| Eligible | Ineligible | p | |
|---|---|---|---|
| (n=41) | (n=58) | ||
| Age (years), mean±SD | 40.3±15.8 | 45.6±16.0 | 0.304 |
| Sex (female), n (%) | 19 (47.5) | 22 (39.3) | 0.106 |
| Active smoker, n (%) | 4 (9.8) | 3 (5.3) | 0.444 |
| Duration of disease, M (IQR) | 5 (1–23) | 5 (1–38) | 0.893 |
| Extension of the disease, n (%) | |||
| Pancolitis | 27 (65.9) | 30 (53.6) | 0.048 |
| Left-sided colitis | 29 (22.0) | 24 (42.9) | |
| Proctitis | 5 (12.2) | 2 (3.6) | |
| Partial Mayo score, M (IQR) | 6 (3–9) | 7 (1–9) | 0.753 |
| Use of immunosuppressants, n (%) | 41 (100) | 51 (87.9) | 0.039 |
| Anti-TNF naive, n (%) | 41 (100) | 54 (94.7) | 0.638 |
IQR: interquartile range; M: median; SD: standard deviation.
The most common cause of ineligibility was prescription of the drug for a reason other than a moderate-to-severe flare-up or perianal disease. This group included patients with mild or no clinical activity, in which the biologic was prescribed to prevent recurrence or corticosteroid dependence, among other reasons. Symptomatic stenosis in CD and use of topical drugs in UC were the next most common causes of ineligibility (Tables 3 and 4).
Causes of ineligibility for a randomised clinical trial in Crohn disease.
| Causes | Frequency |
|---|---|
| Symptomatic stenosis | 24.7% |
| High-dose corticosteroids | 8.9% |
| Primary failure to anti-TNF | 7.5% |
| Recent surgery, stoma or short bowel syndrome | 6.8% |
| Abscess | 2.7% |
| Tuberculosis infection | 2.7% |
| Recent diagnosis | 2.7% |
| HIV, HVB or HVC infection | 2.1% |
| Comorbidities | 1.4% |
| History of cancer | 0.7% |
| Imminent surgery | 0.7% |
Anti-TNF: anti-tumour necrosis factor; HBV: hepatitis B virus; HCV: hepatitis C virus; HIV: human immunodeficiency virus.
Causes of ineligibility for a randomised clinical trial in ulcerative colitis.
| Causes | Frequency |
|---|---|
| Topical treatment | 43.1% |
| Treatment indication | 27.6% |
| High-dose corticosteroids | 15.5% |
| Tuberculosis infection | 5.2% |
| Recent diagnosis | 5.2% |
| History of cancer | 1.7% |
| HIV, HVB or HVC infection | 1.7% |
HVB: hepatitis B virus; HCV: hepatitis C virus; HIV: human immunodeficiency virus.
The percentage response after 1 year of treatment in CP was 68.4%. No significant differences in the clinical response (overall, CD and UC) were observed between patients who were eligible and ineligible for an RCT (Fig. 1).
The overall percentage of remission was 55.3%. Remission percentages were higher among ineligible patients than eligible patients (57.5% vs. 50.8% in CD and 63.8% vs. 51.2% in UC), but these differences were not statistically significant.
Part 2. Discrepancy between efficacy and effectivenessA total of 171 patients (130 CD and 41 UC) who met eligibility criteria were included. The clinical benefit at 1 year was higher in CP than in the theoretical RCT evaluation (68.4% vs. 44.4%, p<0.001). Up to 35% of patients with clinical benefit in CP would have been considered as failures had they been included in an RCT.
The percentage of patients with clinical remission after 1 year of follow-up was 50.9% in CP vs. 39.2% using the evaluation criteria from a pivotal trial (p<0.001). Fig. 2 shows the clinical outcomes in patients with CD and UC with both evaluation criteria.
The most common reasons for discrepancy between CP and RCT evaluation were the need for escalation (75.6%) and treatment with a new corticosteroid cycle (12.2%). Other less common causes were a delay in treatment administration, surgery and infusion reaction.
DiscussionThe use of anti-TNF drugs in patients with IBD was approved by regulatory agents based on data from pivotal RCTs.4–6,10–12 RCTs use strict inclusion and exclusion criteria to select a homogeneous population.14 Patients with more severe forms, older patients, those with comorbidities, suspected poor adherence or receiving treatment with drugs that could interfere with the study drug are systematically excluded. These studies are designed to maximise the internal validity at the expense of the external validity. The results obtained from an RCT can therefore be reproduced in an analogous population under similar conditions. However, it is common (and often erroneous) to assume that these findings are applicable to general CP, including the situations that were excluded from the original study.
In our study, more than half of patients with IBD treated with anti-TNF drugs in routine practice would not meet the inclusion criteria for a pivotal trial. Only 45.6% of our patients met the selection criteria for 1 or more of the 6 RCTs. Ha et al.15 retrospectively reviewed 206 patients with IBD with moderate-severe disease activity followed up in Mount Sinai Medical Center. Only 34% of patients with CD and 25% of patients with UC would have met the eligibility criteria for 1 or more of the pivotal studies on infliximab, adalimumab, certolizumab or natalizumab. The external validity of RCTs has been evaluated in other chronic diseases with similar results. For example, only 42% and 16% of 2 cohorts of patients with early onset rheumatoid arthritis met eligibility criteria for an RCT that compared methotrexate vs. etanercept, while this percentage was only 5% in the ATTRACT study (infliximab plus methotrexate vs. monotherapy with methotrexate). The authors recommended that the inclusion criteria for RCTs on rheumatoid arthritis should be modified in order to improve the applicability of the findings to CP.16 In another study, Herland et al.17 found that only 5.4% of patients with asthma and 17% of patients with chronic obstructive pulmonary disease met the most common criteria for the RCT. The authors questioned whether the study results could be extrapolated to the real patient population. The percentage representativeness was higher in our study because we focused only on patients with IBD treated with anti-TNF drugs, and not on the entire IBD population.
In real CP, biologics are administered by physicians in different healthcare structures (from IBD units with their own day hospital and specialised nursing staff to professionals who work part time with patients with IBD). The patients are heterogeneous and, as we have seen, treatments are given in situations not considered in the pivotal studies. They can present comorbidities, treatment with other drugs that may interact with the anti-TNF and situations that are more serious or complex than those included in the RCTs. Observational studies, case series, post-marketing RCTs for specific situations, and expert recommendations add to and complement the information gleaned from pivotal RCTs. We have evidence of treatment with biologics in the prevention of recurrence in CD18 and severe UC,19,20 pouchitis,21 abdominal phlegmons,22 patients infected with hepatitis B or C virus23 or during pregnancy,24 situations that were excluded from the pivotal RCTs. Moreover, some expert recommendations come from the experience acquired in observational studies in non-selected populations.
We did not find significant differences in age, sex, smoking habits or use of immunosuppressants between eligible and ineligible patients. In contrast with the pivotal studies, more than 90% of our patients had received or were receiving treatment with immunosuppressants. This is in line with the recommendations of the European Crohn's and Colitis Organization (ECCO) for the use of anti-TNF in CP,3 and is similar to the figure of 98% reported in another study.9 There were, however, differences in the location and pattern of CD, with a higher percentage of ileal forms and stenosing and penetrating patterns in ineligible patients. Some of the RCT exclusion criteria, such as the presence of fibrotic stenoses, abscesses or the use of anti-TNF to prevent recurrence—which are more common in ileal CD—could explain these differences.
We did not find any differences in the response rates between eligible and ineligible patients treated with anti-TNF. Ineligible patients add greater heterogeneity to observational studies, and there could be patient groups with different response rates to the same treatment. For example, only 45% of patients with pouchitis treated with anti-TNF presented a response,21 while success in the prevention of clinical recurrence at 1 year was 80% in the Regueiro study.18
The methodological heterogeneity makes it difficult to compare our results with those of other authors. It would be interesting for scientific societies to establish recommendations that would enable this comparison. Some studies evaluate remission and response rates using activity scores, while others evaluate the clinical benefit according to the clinician's opinion. Some include all patients who begin treatment, while others only evaluate induction responders. Finally, the follow-up time varies, and the results are expressed as mean follow-up time or different time intervals. Despite this, the clinical benefit of 68.4% observed in our study is similar to other observational studies. In the Lovaine cohort,8 with 614 patients with CD treated with infliximab, the clinical benefit was 56.5%, with a mean follow-up of 4.6 years. In another English series, 210 patients were evaluated retrospectively: 65.9% presented a response and 40% achieved clinical remission, with a mean follow-up of 24 months.25 A similar Canadian cohort with 133 patients observed a percentage response at 1 year of around 60%.9 There are similar findings with infliximab in UC (80%)26 and adalimumab in CD (63%).27
However, there is a large discrepancy between the results in CP in most observational series and the data from RCTs. The ACCENT 14 study included 573 patients with CD who were given treatment with 5mg/kg of infliximab. Slightly over half (58%) presented a response at week 2 and were randomised to infliximab (5 or 10mg/kg) or placebo at weeks 2 and 6, and every 8 weeks for 1 year. The percentages of remission and response for the primary responders at week 54 were 28% and 39% in the infliximab 5mg/kg group, higher than the placebo group (17% and 22.5%; p=0.007). The CHARM5 study, similar in design to ACCENT 1, evaluated the efficacy of adalimumab in 854 patients with CD. A first induction phase was carried out with 80 and 40mg of adalimumab at weeks 0 and 2, and randomisation at week 4 to 40mg adalimumab weekly, 40mg every 2 weeks or placebo. Remission and response were achieved in 36% and 43% of primary responders (58%) at week 56 in the adalimumab every 2 weeks group, with significant differences with respect to the placebo group. The ACT 16 study included 364 patients with moderate-severe UC on treatment with infliximab (5 or 10mg/kg) or placebo. Both remission and response at week 54 were significantly higher in the 5mg/kg treatment group (34.7% and 45.5%, respectively) than in the placebo group (19.8%).
In addition to the rigorous selection criteria, RCT investigators establish a strict follow-up protocol, with more frequent, controlled visits than in CP. Any circumstances that could interfere with the study variables are removed. If any protocol violations occur, the subject is withdrawn from the study and considered as failure. For example, in the ACCENT 1 study, the protocol considered as failure any patient who required treatment escalation, received prohibited medication or underwent surgery, regardless of the activity score.4
In the second part of the study, we only considered patients who would be eligible for RCTs, and compared the response to anti-TNF treatment in our CP with the theoretical results under the hypothetical conditions of an RCT. Less clinical benefit was observed under RCT criteria than in CP (44.4% vs. 68.4%). One-third of patients would be considered as failures if they had been included in an RCT. These findings confirm the previous results of a pilot study with 74 patients with IBD presented at the ECCO conference in 2011.28
The main causes of discrepancy between the CP results and those of the theoretical RCT study were the need for escalation (75.6%) and having started a new corticosteroid cycle (12.2%). Dose escalation was necessary in 23% of patients in our series to maintain the response during the first year of treatment. This percentage is similar to the 21% observed in an English cohort at 2 years of follow-up.25 These interventions are common routine practice in patients with loss of response or partial response. In fact, escalation is backed by scientific evidence, and is recommended in IBD clinical guidelines. However, in the pivotal RCTs, any modification of the drug dose or administration interval was considered as a protocol violation, as it interferes with the primary endpoint, which is assessment of the drug efficacy.
The study has some limitations that may lead to bias. Data were extracted from medical records, so there could have been reasons for exclusion or protocol violations for an RCT that were not considered. For example, considering symptomatic stenosis as an exclusion criterion could be controversial, depending on the inflammatory or fibrotic component. There could have been modifications in the concomitant treatments that were not evaluated. Moreover, only 1 reason for ineligibility was recorded, and in cases in which there were more than 1, the investigator selected the most relevant reason. The 1-year results were based on the clinician's opinion in cases where no clinical scores were available, as in other similar studies, and the same criteria were also applied in the theoretical RCT evaluation.
It is important to highlight that RCTs and observational studies have different purposes.29 RCTs are the best approach for studying the efficacy of a treatment in very controlled conditions. However, RCTs do not reflect the way in which the drug is used in CP, and their results do not represent the entire patient population.30 Observational studies offer realistic information on therapeutic response in a heterogeneous population, and better reflect the likelihood of response to a drug administered under CP conditions. Both studies are essential and the health outcomes obtained complement each other in medical decision-making. David Mant31 notes that: “The paradox of the clinical trial is that it is the best way to assess whether an intervention works, but is arguably the worst way to assess who will benefit from it”.
In conclusion, half of patients with IBD treated with anti-TNF in real CP are not represented in pivotal trials. Furthermore, RCTs underestimate the likelihood of response to anti-TNF drugs in IBD.
FundingLuis Ibáñez-Samaniego has received a grant from ISCIII (RÍO-HORTEGA CM14/00023).
AuthorshipDaniel Ginard and Sam Khorrami have contributed equally to the authorship of this manuscript.
Conflict of interestsThe authors declare that they have no conflict of interests.
Please cite this article as: Ginard D, Khorrami S, Pérez-Carazo L, Tavío-Hernández E, López-Sanromán A, García-Alvarado M, et al. Eficacia y efectividad de la terapia biológica en la Enfermedad Inflamatoria Intestinal. Estudio EFIFECT. Gastroenterol Hepatol. 2016;39:369–376.
