Are Asthmatics Enrolled in Randomized Trials Representative of Real-Life Outpatients?

Clinical practice guidelines, the backbone of evidence-based medicine, are based upon the results of meta-analyses and randomized controlled trials (RCTs). In order to exclude potential confounding factors that may affect the results of the RCTs, restrictive inclusion/exclusion criteria are usually applied to recruitment of the sample. However, highly selected, ‘pure' study populations might not be fully representative of the larger, unselected patient populations attending the outpatient clinics in a real-life setting. As a consequence, the extrapolation of RCT results to everyday clinical practice could be potentially limited.

Our group has recently demonstrated that, in real-life settings, >80% of subjects with chronic obstructive pulmonary disease (COPD) are currently treated by protocols based on results of RCTs for which they would not have been eligible [1]. This is mainly due to the presence of comorbidities that may represent exclusion criteria for RCTs of COPD [1]. In asthma, the association with comorbidities only partially resembles that of COPD: on the one hand, a coexistence with cardiovascular diseases, cerebrovascular diseases, depression, diabetes mellitus, dyslipidaemia, or osteoporosis is less frequent [2]; on the other hand, asthma is strongly associated with allergic rhinitis and gastro-oesophageal reflux disease [2], which are not considered exclusion criteria for RCTs. Most importantly, because cigarette smoking is associated with asthma instability, an increased exacerbation rate, and impaired lung function, active cigarette smoking is often a requisite for exclusion from RCTs of asthma [3]. However, the prevalence of smoking habit in asthmatics parallels that in healthy individuals, approaching 30% of the population [4]. Based on these assumptions, we hypothesized that a consistent proportion of patients with mild-to-moderate asthma seen in the ‘real-life' setting of an ordinary asthma clinic would not be enrolled in RCTs of inhaled medications due to the occurrence of the above-mentioned exclusion criteria.

To test this hypothesis, we retrospectively reviewed the clinical charts of consecutive asthmatic outpatients seen at our clinical centre in order to explore to what extent the population enrolled in RCTs is representative of the ‘real-life' setting of an ordinary asthma clinic.

Consecutive asthmatic patients seen between January 2011 and August 2012 at the Outpatient Clinic for Respiratory Diseases of the University of Palermo, Italy, were included in the analysis. The diagnosis of asthma had been confirmed by clinical and functional assessments according to the GINA guidelines [5]. Clinical evaluation included smoking habit, occurrence of symptoms, respiratory functional measurements, and current treatment. Patients undergoing two or more visits during the observational period were considered only once, and the spirometry with the best values was retained for this analysis. Patients aged <15 years and those with an FEV₁ percentage predicted <50% were excluded.

This retrospective observational study was based on the analysis of the medical records of our asthmatic outpatients. First, variables that may represent criteria for inclusion into or exclusion from the RCTs of asthma were identified. The criteria were selected from the RCTs conducted to test the efficacy of inhalation combination treatment (i.e. inhaled corticosteroid plus bronchodilator) for asthma in adults, on which the current GINA guidelines are periodically updated [5]. We arbitrarily decided to include only trials with ≥400 randomized participants and with exclusion conditions that are not changeable with time, such as smoking habit and chronic diseases. For these reasons, acute conditions were not considered for the present analysis. In this context, 12 studies were identified [6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]. Of these, 5 studies [6, 8, 9, 14, 15] reported ‘smoking >10 pack-years' as an exclusion criterion[,] 4 studies [8, 9, 10, 14] used ‘other significant concomitant diseases' or ‘regular use of other medications' as exclusion criteria, 1 study [17] explicitly included smokers[,] and 5 studies [7, 11, 12, 13, 16] did not provide any information on smoking habit or the occurrence of comorbidities.

In the second phase, the selected criteria were applied to consecutive asthmatic outpatients in a stepwise approach, yielding a population of outpatients who were potentially eligible for RCTs. Figure 1 displays the proportion of patients potentially eligible for an RCT. Two approaches were designed, with the aim of specifically addressing the contribution of smoking exposure; thus, model 2 differs from model 1 as it does not incorporate smoking habit as an exclusion criterion.

The statistical analysis was made using SPSS software (release 21 for Windows). Data are presented as means ± SD or as proportions of the entire sample. The effect of ageing was tested by subdividing the sample into age groups of 10 years, and a logistic regression analysis was performed to calculate the odds to be excluded from RCTs for each age group, using the youngest age group (15-25 years) as the reference. Differences at p values <0.05 were considered to be statistically significant.

A total of 1,909 consecutive outpatients with asthma were included in the analysis. The study sample consisted of 1,190 (62.3%) females and 719 (37.7%) males. The mean age was 43.2 ± 16.5 years (range: 16-88), including 225 patients (11.8%) aged ≥65 years and 20 patients (1.0%) aged ≥80 years. The mean FEV₁ percentage predicted was 97.5 ± 15.8%, and the mean FEV₁/FVC ratio was 0.78.3 ± 0.09. Subjects were on inhaled corticosteroids (ICS; 14.3% of the study population), ICS in fixed combination with long-acting β₂-agonists (55.2%), short-acting β₂-agonists as rescue medication (78%), or leukotriene antagonists (montelukast) with or without inhaled drugs (22.3%). The use of ICS and montelukast did not differ between adult (15-64 years) and elderly asthmatics (≥65 years). On the contrary, the percentage of patients using ICS plus long-acting β₂-agonists was higher in the elderly population than in the adult one (79.1 vs. 52.0%; χ² = 58.8; p < 0.001). In contrast, the percentage of patients using short-acting β₂-agonists was lower in the elderly population than in the adult one (67.6 vs. 79.4%; χ² = 16.4; p < 0.001).

Cigarette smoking >10 pack-years (occurring in 34.4% of the population) represented the most common criterion for exclusion from RCTs in the entire sample. Other causes for potential exclusion from RCTs were as follows (sorted in descending order): anxiety or depression (3.3%); respiratory conditions other than asthma/COPD/obstructive sleep apnoea syndrome (2.4%); recent (<1 year) acute myocardial infarction (2.3%); arrhythmia, mainly atrial fibrillation (2.3%); obstructive sleep apnoea syndrome (1.7%); chronic artery disease (1.2%); coexisting COPD (0.9%); chronic heart failure (0.9%); cerebrovascular disease (0.9%); diabetes (0.8%); active malignancy, in any site (0.7%), and other severe chronic diseases/miscellanea (2.3%).

By applying model 1 (fig. 1, left panel), the study population underwent a progressive selection. The application of the exclusion criterion ‘smoking >10 pack-years' caused a reduction in the number of asthmatic subjects from 1,909 to 1,254 (-34.3% from the initial sample). The application of the criterion ‘absence of respiratory diseases other than asthma' led to a decrease in the absolute number of enrolled subjects to 1,186 (-37.9% from the initial sample). The introduction of the parameter ‘absence of cardiovascular and cerebrovascular diseases' resulted in a decrease in the number of eligible patients to 1,136 (-40.5% from the initial sample), and the criterion ‘absence of anxiety or depression' led to a total of 1,106 eligible subjects (-42.1% from the initial sample). Finally, by removing subjects who presented with at least one of the other severe comorbidities that were chosen as exclusion criteria (see Subjects and Methods), a total population of 1,085 individuals with asthma remained that was eligible for clinical pharmacological trials (-43.2% from the initial sample). Model 2 (fig. 1, right panel) did not consider the condition ‘never smokers or <10 pack-years' as an inclusion criterion. By applying this model, a total population of 1,658 individuals with asthma resulted to be eligible for clinical pharmacological trials (-13.1% from the initial sample).

Since the prevalence of comorbidities increases with age, we repeated the above-described analysis by age groups. As expected, in non-smoking patients with asthma (model 1; fig. 2a), the highest proportion of patients who would have been excluded from RCTs was observed in the group of patients aged >85 years (57.1%), and the lowest proportion in the group aged 15-25 years (28.9%). The effect of ageing was even more evident in model 2 (fig. 2b), in which only comorbidities (and not smoking history) were considered exclusion criteria. Again, the proportion of patients who would have been excluded from RCTs appears to steadily increase with age, with a maximum of 57.1% among patients >85 years of age and a minimum of 3.6% among those aged 15-25 years.

A logistic regression analysis was carried out to calculate the odds to be excluded from RCTs according to decade of age, using the youngest age group (15-25 years) as the reference. As depicted in figure 3a (model 1) and figure 3b (model 2), the probability of being excluded from an RCT increases with age. In model 1, the odds ratio ranges from 1.60 (age group: 35-44 years) to 3.20 (age group: 75-84 years). In contrast, in model 2, the odds ratio steadily rises from 1.37 (age group: 25-34 years) to 35.22 (age group: >85 years).

In our retrospective investigation on a large population of individuals with an established diagnosis of mild/moderate asthma, we found that >40% of outpatients would not be eligible for the majority of RCTs, with this proportion raising to 50% in elderly patients. The results of the present study raise concerns on the external validity of RCTs investigating the management of asthmatic outpatients, particularly in elderly patients, and suggest that complementary studies should be conducted in non-selected populations, including smokers and asthmatics with comorbidities.

Studies aimed at exploring to what extent RCT evidence applies to unselected asthmatic patient populations are scarce. In a multicentre study conducted by Herland et al. [18], the percentage of eligible patients for an RCT was equal to 3.3% of the examined sample with ‘pure' asthma (i.e. not mixed with COPD). In a study conducted by Travers et al. [19], the percentage of eligible subjects for an RCT of asthma ranged from 0 to 43% (median: 6). The present study shows higher percentages of eligible patients, very likely due to differences in the selection of inclusion/exclusion criteria. Our study only selected chronic coexisting conditions (i.e. not changeable or not limited in time), since it was aimed at exploring the proportion of patients for whom we have no scientific evidence from RCTs. We were interested in chronic or irreversible conditions that represent a permanent exclusion criterion for an RCT; patients with an acute event should be referred to RCTs specifically tailored to acute asthma, which are not considered in the current study. As an example, our study did not consider acute respiratory infections or sudden need of oral corticosteroids. Indeed, the treatment of acute and chronic stages of the disease differs, and as a consequence the management of acute asthma should be based on RCTs that are different from those considered in the present study. Similarly, our study did not consider patients with an FEV₁ percentage predicted <50%, since these patients are normally included in RCTs of severe asthma; therefore, the appropriate treatment for severe asthma should rely on RCTs that have not been considered for the purpose of the present study. Moreover, compared with the studies by Herland et al. [18] and Travers et al. [19], the current study included a larger number of subjects with a clinically and functionally established diagnosis of asthma.

The most frequent criterion that would lead to the exclusion from RCTs was cigarette smoking >10 pack-years, which was observed in 34% of the population. The high prevalence of smoking in our population of mild/moderate asthmatics is consistent with the results of a previous study indicating that the prevalence of smoking among asthmatics resembles that in the general population, ranging from 17 to 35% [4]. Despite these observations, current or former heavy smokers are usually excluded from RCTs of inhalation therapy in asthma, a condition that clearly reduces the strength of the information on the best approach to managing smoker asthmatics [3]. Indeed, in asthmatic patients, cigarette smoking is associated with an accelerated decline in lung function [20], worse asthma control and quality of life [21], and a lack of response to treatment [22].

The association between asthma and comorbidities has largely been described [2, 23, 24]. A cross-sectional study conducted by general practitioners in Italy [2] showed a weak association between asthma and depression, diabetes mellitus, osteoporosis, and gastro-oesophageal reflux disease. In addition, people with asthma have an increased risk of disturbed sleep [25]. The relationship between asthma and cardiovascular diseases has been only partially explored; asthma and atherosclerosis may share inflammatory patterns [26], and coronary heart disease seems the major cardiovascular condition associated with asthma [27]. Nevertheless, asthmatics with cardiovascular diseases are often excluded from RCTs, probably because cardiovascular complications in asthmatic patients could be attributed to asthma treatment [28, 29]. Anxiety and depression may deserve special attention in asthmatics. The concomitance of asthma and anxiety and/or depression is related to poor asthma control [30], possibly due to low adherence to treatment [31]. Since RCTs are usually designed to exclude patients with poor adherence to treatment, patients with anxiety and depression could be systematically excluded from RCTs, even if these conditions are not listed among the exclusion criteria.

The issue of the proportion of individuals with asthma who would have met the eligibility criteria for the RCTs appears particularly relevant in elderly patients. Little is known about the age-related differences in external validity of the results of RCTs. In this regard, subgroup analyses of different age classes have never been carried out in previous reports on this topic [18, 19]. The current study showed that 1 out of 2 asthmatics aged >65 years would have been excluded from RCTs. These observations raise concerns on the generalizability of RCT results to this age group, mostly because of the complexity of assessing the side effects of asthma medications.

Asthma per se is associated with an increased risk of comorbidity regardless of age. The concomitance of multiple illnesses is typical of the elderly patients, as a result of two processes: the association between age and incidence of degenerative diseases and the development over time of complications of the existing diseases. Indeed, comorbidity is considered one of the hallmarks of geriatric patients, and a fundamental component of their complexity. In a cohort of patients aged ≥65 years, Soriano et al. [32] reported an association between asthma and diseases of different organ systems (especially cardiac and respiratory). Moreover, in elderly patients, unintentional non-adherence to inhalation therapy represents a complex problem often related to comorbidities requiring multiple medications [33, 34].

The issue underlying the question ‘to whom do the results of this trial apply' was addressed by Rothwell [35], who summarized issues concerning the external validity of randomised RCTs. RCTs and systematic reviews are the most reliable methods of determining treatment effects, but their external validity might be undermined by highly selective recruitment of individuals not truly representative of the wider spectrum of patients seen in clinical practice.

Our findings are by no means intended to argue against RCTs or asthma guidelines; rather, they encourage the design of ‘pragmatic studies' in order to extend the applicability of RCT results to real-life settings. In this scenario, pragmatic studies (or real-life studies), designed to assess the efficacy of a therapeutic intervention in conditions that mimic everyday clinical practice, are gaining widespread recognition and support among clinicians [36]. Pragmatic studies are expected to be conducted after RCTs, and not instead of RCTs, with the smallest number of exclusion criteria, on large samples, with patient-centred outcomes, and with clinical interventions more reflective of those used in routine care. Recently, Wong et al. [37] discussed the applicability of guidelines for asthma and COPD to real-life settings and pointed out that only 6% of the published studies on asthma treatment are categorized as ‘observational' (updated August 2013). In addition, by the application of the Pragmatic-Explanatory Continuum Indicator Summary (PRECIS) approach, they showed that the external applicability of RCT results may be quite heterogeneous [37, 38]. The same features that ensure the internal validity of an RCT can also severely limit its external validity, as pointed out by Chetta and Olivieri [39].

Our study has the inherent limitation of having been conducted on a population of Caucasians, and its findings cannot be extended to other ethnic groups. In addition, our outpatient population of asthmatics is likely to be less representative of the population than the entire population of asthmatics. However, the sample of subjects included in this study is quite similar to those of the subjects enrolled in RCTs, making our results even more significant than those that would be obtained from the general population of asthmatics.

The results of our retrospective study show that, in real-life settings, >40% of subjects with asthma are currently treated by protocols based on results of RCTs for which they would not have been eligible. This proportion increases in elderly patients with comorbidities. Indeed, comorbidities represent permanent factors that render patients ineligible for RCTs. Under this condition, for such patients (not rare cases in real life), we may not have enough scientific evidence supporting their pharmacological management. These findings limit the external validity of RCTs - which, nevertheless, remain the cornerstone of evidence-based medicine. Pragmatic studies should be implemented in order to assess not only whether a drug works in patients with asthma but also whether a drug works in those patients with asthma encountered in everyday practice.

The authors have nothing to disclose.