Chapter 7 Estimating the effect of the 2005 change in BCG policy in England: A retrospective cohort study
7.1 Introduction
In 2005, England changed from universal Bacillus Calmette–Guérin (BCG) vaccination of school-age children to targeted BCG vaccination of high-risk children at birth. In this chapter I aimed to assess the effects of this change in vaccination policy on the populations targeted by each vaccination scheme.
I combined notification data from the Enhanced TB Surveillance (ETS) system, with demographic data from the Labour Force Survey (LFS) to construct retrospective cohorts of individuals in England relevant to both the universal, and targeted vaccination programmes between Jan 1, 2000 and Dec 31, 2010. For each cohort, I estimated incidence rates over a 5 year follow-up period and used Poisson and negative binomial regression models in order to estimate the impact of the change in policy on TB. This work was adapted from [76]24 supervised by Hannah Christensen and Ellen Brooks-Pollock.[76] Nicky Welton provided guidance on the statistical methods used.
7.2 Background
In 2005 England changed its Bacillus Calmette–Guérin (BCG) vaccination policy against tuberculosis (TB) from a universal programme aimed at 13 and 14 year olds to a targeted programme aimed at high-risk neonates (see Chapter 2). High risk babies are identified by local TB incidence and by the parents’ and grandparents’ country of origin. The change in policy was motivated by evidence of reduced TB transmission,[20,30,31] and high effectiveness of the BCG vaccine in children,[4,23,24] and variable effectiveness in adults.[27] Little work has been done to evaluate the impact of this change in vaccination policy.
Globally, several countries with low TB incidence have moved from universal vaccination, either of those at school-age or neonates, to targeted vaccination of neonates considered at high-risk of TB (see Chapter 2).[5] In Sweden, which discontinued universal vaccination of neonates in favour of targeted vaccination of those at high risk, incidence rates in Swedish-born children increased slightly after the change in policy.[74] In France, which also switched from universal vaccination of neonates to targeted vaccination of those at high-risk, a study found that targeted vaccination of neonates may have reduced coverage in those most at risk.[75]
The number of TB notifications in England increased from 6929 in 2004 to 8280 in 2011 but has since declined to 5137 in 2017 (see Chapter 4).[20] A recent study found that this reduction may be linked to improved TB interventions.[88] Directly linking trends in TB incidence to transmission is complex because after an initial infection an individual may either develop active disease, or enter a latent stage which then may later develop into active disease. Incidence in children is a proxy of TB transmission, because any active TB disease in this population is attributable to recent transmission. Using this approach it is thought that TB transmission has been falling in England for the last 5 years, a notion supported by strain typing.[20] However, this does not take into account the change in BCG policy, which is likely to have reduced incidence rates in children.
Although the long term effects of BCG vaccination such as reducing the reactivation of latent cases and decreasing onwards transmission are not readily detectable over short time scales the direct effects of vaccination on incidence rates can be estimated in vaccinated populations, when compared to comparable unvaccinated populations.[89] Here, I aimed to estimate the impact of the 2005 change in BCG policy on incidence rates, in both the UK and non-UK born populations, directly affected by it.
7.3 Methods
7.3.1 Data source
Data on all notifications from the ETS system from Jan 1, 2000 to Dec 31, 2015 were obtained from Public Health England (PHE). The ETS is maintained by PHE, and contains demographic, clinical, and microbiological data on all notified cases in England (see Chapter 4). A descriptive analysis of TB epidemiology in England is published each year, which fully details data collection and cleaning.[20]
I obtained yearly population estimates from the April to June LFS for 2000-2015. The LFS is a study of the employment circumstances of the UK population, and provides the official measures of employment and unemployment in the UK (see Chapter 4). Reporting practices have changed with time so the appropriate variables for age, country of origin, country of birth, and survey weight were extracted from each yearly extract, standardised, and combined into a single data-set (see Section 4.2.2).
7.3.2 Constructing Retrospective cohorts
I constructed retrospective cohorts of TB cases and individuals using the ETS and the LFS. Tuberculosis cases were extracted from the ETS based on date of birth and date of TB notification.
Cohort 1: individuals aged 14 years between 2000 and 2004, who were notified with TB while aged between 14 and 19 years.
Comparison cohort 1: individuals aged 14 years between 2005 and 2010, who were notified with TB while aged between 14 and 19 years.
Cohort 2: individuals born between 2005 and 2010, who were notified with TB while aged 0 to 5 years.
Comparison cohort 2: individuals born between 2000 and 2004, who were notified with TB while aged 0 to 5 years.
Cohorts were stratified by vaccination programme using age criteria and then stratified further by whether the scheme was in place during the time period they entered the study. Each cohort was further stratified by UK birth status, with both non-UK born and UK born cases assumed to have been exposed to England’s vaccination policy. Corresponding population cohorts were calculated using the LFS population estimates, resulting in eight population level cohorts, each with 5 years of follow-up (Table 7.1).
| Cohort | Vaccination programme | Eligible for the programme* | Birth status | Age at study entry | Year of study entry |
|---|---|---|---|---|---|
| Cohort 1 | Universal | Yes | UK born | 14 | 2000-2004 |
| Comparison cohort 1 | Universal | No | UK born | 14 | 2005-2010 |
| Cohort 1 | Universal | Yes | Non-UK born | 14 | 2000-2004 |
| Comparison cohort 1 | Universal | No | Non-UK born | 14 | 2005-2010 |
| Comparison cohort 2 | Targeted | No | UK born | Birth | 2000-2004 |
| Cohort 2 | Targeted | Yes | UK born | Birth | 2005-2010 |
| Comparison cohort 2 | Targeted | No | Non-UK born | Birth | 2000-2004 |
| Cohort 2 | Targeted | Yes | Non-UK born | Birth | 2005-2010 |
| * Eligible signifies that the cohort fit the criteria for the programme | |||||
| and entered the study during the time period it was in operation | |||||
| not that the cohort was vaccinated by the programme. |
7.4 Statistical methods overview
I estimated incidence rates (with 95% confidence intervals) by year, age and place of birth as (number of cases) divided by (number of individuals of corresponding age) (see Chapter 4). UK birth status was incomplete, with some evidence of a missing not at random mechanism (MNAR). I imputed the missing data using a gradient boosting method (see Section 7.4.2). I then used descriptive analysis to describe the observed trends in age-specific incidence rates over the study period, comparing incidence rates in the study populations relevant to both vaccination programmes before and after the change in BCG policy.
I calculated Incidence Rate Ratios (IRRs) for the change in incidence rates associated with the change in BCG vaccination policy (modelled as a binary breakpoint at the start of 2005) for both the UK born and non-UK born populations that were relevant to the universal programme, and for the targeted programme using a series of increasingly complex models. I considered the following covariates: age,[20,27] incidence rates in both the UK born and non-UK born who were not in the age group of interest,[20] and year of study entry (as a random intercept). I first investigated a univariable Poisson model, followed by combinations of covariates (Table 7.2). I also investigated a negative binomial model adjusting for the same covariates as in the best fitting Poisson model. The models were estimated with a Bayesian approach using Markov Chain Monte Carlo (MCMC), with default weakly informative priors (see Section 7.4.3). Model fit, penalised by model complexity, was assessed using the leave one out cross validation information criterion (LOOIC) and its standard error.[90] Models were ranked by goodness of fit, using their LOOIC, with a smaller LOOIC indicating a better fit to the data after adjusting for the complexity of the model. No formal threshold for a change in the LOOIC was used, with changes in the LOOIC being evaluated in the context of their standard error. The inclusion of the change in policy in the best fitting model was tested by refitting the model excluding the change in policy and estimating the improvment in the LOOIC. Once the best fitting model had been identified I estimated the number of cases prevented, from 2005 until 2015, for each vaccination programme in the study population relevant to that programme (see Section 7.4.4).
| Model | Description |
|---|---|
| Model 1 | Poisson model adjusting for no fixed effects. |
| Model 2 | Poisson model adjusting with fixed effects for the change in policy. |
| Model 3 | Poisson model adjusting with fixed effects for the change in policy and incidence rates in the UK born. |
| Model 4 | Poisson model adjusting with fixed effects for the change in policy and incidence rates in the non-UK born. |
| Model 5 | Poisson model adjusting with fixed effects for the change in policy and incidence rates in the UK born and non-UK born populations. |
| Model 6 | Poisson model adjusting with fixed effects for the change in policy and age. |
| Model 7 | Poisson model adjusting with fixed effects for the change in policy, age, and incidence rates in the UK born. |
| Model 7 (Negative Binomial) | Negative binomial model adjusting with fixed effects for the change in policy, age, and incidence rates in the UK born. |
| Model 8 | Poisson model adjusting with fixed effects for the change in policy, age, and incidence rates in the non-UK born. |
| Model 8 (Negative Binomial) | Negative binomial model adjusting with fixed effects for the change in policy, age, and incidence rates in the non-UK born. |
| Model 9 | Poisson model adjusting with fixed effects for the change in policy, age, and incidence rates in the UK born and non-UK born populations. |
| Model 10 | Poisson model with a random intercept for year of study entry, adjusting for no fixed effects. |
| Model 11 | Poisson model with a random intercept for year of study entry, adjusting with fixed effects for the change in policy. |
| Model 12 | Poisson model with a random intercept for year of study entry, adjusting with fixed effects for the change in policy and incidence rates in the UK born. |
| Model 13 | Poisson model with a random intercept for year of study entry, adjusting with fixed effects for the change in policy and incidence rates in the non-UK born. |
| Model 14 | Poisson model with a random intercept for year of study entry, adjusting with fixed effects for the change in policy and incidence rates in the UK born and non-UK born populations. |
| Model 15 | Poisson model with a random intercept for year of study entry, adjusting with fixed effects for the change in policy and age. |
| Model 16 | Poisson model with a random intercept for year of study entry, adjusting with fixed effects for the change in policy, age, and incidence rates in the UK born. |
| Model 16 (Negative Binomial) | Negative binomial model with a random intercept for year of study entry, adjusting with fixed effects for the change in policy, age, and incidence rates in the UK born. |
| Model 17 | Poisson model with a random intercept for year of study entry, adjusting with fixed effects for the change in policy, age, and incidence rates in the non-UK born. |
| Model 17 (Negative Binomial) | Negative binomial model with a random intercept for year of study entry, adjusting with fixed effects for the change in policy, age, and incidence rates in the non-UK born. |
| Model 18 | Poisson model with a random intercept for year of study entry, adjusting with fixed effects for the change in policy, age, and incidence rates in the UK born and non-UK born populations. |
7.4.1 Implementation overview
R 3.5.0 was used for all analysis.[56] Reproducibility was ensured by using R package infrastructure25. Missing data imputation using a gradient boosting model (GBM) was implemented using the h2o package (see Section 7.4.2).[91] Incidence rates, with 95% confidence intervals, were calculated using the epiR package (see Chapter 4).[60] The brms package,[92] and Stan,[93] was used to perform Markov Chain Monte Carlo (MCMC). Models were run until convergence (4 chains with a burn in of 10,000, and 10,000 sampled iterations each), with convergence being assessed using trace plots and the R hat diagnostic.[93] All numeric confounders were centered and scaled by their standard deviation, and age was adjusted for using single year of age categories.
7.4.2 Imputation of UK birth status
As I was imputing a single variable, I reformulated the imputation as a categorical prediction problem. This allowed the use of more complex, high-performing models compared to those usually used for imputation, whilst also allowing the results to be validated using predictive modelling performance metrics. I included year of notification, sex, age, PHE Centre (PHEC), occupation, ethnic group, Index of Multiple Deprivation (2010) categorised into five groups for England (IMD rank), and risk factor count (risk factors considered; drug use, homelessness, alcohol misuse/abuse and prison). However, I could not account for a possible missing not at random mechanism not captured by these covariates. To train the model I first split the data with complete UK birth status into a training set (80%), a calibration set (5%), and a test set (15%). I then fit a gradient boosted machine with 10,000 trees, early stopping (at a precision of \(1 \times 10^{-5}\), with 10 stopping rounds), a learning rate of 0.1, and a learn rate annealing of 0.99. Gradient boosted machines are a tree based method that can incorporate complex non-linear relationships and interactions.[91] Much like a random forest model they work by ensembling a group of trees, but unlike a random forest model each tree is additive aiming to reduce the residual loss from previous trees. Once the model had been fit to the training set I performed platt scaling (fits a logistic regression model to model predictions in order to return a probability) using the calibration dataset. The fitted imputation model had a logloss (the negative of the log likelihood) of 0.28 on the test set, with an area under the curve (AUC) of 0.93, both of which indicate robust performance on unseen data. I found that ethnic group was the most important variable for predicting UK birth status, followed by age and PHEC.
Using the fitted model I predicted the birth status for notifications where this was missing, using the F1 optimal threshold as the probability cut-off. It is common to impute missing values multiple times, to account for within- and between imputation variability. However, I considered this unnecessary for this analysis as the amount of missing data was small, this analysis considered only aggregate counts, my model metrics indicated a robust level of performance out of bag and any unaccounted for uncertainty would be outweighed by the uncertainty in the population denominator.[88] I found that cases with imputed birth status had a similar proportion of UK born to non-UK born cases as in the complete data (Table 7.3).
| Status | Birth Status | Proportion of Cases (%) | Cases |
|---|---|---|---|
| Complete | 106765 | ||
| UK Born | 27.3 | 29096 | |
| Non-UK Born | 72.7 | 77669 | |
| Imputed | 8055 | ||
| UK Born | 32.7 | 2634 | |
| Non-UK Born | 67.3 | 5421 |
Inclusion of imputed values for UK birth status should reduce bias caused by any missing at random mechanism captured by predictors included in the model. Graphical evaluation of UK birth status indicated that missingness has reduced over time, indicating a missing at random mechanism (see Chapter 4). If only the complete case data had been included in the analysis then incidence rates would have reduced over the study period due to this mechanism, this may have biased the estimate of the impact of the change in policy.
7.4.3 Prior choice
Default weakly informative priors were used based on those provided by the brms package.[92] For the population-level effects this was an improper flat prior over the reals. For both the standard deviations of group level effects and the group level intercepts this was a half student-t prior with 3 degrees of freedom and a scale parameter that depended on the standard deviation of the response after applying the link function.
7.4.4 Estimating the magnitude of the estimated impact of the change in BCG policy
I estimated the magnitude of the estimated impact from the change in BCG policy by applying the IRR estimates from the best fitting model for each cohort to the observed number of notifications from 2005 until 2015 in the study population. For the cohorts relevant to the universal school-age vaccination scheme I estimated the number of prevented cases by first aggregating cases (\(C_0\)) and then using the following equation,
\[\begin{equation} C^i_P = C_0 (1 - I^i),\ \text{Where}\ i = e,\ l,\ u. \end{equation}\]
Where \(C^i_P\) is the predicted number of cases prevented using the median (\(e\)), 2.5% bound (\(l\)) and 97.5% bound (\(u\)) of the IRR estimate (\(I^i\)). For the cohorts relevant to the targeted high-risk neonatal scheme I used a related equation,
\[\begin{equation} C^i_P = C_{NE}(1 - I^i),\ \text{Where}\ i = e,\ l,\ u. \end{equation}\]
Where \(C_{NE}\) is the number of cases observed assuming that the cohort was not exposed to targeted high-risk neonatal vaccination. As from 2005 onwards this cohort were in fact exposed to this vaccination scheme an additional step was required. This first required calculating the number of cases that would be expected if the cohort had not been exposed to the scheme,
\[\begin{equation} C_{NE} = \frac{C_0}{I^i} \end{equation}\]
Then combining this with the previous equation so that \(C^i_P\) can be estimated using observed data (\(C_0\)),
\[\begin{equation} C^i_P = \frac{C_0(1 - I^i)}{I^i},\ \text{Where}\ i = e,\ l,\ u. \end{equation}\]
7.5 Results
7.5.1 Descriptive analysis
During the study period there were 114,820 notifications of TB in England, of which 93% (106765/114820) had their birth status recorded. Of notifications with a known birth status 27% (29096/106765) were UK born, in comparison to 33% (2634/8055) in cases with an imputed birth status (see Chapter 4 for details). There were 1729 UK born cases and 2797 non-UK born cases in individuals relevant to the universal schools scheme, and 1431 UK born cases and 238 non-UK born cases relevant to the targeted neonatal scheme, who fit the age criteria during the study period. Univariable evidence for differences between mean incidence rates before and after the change in BCG policy in the UK born was weak. In the non-UK born incidence rates were lower after the change in BCG policy in both the cohort relevant to the universal school-age scheme and the cohort relevant to the targeted neonatal scheme (Figure 7.1).
Figure 7.1: Mean incidence rates per 100,000, with 95% confidence intervals for each retrospective cohort, stratified by the vaccination policy and UK birth status. The top and bottom panels are on different scales in order to highlight trends in incidence rates over time.
Trends in incidence rates varied by age group and UK birth status. From 2000 until 2012 incidence rates in the UK born remained relatively stable but have since fallen year on year. In comparison, incidence rates in the non-UK born increased from 2000 until 2005, since when they have also decreased year on year (see Chapter 4). In 14-19 year old’s, who were UK born, incidence rates remained relatively stable throughout the study period, except for the period between 2006 to 2009 in which they increased year on year. This trend was not observed in the non-UK born population aged 14-19, where incidence rates reached a peak in 2003, since when they have consistently declined. In those aged 0-5, who were UK born, incidence rates also increased year on year after the change in BCG policy, until 2008 since when they have declined. This does not match with the observed trend in incidence rates in the non-UK born population, aged 0-5, in which incidence rates declined steeply between 2005 and 2006, since when they have remained relatively stable (Figure 7.2).
Figure 7.2: Incidence rates (with 95% confidence intervals) per 100,000 per year for UK born population and non-UK born population, aged 0-5 and therefore directly affected by the targeted neonatal vaccination programme, and aged 14-19 and therefore directly affected by the universal school-age scheme.
7.5.2 Adjusted estimates of the effects of the change in policy on school-age children
In the UK born cohort relevant to universal vaccination there was some evidence, across all models that adjusted for age, that ending the scheme was associated with a modest increase in TB rates (Table 7.4). Using the LOOIC goodness of fit criteria the best fitting model was found to be a negative binomial model that adjusted for the change in policy, age, and incidence rates in the UK born (Table 7.5). In this model there was some evidence of an assocation between the change in policy and an increase in incidence rates in those at school-age who were UK born, with an IRR of 1.08 (95%CI 0.97, 1.19). Dropping the change in policy from the model resulted in a small decrease in the LOOIC (0.52 (SE 2.63)) but the change was too small, with too large a standard error, to conclusively state that the excluding the change in policy from the model improved the quality of model fit. I found that it was important to adjust for UK born incidence rates, otherwise the impact from the change in BCG vaccination policy was over-estimated.
For the comparable non-UK born cohort who were relevant to the universal vaccination there was evidence, in the best fitting model, that ending the scheme was associated with a decrease in incidence rates (IRR: 0.74 (95%CI 0.61, 0.88)). The best fitting model was a negative binomial model which adjusted for the change in policy, age, incidence rates in the non-UK born, and year of eligibility as a random effect (Table 7.5). I found that omitting the change in policy from the model resulted in poorer model fit (LOOIC increase of 3.02 (SE 3.52)), suggesting that the policy change was an important factor explaining changes in incidence rates, after adjusting for other covariates. All models that adjusted for incidence rates in the UK born or non-UK born estimated similar IRRs (Table 7.6).
| Model | IRR (CI 95%)* | Policy Change | Age | UK born rates | Non-UK born rates | Year of study entry | DoF** | LPD*** | LOOIC (se)**** |
|---|---|---|---|---|---|---|---|---|---|
| Model 7 (Negative Binomial) | 1.08 (0.97, 1.19) | Yes | Yes | Yes | No | No | 9 | -211 | 439 (10) |
| Model 7 | 1.08 (1.00, 1.17) | Yes | Yes | Yes | No | No | 8 | -211 | 443 (14) |
| Model 9 | 1.12 (1.01, 1.25) | Yes | Yes | Yes | Yes | No | 9 | -210 | 445 (14) |
| Model 16 | 1.08 (0.97, 1.21) | Yes | Yes | Yes | No | Yes | 20 | -207 | 445 (14) |
| Model 18 | 1.12 (0.97, 1.28) | Yes | Yes | Yes | Yes | Yes | 21 | -207 | 447 (15) |
| Model 8 | 1.16 (1.04, 1.29) | Yes | Yes | No | Yes | No | 8 | -213 | 449 (17) |
| Model 6 | 1.06 (0.98, 1.15) | Yes | Yes | No | No | No | 7 | -215 | 452 (17) |
| Model 17 | 1.15 (1.00, 1.32) | Yes | Yes | No | Yes | Yes | 20 | -209 | 452 (17) |
| Model 15 | 1.06 (0.94, 1.20) | Yes | Yes | No | No | Yes | 19 | -209 | 453 (17) |
| Model 1 | 1.00 (1.00, 1.00) | No | No | No | No | No | 1 | -254 | 513 (26) |
| Model 2 | 1.06 (0.98, 1.14) | Yes | No | No | No | No | 2 | -252 | 515 (25) |
| Model 4 | 1.00 (0.90, 1.10) | Yes | No | No | Yes | No | 3 | -251 | 516 (25) |
| Model 3 | 1.06 (0.98, 1.15) | Yes | No | Yes | No | No | 3 | -252 | 518 (26) |
| Model 5 | 0.98 (0.89, 1.09) | Yes | No | Yes | Yes | No | 4 | -249 | 518 (24) |
| Model 13 | 0.94 (0.78, 1.12) | Yes | No | No | Yes | Yes | 15 | -237 | 518 (27) |
| Model 10 | 1.00 (1.00, 1.00) | No | No | No | No | Yes | 13 | -244 | 521 (28) |
| Model 11 | 1.06 (0.94, 1.20) | Yes | No | No | No | Yes | 14 | -244 | 522 (28) |
| Model 14 | 0.93 (0.78, 1.11) | Yes | No | Yes | Yes | Yes | 16 | -236 | 522 (27) |
| Model 12 | 1.06 (0.93, 1.20) | Yes | No | Yes | No | Yes | 15 | -243 | 526 (28) |
| * Incidence Rate Ratio, with 95% credible intervals, | |||||||||
| ** Degrees of Freedom, | |||||||||
| *** Computed log pointwise predictive density, | |||||||||
| **** Leave one out information criterion, with standard error, |
| Variable | UK born | Non-UK born |
|---|---|---|
| Policy change** | ||
| Pre-change | Reference | Reference |
| Post-change | 1.08 (0.97, 1.19) | 0.74 (0.61, 0.88) |
| Age | ||
| 14 | Reference | Reference |
| 15 | 1.18 (0.98, 1.42) | 1.03 (0.87, 1.22) |
| 16 | 1.24 (1.03, 1.50) | 1.25 (1.07, 1.47) |
| 17 | 1.59 (1.33, 1.91) | 1.40 (1.19, 1.63) |
| 18 | 1.92 (1.60, 2.30) | 1.47 (1.26, 1.73) |
| 19 | 1.80 (1.49, 2.17) | 1.47 (1.24, 1.73) |
| UK born incidence rate (per standard deviation) | 1.08 (1.03, 1.14) | - |
| Non-UK born incidence rate (per standard deviation) | - | 1.11 (1.03, 1.19) |
| Year of study elibility, group level | - | |
| Intercept (standard deviation) | - | 1.13 (1.05, 1.26) |
| Year of study elibility, individual level | - | |
| 2000 | - | 1.10 (0.96, 1.29) |
| 2001 | - | 1.06 (0.93, 1.24) |
| 2002 | - | 1.07 (0.94, 1.25) |
| 2003 | - | 0.90 (0.76, 1.03) |
| 2004 | - | 0.89 (0.75, 1.02) |
| 2005 | - | 0.98 (0.85, 1.12) |
| 2006 | - | 1.13 (0.99, 1.33) |
| 2007 | - | 1.04 (0.91, 1.20) |
| 2008 | - | 0.96 (0.83, 1.09) |
| 2009 | - | 0.95 (0.81, 1.08) |
| 2010 | - | 0.96 (0.82, 1.11) |
| * Incidence Rate Ratio (95% Credible Interval), | ||
| **There was an improvement in the LOOIC score of 0.52 (SE 2.63) from dropping the change in policy from the model in the UK born cohort and a -3.02 (SE 3.52) improvement in the non-UK born cohort. |
| Model | IRR (CI 95%)* | Policy Change | Age | UK born rates | Non-UK born rates | Year of study entry | DoF** | LPD*** | LOOIC (se)**** |
|---|---|---|---|---|---|---|---|---|---|
| Model 17 (Negative Binomial) | 0.74 (0.61, 0.88) | Yes | Yes | No | Yes | Yes | 21 | -228 | 483 (10) |
| Model 17 | 0.74 (0.62, 0.87) | Yes | Yes | No | Yes | Yes | 20 | -223 | 492 (16) |
| Model 18 | 0.73 (0.61, 0.87) | Yes | Yes | Yes | Yes | Yes | 21 | -222 | 493 (16) |
| Model 15 | 0.64 (0.53, 0.78) | Yes | Yes | No | No | Yes | 19 | -224 | 496 (18) |
| Model 16 | 0.65 (0.54, 0.78) | Yes | Yes | Yes | No | Yes | 20 | -223 | 496 (17) |
| Model 8 | 0.79 (0.73, 0.86) | Yes | Yes | No | Yes | No | 8 | -239 | 507 (20) |
| Model 9 | 0.79 (0.72, 0.86) | Yes | Yes | Yes | Yes | No | 9 | -238 | 511 (20) |
| Model 11 | 0.64 (0.52, 0.78) | Yes | No | No | No | Yes | 14 | -241 | 522 (22) |
| Model 10 | 1.00 (1.00, 1.00) | No | No | No | No | Yes | 13 | -241 | 523 (22) |
| Model 12 | 0.64 (0.53, 0.79) | Yes | No | Yes | No | Yes | 15 | -241 | 525 (22) |
| Model 13 | 0.64 (0.52, 0.79) | Yes | No | No | Yes | Yes | 15 | -241 | 526 (23) |
| Model 14 | 0.64 (0.52, 0.79) | Yes | No | Yes | Yes | Yes | 16 | -241 | 530 (23) |
| Model 7 | 0.66 (0.62, 0.70) | Yes | Yes | Yes | No | No | 8 | -248 | 532 (23) |
| Model 6 | 0.65 (0.61, 0.69) | Yes | Yes | No | No | No | 7 | -253 | 539 (27) |
| Model 4 | 0.70 (0.65, 0.76) | Yes | No | No | Yes | No | 3 | -270 | 556 (31) |
| Model 5 | 0.70 (0.64, 0.76) | Yes | No | Yes | Yes | No | 4 | -270 | 559 (31) |
| Model 2 | 0.65 (0.61, 0.69) | Yes | No | No | No | No | 2 | -275 | 561 (33) |
| Model 3 | 0.65 (0.61, 0.69) | Yes | No | Yes | No | No | 3 | -273 | 561 (32) |
| Model 1 | 1.00 (1.00, 1.00) | No | No | No | No | No | 1 | -341 | 692 (51) |
| * Incidence Rate Ratio, with 95% credible intervals, | |||||||||
| ** Degrees of Freedom, | |||||||||
| *** Computed log pointwise predictive density, | |||||||||
| **** Leave one out information criterion, with standard error, |
7.5.3 Adjusted estimates of the effect of the change in policy in those relevant to the targeted neonatal programme
For the UK born cohort relevant to the targeted neonatal vaccination programme (see Section 7.3.2) the evidence of an association between the change in policy and TB incidence was mixed across all models and credible intervals were wide compared to models for the UK born cohort relevant to the universal school-age vaccination programme (Table 7.7). The best fitting model was a Poisson model which adjusted for the change in policy, age, UK born incidence rates, and year of study entry with a random effect (Table 7.8). In this model, there was weak evidence of an association between the change in BCG policy and an decrease in incidence rates in UK born neonates, with an IRR of 0.96 (95%CI 0.82, 1.14). There was weak evidence to suggest that dropping the change in policy from this model improved the quality of the fit, with an improvement in the LOOIC score of 0.92 (SE 1.07). This suggests that the change in policy was not an important factor for explaining incidence rates, after adjusting for covariates. Models which also adjusted for non-UK born incidence rates estimated that the change in policy was associated with no change in incidence rates in the relevant cohort of neonates.
For the comparable non-UK born cohort who were relevant to the targeted neonatal vaccination programme there was evidence, across all models, that the change in policy was associated with a large decrease in incidence rates (IRR: 0.62 (95%CI 0.44, 0.88)) (Table 7.8 in the best fitting model). The best fitting model was a negative binomial model that adjusted for the change in policy, age, and non-UK born incidence rates (Table 7.8). All models which at least adjusted for age estimated comparable effects of the change in policy (Table 7.9).
| Model | IRR (CI 95%)* | Policy Change | Age | UK born rates | Non-UK born rates | Year of study entry | DoF** | LPD*** | LOOIC (se)**** |
|---|---|---|---|---|---|---|---|---|---|
| Model 16 | 0.96 (0.82, 1.14) | Yes | Yes | Yes | No | Yes | 20 | -192 | 415 (12) |
| Model 16 (Negative Binomial) | 0.96 (0.82, 1.13) | Yes | Yes | Yes | No | Yes | 21 | -196 | 415 (10) |
| Model 18 | 0.99 (0.82, 1.18) | Yes | Yes | Yes | Yes | Yes | 21 | -192 | 417 (13) |
| Model 7 | 0.96 (0.88, 1.05) | Yes | Yes | Yes | No | No | 8 | -200 | 420 (15) |
| Model 9 | 1.00 (0.89, 1.12) | Yes | Yes | Yes | Yes | No | 9 | -200 | 422 (15) |
| Model 8 | 1.02 (0.91, 1.15) | Yes | Yes | No | Yes | No | 8 | -203 | 427 (16) |
| Model 6 | 0.95 (0.87, 1.03) | Yes | Yes | No | No | No | 7 | -204 | 428 (16) |
| Model 15 | 0.95 (0.83, 1.09) | Yes | Yes | No | No | Yes | 19 | -198 | 428 (14) |
| Model 17 | 1.02 (0.87, 1.20) | Yes | Yes | No | Yes | Yes | 20 | -198 | 429 (14) |
| Model 14 | 1.10 (0.92, 1.33) | Yes | No | Yes | Yes | Yes | 16 | -206 | 442 (16) |
| Model 5 | 1.08 (0.97, 1.21) | Yes | No | Yes | Yes | No | 4 | -216 | 445 (18) |
| Model 12 | 0.98 (0.83, 1.15) | Yes | No | Yes | No | Yes | 15 | -209 | 448 (17) |
| Model 4 | 1.12 (1.00, 1.24) | Yes | No | No | Yes | No | 3 | -219 | 449 (18) |
| Model 3 | 0.97 (0.89, 1.06) | Yes | No | Yes | No | No | 3 | -219 | 450 (19) |
| Model 13 | 1.14 (0.97, 1.35) | Yes | No | No | Yes | Yes | 15 | -211 | 452 (16) |
| Model 1 | 1.00 (1.00, 1.00) | No | No | No | No | No | 1 | -229 | 462 (21) |
| Model 2 | 0.95 (0.87, 1.03) | Yes | No | No | No | No | 2 | -228 | 463 (20) |
| Model 10 | 1.00 (1.00, 1.00) | No | No | No | No | Yes | 13 | -220 | 466 (19) |
| Model 11 | 0.95 (0.83, 1.09) | Yes | No | No | No | Yes | 14 | -219 | 467 (19) |
| * Incidence Rate Ratio, with 95% credible intervals, | |||||||||
| ** Degrees of Freedom, | |||||||||
| *** Computed log pointwise predictive density, | |||||||||
| **** Leave one out information criterion, with standard error, |
| Variable | UK born | Non-UK born |
|---|---|---|
| Policy change** | ||
| Pre-change | Reference | Reference |
| Post-change | 0.96 (0.82, 1.14) | 0.62 (0.44, 0.88) |
| Age | ||
| 0 | Reference | Reference |
| 1 | 1.39 (1.20, 1.61) | 0.49 (0.30, 0.83) |
| 2 | 1.24 (1.06, 1.44) | 0.49 (0.30, 0.80) |
| 3 | 1.21 (1.03, 1.41) | 0.42 (0.26, 0.68) |
| 4 | 0.90 (0.76, 1.06) | 0.41 (0.25, 0.66) |
| 5 | 0.89 (0.75, 1.06) | 0.27 (0.16, 0.45) |
| UK born incidence rate (per standard deviation) | 1.12 (1.06, 1.18) | - |
| Non-UK born incidence rate (per standard deviation) | - | 1.25 (1.04, 1.51) |
| Year of study elibility, group level | - | |
| Intercept (standard deviation) | 1.13 (1.04, 1.26) | - |
| Year of study elibility, individual level | - | |
| 2000 | 0.83 (0.68, 0.99) | - |
| 2001 | 0.93 (0.79, 1.07) | - |
| 2002 | 1.08 (0.95, 1.28) | - |
| 2003 | 1.07 (0.93, 1.26) | - |
| 2004 | 1.12 (0.97, 1.32) | - |
| 2005 | 1.02 (0.89, 1.17) | - |
| 2006 | 1.02 (0.89, 1.17) | - |
| 2007 | 0.97 (0.83, 1.11) | - |
| 2008 | 1.01 (0.88, 1.15) | - |
| 2009 | 1.01 (0.88, 1.16) | - |
| 2010 | 0.98 (0.85, 1.13) | - |
| * Incidence Rate Ratio (95% Credible Interval), | ||
| **There was an improvement in the LOOIC score of 0.92 (SE 1.07) from dropping the change in policy from the model in the UK born cohort and a -3.45 (SE 4.63) improvement in the non-UK born cohort. |
| Model | IRR (CI 95%)* | Policy Change | Age | UK born rates | Non-UK born rates | Year of study entry | DoF** | LPD*** | LOOIC (se)**** |
|---|---|---|---|---|---|---|---|---|---|
| Model 8 (Negative Binomial) | 0.62 (0.44, 0.88) | Yes | Yes | No | Yes | No | 9 | -138 | 293 (15) |
| Model 8 | 0.64 (0.47, 0.86) | Yes | Yes | No | Yes | No | 8 | -137 | 295 (18) |
| Model 9 | 0.62 (0.45, 0.85) | Yes | Yes | Yes | Yes | No | 9 | -137 | 297 (18) |
| Model 6 | 0.47 (0.38, 0.58) | Yes | Yes | No | No | No | 7 | -139 | 298 (19) |
| Model 7 | 0.48 (0.39, 0.60) | Yes | Yes | Yes | No | No | 8 | -139 | 298 (19) |
| Model 17 | 0.63 (0.44, 0.89) | Yes | Yes | No | Yes | Yes | 20 | -135 | 298 (18) |
| Model 18 | 0.61 (0.42, 0.87) | Yes | Yes | Yes | Yes | Yes | 21 | -135 | 300 (18) |
| Model 15 | 0.47 (0.35, 0.62) | Yes | Yes | No | No | Yes | 19 | -136 | 301 (20) |
| Model 16 | 0.48 (0.36, 0.63) | Yes | Yes | Yes | No | Yes | 20 | -136 | 301 (19) |
| Model 4 | 0.82 (0.61, 1.10) | Yes | No | No | Yes | No | 3 | -147 | 304 (17) |
| Model 5 | 0.78 (0.58, 1.06) | Yes | No | Yes | Yes | No | 4 | -147 | 306 (18) |
| Model 13 | 0.83 (0.59, 1.16) | Yes | No | No | Yes | Yes | 15 | -145 | 308 (18) |
| Model 14 | 0.78 (0.55, 1.12) | Yes | No | Yes | Yes | Yes | 16 | -144 | 310 (19) |
| Model 3 | 0.52 (0.42, 0.64) | Yes | No | Yes | No | No | 3 | -152 | 314 (22) |
| Model 12 | 0.51 (0.38, 0.69) | Yes | No | Yes | No | Yes | 15 | -148 | 317 (23) |
| Model 2 | 0.49 (0.40, 0.61) | Yes | No | No | No | No | 2 | -156 | 319 (22) |
| Model 11 | 0.49 (0.37, 0.65) | Yes | No | No | No | Yes | 14 | -152 | 322 (23) |
| Model 10 | 1.00 (1.00, 1.00) | No | No | No | No | Yes | 13 | -150 | 330 (25) |
| Model 1 | 1.00 (1.00, 1.00) | No | No | No | No | No | 1 | -171 | 346 (27) |
| * Incidence Rate Ratio, with 95% credible intervals, | |||||||||
| ** Degrees of Freedom, | |||||||||
| *** Computed log pointwise predictive density, | |||||||||
| **** Leave one out information criterion, with standard error, |
7.5.4 Magnitude of the estimated impact of the change in BCG policy
I estimate that the change in vaccination policy was associated with preventing 385 (95%CI -105, 881) cases from 2005 until the end of the study period (2015) in the directly impacted populations with 5 years of follow up (Table 7.10). The majority of the cases prevented were in the non-UK born, with cases increasing slightly overall in the UK born. This was due to cases increasing in the UK born at school-age, and decreasing in UK born neonates, although both these estimates had large credible intervals.
| Vaccination Programme (age) | Birth Status | Cases Prevented (95% CI*) | Notified Cases |
|---|---|---|---|
| Universal school-age (14) | -291 (24, -571) | 2364 | |
| UK born | 76 (188, -26) | 969 | |
| Non-UK born | -367 (-165, -546) | 1395 | |
| Targeted high-risk neonates (0) | 94 (-81, 310) | 906 | |
| UK born | 30 (-95, 173) | 800 | |
| Non-UK born | 65 (14, 137) | 106 | |
| Change in Policy** | 385 (-105, 881) | 3270 | |
| UK born | -46 (-284, 199) | 1769 | |
| Non-UK born | 431 (179, 682) | 1501 | |
| *95% CI: 95% Credible Interval, | |||
| ** Estimated total number of cases prevented due to the change in vaccination policy in 2005 |
7.6 Discussion
In the non-UK born I found evidence of an association between the change in BCG policy and a decrease in TB incidence rates in both those at school-age and neonates, after 5 years of follow up. I found some evidence that the change in BCG policy was associated with a modest increase in incidence rates in the UK born population who were relevant to the universal school-age scheme and weaker evidence of a small decrease in incidence rates in the UK born population relevant to the targeted neonatal scheme. Overall, I found that the change in policy was associated with preventing 385 (95%CI -105, 881) cases in the study population, from 2005 until 2015, with the majority of the cases prevented in the non-UK born.
I was unable to estimate the impact of the change in BCG policy after 5 years post vaccination, so both the estimates of the positive and negative consequences are likely to be underestimates of the ongoing impact. TB is a complex disease and the BCG vaccine is known to offer imperfect protection, which has been shown to vary both spatially and with time since vaccination (see Chapter 2).[25,28] By focusing on the impact of the change in policy on the directly affected populations within a short period of time, and by employing a multi-model approach I have limited the potential impact of these issues. This study was based on a routine observational dataset (ETS), and a repeated survey (LFS) both of which may have introduced bias. Whilst the LFS is a robust data source, widely used in academic studies,[45,94,95] it is susceptible to sampling errors particularly in the young, and in the old, which may have biased the estimated incidence rates. As the ETS is routine surveillance system some level of missing data is inevitable (see Chapter 4). However, UK birth status is relatively complete (93% (106765/114820)) and I imputed missing values using an approach which accounted for MAR mechanisms for the variables included in the imputation model. I was unable to adjust for known demographic risk factors for TB, notably socio-economic status,[15,78] and ethnicity.[15,78,82] However, this confounding is likely to be mitigated by the use of multiple cohorts and the adjustment for incidence rates in the UK born and non-UK born. Finally, I have assumed that the effect I have estimated for the change in BCG policy is due to the changes in BCG vaccination policy as well as other associated changes in TB control policy, after adjusting for hypothesised confounders. However, there may have been additional policy changes which I have not accounted for.
Whilst little work has been done to assess the impact of the 2005 change in BCG vaccination several other studies have estimated the impact of changing BCG vaccination policy, although typically only from universal vaccination of neonates to targeted vaccination of high-risk neonates. A previous study in Sweden found that incidence rates in Swedish-born children increased after high-risk neonatal vaccination was implemented in place of a universal neonatal program, this corresponds with our finding that introducing neonatal vaccination had little impact on incidence rates in UK born neonates. Theoretical approaches have indicated that targeted vaccination of those at high-risk may be optimal in low incidence settings.[96] Our study extends this work by also considering the age of those given BCG vaccination, although I was unable to estimate the impact of a universal neonatal scheme as this has never been implemented nationally in England. It has previously been shown that targeted vaccination programmes may not reach those considered most at risk,[97] our findings may support this view as I observed only a small decrease in incidence rates in UK born neonates after the introduction of the targeted neonatal vaccination programme. Alternatively, the effectiveness of the BCG in neonates, in England, may be lower than previously thought as I only observed a small decrease in incidence rates, whilst a previous study estimated BCG coverage at 68% (95%CI 65%, 71%) amongst those eligible for the targeted neonatal vaccination programme.[98] Chapter 5 also found evidence that incidence rates would increase in UK born population relevant to school-age BCG programme.
This study indicates that the change in England’s BCG vaccination policy was associated with a modest increase in incidence in the UK born that were relevant to the school-age vaccination programme, and with a small reduction in incidence in the UK born that were relevant to the high-risk neonatal vaccination programme, although both these estimates had wide credible intervals. I found stronger evidence of an association between the change in policy and a decrease in incidence rates in the non-UK born populations relevant to both programmes. This suggests that the change of vaccination policy to target high-risk neonates may have resulted in an increased focus on high-risk non-UK born individuals who may not have been the direct targets of the vaccination programme. Further validation is required using alternative study designs, but this result should be considered when vaccination policy changes are being considered. These results should be interpreted carefully, especially in the non-UK born, as I could not fully rule out the impact of other TB control measures that may have been changed at the same time as vaccination policy. The severity of TB disease is known to differ across age groups with children having a higher incidence of TB meningitis, which can be severe, compared to other age groups.[20] This variation should also be considered when evaluating these results.
It is well established that interventions against infectious diseases, such as TB, should be evaluated not only for their direct effects but also for future indirect effects via ongoing transmission. Statistical approaches such as those used in this chapter are not appropriate for capturing these future indirect effects, and instead dynamic disease models should be used. In Chapter 8 I develop such a dynamic disease model, Chapter 9 then fits this model to the available data, and Chapter 10 compares the impact of continuing with the BCG school’s scheme post 2005 to universal neonatal vaccination. In addition, this study could not evaluate the impact of the neonatal programme on the high-risk population it targets, due to a lack of reliable data. Improved coverage data for the BCG programme is required to more fully evaluate its ongoing impact.
7.7 Summary
In the non-UK born, I found evidence for an association between a reduction in incidence rates and the change in BCG policy (school-age IRR: 0.74 (95%CI 0.61, 0.88), neonatal IRR: 0.62 (95%CI 0.44, 0.88)).
I found some evidence that the change in BCG policy was associated with a increase in incidence rates in the UK born school-age population (IRR: 1.08 (95%CI 0.97, 1.19)) and weaker evidence of an association with a reduction in incidence rates in UK born neonates (IRR: 0.96 (95%CI 0.82, 1.14)).
Overall, I found that the change in BCG policy was associated with directly preventing 385 (95% CI -105, 881) TB cases.
Withdrawing universal vaccination at school-age and targeting BCG vaccination towards high-risk neonates was associated with reduced incidence of TB in England. This was largely driven by reductions in the non-UK born. There was a slight increase in UK born school-age cases.
The code for the analysis contained in this chapter can be found at: doi.org/10.5281/zenodo.258305626
References
4 Roy A, Eisenhut M, Harris RJ et al. Effect of BCG vaccination against Mycobacterium tuberculosis infection in children: systematic review and meta-analysis. BMJ (Clinical research ed) 2014;349:g4643–3.
5 Zwerling A, Behr MA, Verma A et al. The BCG world atlas: A database of global BCG vaccination policies and practices. PLoS medicine 2011;8:e1001012.
15 Bhatti N, Law MR, Morris JK et al. Increasing incidence of tuberculosis in England and Wales: a study of the likely causes. BMJ (Clinical research ed) 1995;310:967–9.
20 PHE. Tuberculosis in England 2016 Report (presenting data to end of 2015). Public Health England 2016;Version 1.:173.
23 Rodrigues LC, Diwan VK, Wheeler JG. Protective effect of BCG against tuberculous meningitis and miliary tuberculosis: a meta-analysis. International journal of epidemiology 1993;22:1154–8.
24 Colditz GA, Brewer TF, Berkey CS et al. Efficacy of BCG Vaccine in the Prevention of Tuberculosis. JAMA 1994;271:698.
25 Mangtani P, Abubakar I, Ariti C et al. Protection by BCG Vaccine Against Tuberculosis: A Systematic Review of Randomized Controlled Trials. Clinical infectious diseases : an official publication of the Infectious Diseases Society of America 2014;58:470–80.
27 Zwerling A, Behr MA, Verma A et al. The BCG World Atlas: a database of global BCG vaccination policies and practices. PLoS medicine 2011;8:e1001012.
28 Abubakar I, Pimpin L, Ariti C et al. Systematic review and meta-analysis of the current evidence on the duration of protection by bacillus Calmette-Guérin vaccination against tuberculosis. Health technology assessment 2013;17:1–372, v–vi.
30 Fine P. Stopping routine vaccination for tuberculosis in schools. BMJ (Clinical research ed) 2005;331:647–8.
31 Teo SSS, Shingadia DV. Does BCG have a role in tuberculosis control and prevention in the United Kingdom? Archives of Disease in Childhood 2006;91:529–31.
45 French CE, Antoine D, Gelb D et al. Tuberculosis in non-UK-born persons, England and Wales, 2001-2003. Int J Tuberc Lung Dis 2007;11:577–84.
56 R Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria: 2016.
60 Stevenson M, Nunes T, Heuer C et al. epiR: Tools for the Analysis of Epidemiological Data. 2017.
74 Romanus V, Svensson Å, Hallander HO. The impact of changing BCG coverage on tuberculosis incidence in Swedish-born children between 1969 and 1989. Tubercle and Lung Disease 1992;73:150–61.
75 Guthmann JP, Antoine D, Fonteneau L et al. Assessing BCG vaccination coverage and incidence of paediatric tuberculosis following two major changes in BCG vaccination policy in France. 2011;1–6.
76 Abbott S, Christensen H, Welton NJ et al. Estimating the effect of the 2005 change in bcg policy in england: A retrospective cohort study, 2000 to 2015. Eurosurveillance 2019;24:1900220. doi:10.2807/1560-7917.ES.2019.24.49.1900220
78 Parslow R, El-Shimy NA, Cundall DB et al. Tuberculosis, deprivation, and ethnicity in Leeds, UK, 1982-1997. Archives of disease in childhood 2001;84:109–13.
82 Abubakar I, Laundy MT, French CE et al. Epidemiology and treatment outcome of childhood tuberculosis in England and Wales: 1999-2006. Archives of Disease in Childhood 2008;93:1017–21.
88 Thomas HL, Harris RJ, Muzyamba MC et al. Reduction in tuberculosis incidence in the UK from 2011 to 2015: a population-based study. Thorax 2018;thoraxjnl–2017–211074.
89 Parikh SR, Andrews NJ, Beebeejaun K et al. Effectiveness and impact of a reduced infant schedule of 4CMenB vaccine against group B meningococcal disease in England : a national observational cohort study. The Lancet 2013;388:2775–82.
90 Vehtari A, Gelman A, Gabry J. Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC. The American Statistician 2016;27:1–20.
91 ai H. R Interface for H2O. 2018.
92 Bürkner P-C. brms: An R package for Bayesian multilevel models using Stan. Journal of Statistical Software 2017;80:1–28. doi:10.18637/jss.v080.i01
93 Carpenter B, Gelman A, Hoffman M et al. Stan: A probabilistic programming language. Journal of Statistical Software, Articles 2017;76:1–32. doi:10.18637/jss.v076.i01
94 Davies R, Jones M, Lloyd-Williams H. Age and Work-Related Health: Insights from the UK Labour Force Survey. British Journal of Industrial Relations 2016;54:136–59.
95 Lindley J. The over-education of UK immigrants and minority ethnic groups: Evidence from the Labour Force Survey. Economics of Education Review 2009;28:80–9.
96 Manissero D, Lopalco PL, Levy-Bruhl D et al. Assessing the impact of different BCG vaccination strategies on severe childhood TB in low-intermediate prevalence settings. Vaccine 2008;26:2253–9.
97 Feiring B, Laake I, Molden T et al. Do selective immunisation against tuberculosis and hepatitis B reach the targeted populations ? A nationwide register-based study evaluating the recommendations in the Norwegian Childhood Immunisation Programme. Vaccine 2016;34:2015–20.
98 Nguipdop-Djomo P, Mangtani P, Pedrazzoli D et al. Uptake of neonatal BCG vaccination in England: Performance of the current policy recommendations. Thorax 2014;69:87–9.
Paper: https://doi.org/10.2807/1560-7917.ES.2019.24.49.1900220 Preprint: https://doi.org/10.1101/567511↩
Alternatively code link: https://github.com/seabbs/DirectEffBCGPolicyChange↩