Reassessing the evidence for universal school-age Bacillus Calmette Guerin (BCG) vaccination in England and Wales: re-evaluating and updating a modelling study
Sam Abbott, Hannah Christensen, Ellen Brooks-Pollock
Source:vignettes/paper.Rmd
paper.RmdAuthors:
Sam Abbott, Bristol Medical School: Population Health Sciences, University of Bristol, Bristol, UK
Hannah Christensen, Bristol Medical School: Population Health Sciences, University of Bristol, Bristol, UK
Ellen Brooks-Pollock, Bristol Medical School: Population Health Sciences, University of Bristol, Bristol, UK
Correspondence to: Sam Abbott, Bristol Medical School: Population Health Sciences, University of Bristol, Bristol BS8 2BN, UK; sam.abbott@bristol.ac.uk; 01173310185
Words: Title: 21 Abstract: 274 Paper: 4027
Abstract
Objectives
In 2005, England and Wales switched from universal BCG vaccination against tuberculosis (TB) disease for school-age children to targeted vaccination of neonates. We aimed to recreate and re-evaluate a previously published model, the results of which informed this policy change.
Design
We recreated an approach for estimating the impact of ending the BCG schools scheme, correcting a methodological flaw in the model, updating the model with parameter uncertainty, and improving parameter estimates where possible. We investigated scenarios for the assumed annual decrease in TB incidence rates considered by the UK’s Joint Committee on Vaccination and Immunisation and explored alternative scenarios using notification data.
Outcome measures
The number of vaccines needed to prevent a single notification, and the average annual additional notifications caused by ending the policy change.
Conclusions
The impact of ending the BCG schools scheme was found to be greater than previously thought when notification data were used. Our results highlight the importance of independent evaluations of modelling evidence, including uncertainty, and evaluating multiple scenarios when forecasting the impact of changes in vaccination policy.
Strengths and limitations of this study
This study reevaluates a key piece of the quantitative evidence used to motivate the change in BCG vaccination policy in 2005, correcting a methodological flaw in the original model.
The inclusion of parameter uncertainty, and measurement error, allowed the uncertainty in the final estimates to be presented. Previously published estimates may have been spuriously precise.
As this study used a historical approach the model used is not the accurate method for assessing the impact of ending the BCG schools scheme. However, it provides an estimate that is based on the available data and on the framework used to inform policy making. This allowed the strength of some the quantitative evidence used in the decision-making process to be assessed.
A weakness of the modelling framework used in this study is that it did not include the whole population or age groups outside those directly affected by vaccination. The exclusion of these factors means that our results are likely to underestimate the impact of ending the BCG schools scheme.
This study only considered the impact of ending the BCG schools scheme and not the impact of the introduction of the targeted neonatal vaccination program. This should be considered when evaluating the change in policy as a whole.
Introduction
The Bacillus Calmette–Guérin (BCG) vaccine remains the only licensed vaccine for use against Tuberculosis (TB), although its use globally is controversial due to evidence of variable effectiveness,[1] and waning protection 10-15 years after vaccination.[2] Global usage of the BCG varies between no vaccination, universal vaccination, and high-risk group vaccination and may target either neonates or school-aged children.[3,4] The World Health Organization (WHO) recommends vaccination for all neonates as early as possible after birth in high burden settings, with vaccination in low burden settings being dependent on the country specific epidemiology of TB.[5] This recommendation is based on the strong evidence that the BCG is highly protective in children,[6,7] whilst its effectiveness has been shown to vary with latitude when given later in life.[8]
In England and Wales, universal school-aged (at 13-14 years old) vaccination (hereafter referred to as the BCG schools scheme) was introduced after a MRC trial in the 1950s estimated BCG’s effectiveness at 80% in the ethnic White UK born population.[9] The policy remained in place until 2005, when England and Wales changed to targeted vaccination of high-risk neonates. The 2005 change in BCG vaccination policy was motivated by evidence of decreased transmission of TB, an increasing proportion of TB cases occurring in the non-UK born,[10] and modelling evidence that suggested stopping the BCG schools scheme would have minimal long term effects on incidence rates.[11] Due to the complex nature of both TB and the BCG vaccine, the ongoing impact of this change in policy is hard to directly estimate, with decision makers relying on expert opinion, evidence from surveillance data, and insight from modelling studies.
In 1987, an assessment of the school-age vaccination program was carried out in England and Wales.[11] This study was used, combined with a sensitivity analysis of notification rates, as supporting evidence by the Joint Committee on Vaccination and Immunisation (JCVI) BCG subgroup for the change in vaccination policy.[12,13] This paper aims to re-evaluate this modelling, and re-estimate the predicted impact of stopping the schools scheme. Re-evaluating this work allows for the strength of the evidence used in decision making to be assessed and may highlight any issues with the approach used.
Methods
Modelling the impact of ending the BCG schools scheme
We implemented, and updated, Sutherland et al.’s model for estimating the impact of ending the BCG schools scheme, which is outlined briefly below.[11] This model was based on data from TB notification surveys conducted in 1973, 1978, and 1983.[14] These were used to estimate incidence rates, stratified by BCG vaccination status, in the ethnic White UK born population of England and Wales aged 15-19 years old, 20-24 years old and 25-29 years old. Future incidence rates were forecast by assuming an annual decrease in incidence rates, which was based on historic trends.[11,15] Primary impacts from ending the schools scheme, including the number of vaccines required to prevent a single notification, were estimated by calculating the difference in incidence rates between the vaccinated and unvaccinated populations. Additional notifications from TB transmission were then calculated using a transmission chain model and combined with the primary impact estimates, to calculate the number of annual additional notifications due to ending BCG vaccination. Based on data availability the model used a 5-year time step.
Original estimation of notification rates
The effectiveness of the BCG vaccine was originally estimated by an MRC trial in 1953 at 78% in the United Kingdom.[9] As a follow up to this trial members of the MRC bio-statistics unit conducted a series of notification surveys attempting to ascertain any change in effectiveness, as well as acting as an estimate of notification rates across different demographics.[14] Surveys of those aged 15-24 years were carried out at 5-year intervals in 1973, 1978 and 1983 in England and Wales. For the 1983 survey records of BCG status, Tuberculin status and ethnicity were extracted from the records of notifying physicians and the records of the local health and education authorities. Total notifications across the study period were then aggregated for the following groups: Tuberculin negative and BCG vaccinated, Tuberculin negative and BCG unvaccinated, Tuberculin positive and not vaccinated and those who did not participate. These totals were then combined with the population estimates for each cohort at 13 years of age to estimate the ethnic make up of the population, and to construct notification rates for each category. Data were drawn from a range of sources including: Office of National Statistics data; annual local authority returns for total tuberculin test results; BCG vaccinations in the schools scheme; and the Labour force survey (1983).
For 1983 this corresponded to 874 notifications in 15-24 year old ethnic White UK born persons in England and Wales; survey participation was 80%. As the number of Tuberculin negative subjects not given BCG was unreported this was estimated at 1.9% of those vaccinated with the BCG.[14] See [14] for full details of the survey and the additional assumptions used to give similar estimates for both the 1973 and 1978 surveys. The findings of these surveys were as follows: in the ethnic White population notification rates had fallen by an annual average of 9% and BCG efficacy had remained high.[11,15]
Evidence suggests that the BCG vaccine has a high efficacy for at least the first 15 years after vaccination, therefore Sutherland et al. extrapolated from the data on the 20-24 cohort to a theoretical 25-29 year old cohort. Data on the notifications in 25-29 year olds was available for the first 6 months of the 1983 survey and this was then scaled up to a yearly estimate using the ratio of notifications from this age group against the total number of notifications recorded in that year. Population estimates for the 25-29 year old cohort were based on data from the 20-24 cohort adjusted for all causes of mortality (0.34%). Migration was ignored. The tuberculin positive cohort had a sharp decline in the previous two age cohorts, therefore it was assumed that this continued. Lastly, the efficacy was estimated as being that seen in the 20-24 cohort but with the same decline in protection seen between the last two cohorts. These assumptions allowed notification rates to be estimated for the 25-29 year old population, resulting in a complete cohort over the projected 15 years of BCG effectiveness.
Original construction of forward estimates
Based on these estimated notification rates Sutherland et al. then sought to quantify the ongoing risk of developing notified TB, projected forward in time, for both the vaccinated and unvaccinated populations. To construct these estimates several key assumptions, based on the results seen in the previous surveys, were made. Firstly, it was assumed that efficacy was not degrading within the ethnic White population and therefore historic estimates would continue to apply into the future. Additionally, it was assumed that the annual decay of 9% in notification rates, across all ethnic White populations, would continue indefinitely.
These assumptions allowed the notification rates in both the BCG vaccinated and unvaccinated groups to be projected forward in time. By assuming that the schools scheme is responsible for the observed variation between vaccinated and unvaccinated rates the rate of prevented cases can then be estimated. By scaling this against a cohort of 100,000 13 year old individuals, the number of prevented cases over a 15-year period can be projected for each cohort. By dividing the total number in a given cohort by the number of prevented cases the estimated number of vaccines required to prevent a single case in the 15-year period can then be calculated.
To estimate the total number of prevented notifications, for each cohort, in England and Wales the total number receiving the BCG and the coverage of the schools scheme was required. The coverage of the BCG schools scheme was estimated from annual reports of the Department of Health and Social Security (DHSS) and was assumed to be 75% for all years. The number of BCG vaccines given each year was estimated from the DHSS returns for the years 1967 to 1981, it was then taken as 75% of the estimated ethnic White population aged 13 years from 1982-1996, for each 5-year period thereafter it was assumed to be 2.1 million.
Using the data on BCG coverage, the number of vaccines given each year, and the projected differences between vaccinated and unvaccinated notification rates allowed the number of prevented notifications, due to vaccination, for each age group to be found for each year. These estimates can then be used to give the total number of prevented notifications for those aged between 15-29 years. To understand these estimates, estimates of the projected yearly notifications if the scheme continues were required. These totals were derived from the vaccinated and unvaccinated rates supplemented with similar projections from the tuberculin positive or otherwise ineligible sourced from the 1983 BCG survey.[14]
Original transmission chain model
Sutherland et al. defined their TB transmission model as follows:
- The total expected number of secondary notifications (\(T\)) arising from any single primary notification was estimated as,
\[ T = (1-d)^z < 1 \]
Where \(d\) is the percentage annual decay in notification rates, and \(z\) is the average interval between the notification of any individual and the notification of the patient who infected them.
- The total expected number of secondary notifications arising from any single primary notification (\(T\)) is related to the number of notifications in each generation using the relative generation size (\(x\)) with the following power series,
\[ T=x+x^2+x^3+ . . = \frac{x}{1-x} \]
- The expected average interval between each primary notification and all secondary notifications (\(Z\)) is defined to be the sum of time to all notifications, weighted by the fraction in each generation, divided by the sum of all notifications.
\[ Z = \frac{xz + x^{2}2z + x^{3}3z + . .}{x+x^2+x^3+ . . }= \frac{z}{1-x} \]
Both 2. and 3. are only valid when \(x < 1\).
Updating the transmission chain model
If we assume a constant decay rate of \(d\)% per year and that the next generation of secondary cases are notified \(z\) years after the person who infected them, then 1 notification in year 0 will result in \((1-d)^z\) secondary notifications \(z\) years later. This is therefore the relative generation size (\(x\)), not the total number of secondary notifications (\(T\)), as (incorrectly) stated by Sutherland et al. Therefore, to correct this we revised 1. to the following in our updated model,
\[ x = (1-d)^z < 1 \]
Sutherland et al. gave annual estimates of when secondary cases occurred. We were unable to reproduce this using the original model and therefore added an additional model step for validation purposes. This did not impact our results as we used the the total number of secondary notifications occurring due to primary notifications rather than an annual estimate as our measure of impact. Implementing the model also required several additional assumptions not detailed in [11]. See the supplementary information for details.
Updating model parameter estimates and incorporating parameter uncertainty
Incidence rates were included as point estimates in [11]; in the updated model we included uncertainty in these rates. We did this by first estimating notifications for 1973, 1978, and 1983, using published incidence rates and population estimates. Samples were then generated using a Poisson distribution.[11,14] These samples were then used to estimate a distribution of incidence rates to replace the point estimates used in the original analysis. Sutherland et al. assumed a serial interval of 2 years between linked notifications. Using a newly available literature source we updated this assumption with an estimate of 1.44 (95% CI 1.29 to 1.63) years.[16]
We considered the original assumption of a 9% annual decrease in incidence rates as well as three scenarios based on those considered by the JCVI BCG subgroup:[12,13] these were a 3.9% decrease, a 1.9% decrease, and no change annual in incidence rates. Data on the annual decrease in incidence rates in the ethnic White UK population were not available so we used two proxy measures. The first proxy measure was the annual change in notifications in England and Wales, which was estimated using data from Public Health England (PHE). The standard deviation of this measure was then calculated using the prop.test function in R.[17] The second proxy used was the annual decrease in the UK born age-specific incidence rates in the English population. These were calculated using notification data from the Enhanced TB surveillance system (ETS) and the June Labour Force Survey.[10] Incidence rates were estimated using the epiR package.[18] Uncertainty was incorporated by sampling from a normal distribution for both proxy measures. Data collection for the ETS began in 2000 and prior to this notification data was only available in years with notifications surveys (1973, 1978, and 1983). We therefore estimated incidence rates between 1984 and 1999, and for the years between notifications surveys (1974-1977 and 1979-1982), using locally estimated scatter plot smoothing (LOESS) regression fitted to incidence rates published in [11] and the estimated incidence rates from 2000 on-wards. LOESS is a local regression method that combines multiple regression models in a k-nearest neighbours meta-model.[19] This approach allows nonlinear trends to be fitted using a series of linear models. For years prior to 1973 the annual decreases were assumed to be the mean of the annual decreases from the previous 3 years. For both proxy measures the annual decreases in incidence rates post 2016 were assumed to be the average of the estimates in 2013-2015.
Statistical analysis
For each scenario, we ran the model for 69 years (1969-2028) with 10,000 parameter samples. We tested the difference between scenarios using the Mann-Whitney test for the number of vaccines needed to prevent a single case in 15 years after vaccination for a cohort aged 13-14 years old at vaccination. As in [11] a 15-year time horizon was used with 5-year intervals. The year closest to the year of the change in vaccination policy (2005), which had model estimates, was used as the baseline.
Results
Model validation
Our model produced results that were comparable with those from [11] (supplementary table S1) when the original model structure and assumptions were used. When estimating the total notifications from ending the BCG schools scheme at different times in ethnic White UK born adults aged 15-29 years old in England and Wales our model had a median absolute error of 2.03% (2.5, 97.5% Q: 0.00%, 3.72%) and a maximum absolute error of 3.91% when compared to [11].
Annual change in TB incidence rates
We found that the assumption of a 9% annual decrease in incidence rates in the ethnic White UK born was not comparable to estimates using either notification data or age-specific incidence rates in the time period studied (figure 1).
Figure 1: Annual percentage change in ethnic White UK born incidence rates for those aged 15-19, 20-24, and 25-29 years old under different scenarios. For the notification and incidence rate scenarios each line represents the median of 10,000 parameter samples.
Vaccines required to prevent a single notification
We found that incorporating uncertainty, did not alter the number of vaccines required to prevent a single notification within 15 years in a cohort vaccinated at school-age, when the annual decrease in incidence rates was assumed to be 9% (supplementary table S2). However, the updated estimate had a wide range (15000 (2.5, 97.5% Q: 12000, 19000) vaccines required in 2004). As the assumed annual decrease in incidence rates was reduced the number of vaccines required to prevent a single notification also reduced. Assuming an annual decrease of 1.9% (one of the scenarios evaluated by the JCVI) resulted in an estimate of 1600 (2.5, 97.5% Q: 1300, 2000) vaccines required to a prevent a single notification in 2004. This assumption was the most comparable, although not equivalent, to estimates derived using notifications (1400 (2.5, 97.5% Q: 1100, 1700), P: 0.077) and age-specific incidence rates (1500 (2.5, 97.5% Q: 460, 4900), P: 0.083). The estimate using incidence rates had a high degree of uncertainty (figure 2). The number of vaccines required increased slightly over time with 1800 (2.5, 97.5% Q: 1500, 2200) required in 2009, 2000 (2.5, 97.5% Q: 1600, 2500) required in 2014, and 2200 (2.5, 97.5% Q: 1800, 2700) required in 2019 when an annual decrease of 1.9% in incidence rates was assumed.
Figure 2: Vaccines required in a cohort of those vaccinated at school-age to prevent a single case of Tuberculosis within 15 years of vaccination in 2004, 2009, 2014, or 2019. The years presented were dictated by the 5-year timestep of the model. The percentage annual decrease scenarios considered were based on those considered by the JCVI BCG subgroup, with the addition of a scenario using aggregate notification data and a scenario using estimates of age-specific incidence rates in the UK born. Each boxplot summarises the output of 10,000 model simulations for each scenario.
Average annual additional cases from ending the BCG schools scheme at various dates
We found that updating parameter values, and incorporating uncertainty, did not alter the average annual primary additional notifications from stopping the BCG schools scheme when the annual decrease was assumed to be 9%. However, when these changes were combined with the updated transmission model we found that the impact of ending BCG vaccination was greater than previously reported with an increase in the number of estimated cases due to onwards transmission (supplementary table S3; supplementary table S4). These estimates were uncertain with 94 (2.5, 97.5% Q: 72, 119) additional annual notifications if vaccination was stopped in 2001. As the assumed annual decrease in incidence rates was reduced the annual number of additional notifications increased with 6099 (2.5, 97.5% Q: 4691, 7719) notifications when the annual decrease was assumed to be 1.9% and vaccination stopped in 2001. The number of annual notifications reduced with time: 5314 (2.5, 97.5% Q: 4082, 6725) from ending vaccination in 2006; 4327 (2.5, 97.5% Q: 3315, 5496) from ending vaccination in 2011, and 2852 (2.5, 97.5% Q: 2074, 3741) from ending vaccination in 2016 (figure 3).
Figure 3: Annual additional (stratified into primary and secondary) notifications in 15-29 year olds from stopping the BCG schools scheme in 2006, and 2011 until 2028. The years presented were dictated by the 5-year timestep of the model. The percentage annual decrease scenarios considered were based on those considered by the JCVI BCG subgroup. Data based scenarios and the JCVI 0% decrease scenario were not presented here as the updated transmission model could not support these scenarios. Each boxplot summarises the output of 10,000 model simulations for each scenario. Secondary notifications are reported assuming they occurred in the same year as the primary notifications that caused them.
Discussion
The existing method for estimating the impact of the BCG schools scheme produced uncertain estimates of the impact of ending the scheme in all years evaluated when parameter uncertainty was included. The approach used to estimate additional notifications due to transmission was found to be incorrect. Once corrected, the transmission model produced much higher estimates of additional notifications caused by ending BCG vaccination than previously reported. Further updating the model with the annual decrease in TB notifications based on both notifications and using age-specific incidence rates resulted in a decrease in the number of vaccines needed to prevent a single case in all years considered. A scenario with a 1.9% annual decrease in incidence rates was most comparable to our results based on notifications. Using this scenario, we found that the number of TB notifications arising from ending school age BCG vaccination was much greater than originally estimated using the scenario considered in Sutherland et al.
This study reassesses a key piece of the quantitative evidence used to motivate the change in BCG vaccination policy in 2005.
In addition to identifying that the public health impact of the change in vaccine policy was likely much larger than originally estimated, our results also provide new insights into the uncertainty of the previously published model predictions by including parameter uncertainty and measurement error and updates these predictions using newly available data. As historical data on incidence rates in the ethnic White UK born in England and Wales were not available, we considered two approaches to proxy them and investigated multiple scenarios based on those explored by the JCVI BCG subgroup. The simulation approach used here, although updated where possible, is not the most accurate method for estimating the impact of ending the BCG schools scheme as it relies on numerous assumptions based on the available knowledge in 1987 and does not account for the role of non-White and non-UK born cases. However, the strength of this work is that the estimates are based on the framework used to inform policy making. This allowed the strength of the model used in the decision-making process to be assessed once parameter uncertainty had been incorporated and for flaws in the model to be identified. This would not have been possible if the impact had been assessed using only the observed data or with an alternative model. It also allowed estimates based on updated data to be compared to historic estimates within the same framework. This would also not have been possible if a different framework had been used. As mentioned, a weakness of the model used in this study is that it did not include the whole population or age groups outside those directly affected by vaccination. Heterogeneous mixing between these groups is also likely to be an important consideration. The exclusion of these factors means that our results are likely underestimate the impact of ending the BCG schools scheme. A final limitation is that this study only considers the impact of ending the BCG schools scheme and not the impact of the introduction of the targeted neonatal vaccination program. This should be considered when evaluating the change in policy as a whole.
Little work has been done to assess the impact of the 2005 change in BCG vaccination policy or to assess the quantitative evidence used in decision making. However, multiple studies have evaluated the cost effectiveness of various BCG programs and the impact of switching between them. A cluster-randomised trial in Brazil found that BCG vaccination of those at school-age was cheaper than treatment and would prevent one TB case per 381 vaccinations even with a vaccine effectiveness of only 34% (8-53%).[20] This is substantially fewer than our estimate of 2000 (2.5, 97.5% Q: 1600, 2500) (the most comparable year from our study). However, the same trial found that for regions close to the equator BCG effectiveness was low in school-age children but unchanged in neonates,[21] highlighting the importance of considering the BCG vaccines reduced effectiveness near the equator when determining vaccination policy.[22] There is also some research which supports universal re-vaccination of those at school-age, in countries with high incidence and universal vaccination of neonates, as it may be cost effective when BCG effectiveness is moderate to high.[21,23] There is some evidence that targeted vaccination of high risk neonates maybe more cost effective than universal vaccination of neonates.[24,25] However, a study in Sweden found that incidence rates in Swedish-born children increased slightly after universal vaccination of neonates was discontinued in favour of targeted vaccination.[26] In France, which switched from universal vaccination of neonates to targeted vaccination in 2007, it has also been shown that targeted vaccination reduced coverage in those most at risk.[27] Targeted vaccination may not be more cost effective than universal vaccination when possible reductions in transmission are considered. Our study indicated that a substantial number of cases due to transmission may be preventable if universal school-age BCG vaccination was still in place. This result is dependent on the effectiveness of BCG vaccination when given later in life, for which there is good evidence in the ethnic White UK born.[9] We did not consider neonatal vaccination which would be less impacted by BCG’s effectiveness reducing when given later in life, but may also be less likely to result in the same reductions in ongoing transmission.
This study indicates that some of the evidence used to justify the 2005 change in BCG vaccination policy may have depended on a methodologically flawed model, resulting in the impact of ending BCG vaccination being underestimated. Modelling evidence can often be complex and difficult to reproduce, it is important that policy makers, or those who work with them, have the skills to assess its quality. This study also highlights the importance of including both parameter and measurement error, as excluding these sources of variation may lead to spuriously precise results. Spurious precision is problematic for policy makers as the worst-case scenario often needs to be considered when making policy decisions. In addition, our exploration of the assumptions used to estimate the annual change in TB incidence rates in the ethnic White UK born illustrates the structural impact of assuming an annual decrease in TB incidence rates. More realistic estimates of the annual decrease in incidence rates resulted in a greatly increased impact of ending the BCG schools scheme. Policy makers should consider these updated estimates when assessing the role of BCG vaccination in those at school-age. However, decisions regarding vaccine policy in the UK require economic evaluation, which discounts costs and benefits in the future; discounting has not been applied in this study which estimates the epidemiological impact of vaccination only.
This study has reassessed some of the evidence previously used in decision making, correcting the transmission model used, and updating the approach with new data. However, as 15 years of detailed surveillance data have been collected since the ending of the BCG schools scheme it is now possible to use regression-based approaches to estimate the direct impact on incidence rates of ending the BCG schools scheme.[28] These approaches could also be used to estimate the impact of vaccinating high-risk neonates, which may outweigh any negative impacts of ending the BCG schools scheme. In addition, the development, and use, of a transmission dynamic model would allow the more accurate estimation of indirect effects and the forecasting of long-term impacts.
Acknowledgements
The authors thank the TB section at Public Health England (PHE) for maintaining the Enhanced Tuberculosis Surveillance (ETS) system; all the healthcare workers involved in data collection for the ETS.
Contributors
SA, HC, and EBP conceived and designed the work. SA undertook the analysis with advice from all other authors. All authors contributed to the interpretation of the data. SA wrote the first draft of the paper and all authors contributed to subsequent drafts. All authors approve the work for publication and agree to be accountable for the work.
Funding
SEA, HC, and EBP are funded by the National Institute for Health Research Health Protection Research Unit (NIHR HPRU) in Evaluation of Interventions at University of Bristol in partnership with Public Health England (PHE). The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR, the Department of Health or Public Health England.
Conflicts of interest
HC reports receiving honoraria from Sanofi Pasteur, and consultancy fees from AstraZeneca, GSK and IMS Health, all paid to her employer.
Accessibility of programming code
The code for the analysis contained in this paper can be found at: https://doi.org/10.5281/zenodo.2635687
Data availability statement
Tuberculosis (TB) notification data is available from the Enhanced Tuberculosis Surveillance system upon request to the TB section at Public Health England. Labour Force Survey data is available from the UK data service if registered at a UK institution. All other data is available is available here: https://doi.org/10.5281/zenodo.2635687
References
1 Mangtani P, Abubakar I, Ariti C et al. Protection by BCG Vaccine Against Tuberculosis: A Systematic Review of Randomized Controlled Trials. Clinical infectious diseases : an official publication of the Infectious Diseases Society of America 2014;58:470–80.
2 Abubakar I, Pimpin L, Ariti C et al. Systematic review and meta-analysis of the current evidence on the duration of protection by bacillus Calmette-Guérin vaccination against tuberculosis. Health technology assessment 2013;17:1–372, v–vi.
3 Pilger D, Nguipdop-Djomo P, Abubakar I et al. BCG vaccination in England since 2005: A survey of policy and practice. BMJ Open 2012;2:e001303.
4 Zwerling A, Behr MA, Verma A et al. The BCG world atlas: A database of global BCG vaccination policies and practices. PLoS medicine 2011;8:e1001012.
5 World Health Organization. Global Tuberculosis Report. 2017.
6 Rodrigues LC, Diwan VK, Wheeler JG. Protective effect of BCG against tuberculous meningitis and miliary tuberculosis: a meta-analysis. International journal of epidemiology 1993;22:1154–8.
7 Colditz GA, Brewer TF, Berkey CS et al. Efficacy of BCG Vaccine in the Prevention of Tuberculosis. JAMA 1994;271:698.
8 Mangtani P, Abubakar I, Ariti C et al. Protection by BCG vaccine against tuberculosis: A systematic review of randomized controlled trials. Clinical Infectious Diseases 2014;58:470–80.
9 Hart PDA, Sutherland IAN. BCG and vole bacillus vaccines in the prevention of tuberculosis in adolescence and early adult life. The American Statistician 1972;46:371–85.
10 Public Health England. Tuberculosis in England 2017 report ( presenting data to end of 2016 ) About Public Health England. 2017.
11 Sutherland I, Springett VH. The effects of the scheme for BCG vaccination of schoolchildren in England and Wales and the consequences of discontinuing the scheme at various dates. Journal of epidemiology and community health 1989;43:15–24.
12 Joint Committee on Vaccination, panel IB. Minutes. 2002.
13 Joint Committee on Vaccination, panel IB. Minutes. 2003.
14 Sutherland I. Effectiveness of BCG vaccination in England and Wales in 1983. Tubercle 1987;68:81–92.
15 Springett VH, Darbyshire JH, Nunn AJ et al. Changes in tuberculosis notification rates in the white ethnic group in England and Wales between 1953 and 1983. Journal of epidemiology and community health 1988;42:370–6.
16 Borgdorff MW, Hof S van den, Kalisvaart N et al. Influence of sampling on clustering and associations with risk factors in the molecular epidemiology of tuberculosis. Am J Epidemiol 2011;174:243–51.
17 R Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria: 2016.
18 Stevenson M, Nunes T, Heuer C et al. epiR: Tools for the Analysis of Epidemiological Data. 2017.
19 Fox J, Weisberg S. An R companion to applied regression. Third. Thousand Oaks CA:: Sage 2019. https://socialsciences.mcmaster.ca/jfox/Books/Companion/
20 Pereira SM, Barreto ML, Pilger D et al. Effectiveness and cost-effectiveness of first BCG vaccination against tuberculosis in school-age children without previous tuberculin test (BCG-REVAC trial): A cluster-randomised trial. Lancet Infectious Diseases 2012;12:300–6.
21 Barreto ML, Pilger D, Pereira SM et al. Causes of variation in BCG vaccine efficacy: Examining evidence from the BCG REVAC cluster randomized trial to explore the masking and the blocking hypotheses. Vaccine 2014;32:3759–64.
22 Fine PEM. Variation in protection by BCG: implications of and for heterologous immunity. Lancet 1995;346:1339–45.
23 Dye C. Making wider use of the world’s most widely used vaccine: Bacille Calmette-Guerin revaccination reconsidered. Journal of The Royal Society Interface 2013;10:20130365–5.
24 Usher C, Adams R, Schmitz S et al. Evaluating the neonatal BCG vaccination programme in Ireland. Archives of Public Health 2016;74:1–12.
25 Hersh AL, Tala-Heikkila M, Tala E et al. A cost-effectiveness analysis of universal versus selective immunization with Mycobacterium bovis bacille Calmette-Guérin in Finland. Int J Tuberc Lung Dis 2003;7:22–9.
26 Romanus V, Svensson Å, Hallander HO. The impact of changing BCG coverage on tuberculosis incidence in Swedish-born children between 1969 and 1989. Tubercle and Lung Disease 1992;73:150–61.
27 Guthmann JP, Antoine D, Fonteneau L et al. Assessing BCG vaccination coverage and incidence of paediatric tuberculosis following two major changes in BCG vaccination policy in France. 2011;1–6.
28 Abbott S, Christensen H, Welton NJ et al. Estimating the effect of the 2005 change in bcg policy in england: A retrospective cohort study, 2000 to 2015. Eurosurveillance 2019;24:1900220. doi:10.2807/1560-7917.ES.2019.24.49.1900220
Results Copy to top
The previously published model was found to contain a methodological flaw and to be spuriously precise. It greatly underestimated the impact of ending school-age vaccination compared to our updated, corrected, model. The updated model produced predictions with wide confidence intervals when parameter uncertainty was included. Model estimates based on an assumption of an annual decrease in TB incidence rates of 1.9% were closest to those estimated using notification data. Using this assumption, we estimate that 1600 (2.5,97.5% Quantiles (Q): 1300, 2000) vaccines would have been required to prevent a single notification in 2004.
Supplementary Information: Reassessing the Evidence for Universal School-age Bacillus Calmette Guerin (BCG) Vaccination in England and Wales
Sam Abbott, Hannah Christensen, Ellen Brooks-Pollock
Model assumptions
The Sutherland et al. model required several additional assumptions on top of those detailed in the methods.[11] Firstly, as incidence rates for those ineligible for the BCG schools scheme are not published, We assumed that they were equal to those in the unvaccinated population. In addition, in order to reproduce the distribution of cases annually (rather than by generation or overall) reported in Sutherland et al. we introduced an additional model step and parameter; the proportion of secondary cases in the first generation that occurred in the first year (\(f\)). This modelling step was only required to reproduce the final table from [11] and did not impact our estimates of the impact of ending the BCG schools scheme. It is included only for validation purposes.
The annual distribution of secondary notifications (\(N\)) was modelled by first estimating the number of secondary notifications that occurred in the current year (\(i\)) due to primary notifications in that year (\(N^{Current}\)) and then estimating how many secondary notifications occurred 5 years later (\(N^{Projected}\)). \(N^{Current}\) was estimated using the number of primary notifications (\(P\)) multiplied by the number of total expected number of secondary notifications per primary notification (\(T\)), the proportion of secondary cases in the first generation that occurred in the first year (\(f\)), and the relative size of the first generation (\(x\)). \(N^{Projected}\) was then estimated by assuming that it was equal to the number of secondary notifications, minus notifications occurring in the first year, that occurred \(Z\) (the expected average interval between each primary notification and all secondary notifications) years ago. As this used the overall number of notifications from the previous time step a decay of \((1 - d)^{5-Z}\) was applied. This approach can be summarised as follows,
\[ N^{Current} = P T f x \] \[ N^{Projected} = (PT - N^{Current})(1 - d)^{5-Z} \] \[ N_i = N^{Current}_i + N^{Projected}_{i-1} \]
We fitted the proportion of secondary cases in the first generation that occurred in the first year (\(f\)) using least squares to the original estimates of the total notifications due to ending the scheme under several scenarios, for several years. We validated the fitted model by comparing the results with those from the original implementation using the mean absolute percentage error, normalised by the original estimate, as the performance metric.
Model validation
The percentage of cases in the first year was fitted to the Sutherland et al. estimates using the least squares method, such that \(f=0.764\). Our model produced estimates that were comparable with those from [11]. The median annual decrease estimated using notifications was 3.06% (2.5, 97.5% Quantiles (Q): -8.32%, 11.45%), with a maximum of 15.14% (2.5, 97.5% Q: 14.22%, 16.03%) in 1987 and a minimum of -10.17% (2.5, 97.5% Q: -10.82%, -9.52%) in 2005. Using age-specific incidence rates we estimated the median annual decrease in incidence rates for 15-19 year olds was 1.65% (2.5, 97.5% Q: -40.49%, 39.97%), 3.16% (2.5, 97.5% Q: -33.95%, 38.30%) for 20-24 year olds, and 2.63% (2.5, 97.5% Q: -36.28%, 37.17%) for 25-29 year olds. There was substantial variation between years and a high degree of uncertainty.
| Year of Ending Scheme | 1988_orig | 1988_rec | 1988_rec_vdiff | 1993_orig | 1993_rec | 1993_rec_vdiff | 1998_orig | 1998_rec | 1998_rec_vdiff | 2003_orig | 2003_rec | 2003_rec_vdiff | 2008_orig | 2008_rec | 2008_rec_vdiff | 2013_orig | 2013_rec | 2013_rec_vdiff |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1986 | 288 | 296 | 8 | 226 | 226 | 0 | 208 | 205 | -3 | 181 | 175 | -6 | 128 | 123 | -5 | 80 | 78 | -2 |
| 1991 | 288 | 296 | 8 | 165 | 166 | 1 | 130 | 131 | 1 | 128 | 126 | -2 | 115 | 111 | -4 | 80 | 78 | -2 |
| 1996 | 288 | 296 | 8 | 165 | 166 | 1 | 90 | 91 | 1 | 77 | 76 | -1 | 80 | 80 | 0 | 72 | 70 | -2 |
Results
| Year of Vaccination | 9% decrease (original parameters) | 9% decrease | 3.9% decrease | 1.9% decrease | 0% decrease | Notifications | Incidence Rates |
|---|---|---|---|---|---|---|---|
| 1969 | 460 | 460 (390, 540) | 460 (390, 540) | 460 (390, 540) | 460 (390, 540) | 460 (390, 540) | 460 (390, 540) |
| 1974 | 940 | 940 (780, 1200) | 880 (720, 1100) | 850 (700, 1100) | 830 (680, 1000) | 860 (710, 1100) | 860 (640, 1100) |
| 1979 | 1400 | 1400 (1100, 1800) | 1100 (910, 1400) | 1000 (820, 1300) | 900 (740, 1100) | 1000 (860, 1300) | 1100 (680, 1700) |
| 1984 | 2200 | 2200 (1800, 2900) | 1300 (1100, 1700) | 1100 (900, 1400) | 900 (740, 1100) | 1600 (1300, 2000) | 1400 (730, 2900) |
| 1989 | 3600 | 3600 (2900, 4600) | 1600 (1300, 2100) | 1200 (990, 1500) | 900 (740, 1100) | 1800 (1500, 2200) | 1700 (760, 3800) |
| 1994 | 5800 | 5800 (4700, 7300) | 2000 (1600, 2500) | 1300 (1100, 1700) | 900 (740, 1100) | 1700 (1400, 2200) | 1600 (640, 4200) |
| 1999 | 9300 | 9200 (7600, 12000) | 2500 (2000, 3100) | 1500 (1200, 1900) | 900 (740, 1100) | 1600 (1300, 2000) | 1500 (510, 4200) |
| 2004 | 15000 | 15000 (12000, 19000) | 3000 (2400, 3800) | 1600 (1300, 2000) | 900 (740, 1100) | 1400 (1100, 1700) | 1500 (460, 4900) |
| 2009 | 24000 | 24000 (19000, 30000) | 3600 (3000, 4600) | 1800 (1500, 2200) | 900 (740, 1100) | 1200 (960, 1500) | 1200 (350, 4300) |
| 2014 | 38000 | 38000 (31000, 48000) | 4500 (3600, 5600) | 2000 (1600, 2500) | 900 (740, 1100) | 1500 (1200, 1900) | 1500 (390, 6000) |
| 2019 | 61000 | 61000 (50000, 78000) | 5400 (4400, 6900) | 2200 (1800, 2700) | 900 (740, 1100) | 2100 (1800, 2700) | 2300 (470, 11000) |
| 2024 | 98000 | 98000 (80000, 120000) | 6600 (5400, 8400) | 2400 (1900, 3000) | 900 (740, 1100) | 3200 (2600, 4100) | 3300 (550, 18000) |
| Year Ending Scheme | 9% decrease (original parameters) | 9% decrease | 3.9% decrease | 1.9% decrease |
|---|---|---|---|---|
| 1971 | 111 | 111 (88, 135) | 204 (161, 249) | 286 (226, 350) |
| 1976 | 90 | 90 (71, 110) | 187 (148, 229) | 276 (219, 337) |
| 1981 | 55 | 55 (44, 68) | 152 (120, 185) | 244 (193, 298) |
| 1986 | 35 | 35 (28, 43) | 125 (99, 152) | 218 (172, 266) |
| 1991 | 25 | 25 (19, 30) | 109 (86, 133) | 203 (161, 247) |
| 1996 | 17 | 17 (13, 21) | 93 (74, 114) | 185 (147, 225) |
| 2001 | 12 | 12 (9, 15) | 79 (62, 96) | 166 (132, 203) |
| 2006 | 8 | 8 (6, 10) | 65 (51, 79) | 145 (115, 177) |
| 2011 | 6 | 5 (4, 7) | 50 (39, 62) | 118 (92, 145) |
| 2016 | 3 | 3 (2, 4) | 32 (23, 40) | 78 (58, 99) |
| 2021 | 1 | 1 (1, 2) | 15 (9, 21) | 39 (24, 54) |
| Year Ending Scheme | 9% decrease (original parameters) | 9% decrease | 3.9% decrease | 1.9% decrease |
|---|---|---|---|---|
| 1971 | 107 | 889 (702, 1115) | 3760 (2945, 4735) | 10855 (8454, 13697) |
| 1976 | 74 | 616 (472, 786) | 3179 (2439, 4045) | 9852 (7584, 12514) |
| 1981 | 46 | 380 (291, 485) | 2575 (1976, 3275) | 8705 (6703, 11063) |
| 1986 | 29 | 240 (184, 306) | 2118 (1625, 2693) | 7772 (5980, 9874) |
| 1991 | 20 | 169 (130, 216) | 1848 (1417, 2348) | 7229 (5553, 9185) |
| 1996 | 14 | 118 (90, 150) | 1583 (1214, 2008) | 6602 (5072, 8385) |
| 2001 | 10 | 82 (62, 104) | 1338 (1026, 1697) | 5933 (4553, 7522) |
| 2006 | 7 | 56 (43, 72) | 1101 (843, 1397) | 5168 (3965, 6554) |
| 2011 | 4 | 37 (28, 48) | 851 (647, 1086) | 4209 (3221, 5357) |
| 2016 | 2 | 21 (15, 28) | 539 (391, 710) | 2772 (2015, 3642) |
| 2021 | 1 | 9 (6, 13) | 257 (160, 370) | 1368 (853, 1966) |