Chapter 11 Discussion
This thesis has assessed the impact of the 2005 change in BCG vaccination policy in some detail. The aim of this chapter is to provide an overview of the principle findings of this thesis; interpret these findings; discuss the overall strengths and weaknesses of this thesis; outline the potential implications; explore the opportunities for public engagement that this work allowed; and describe potential future research. Each results chapter contains a detailed discussion of the approach used, the results, and the strengths and weaknesses of the findings, for that chapter. Consequently, the aim of this chapter is to summarise and discuss the findings from this thesis as a whole.
11.1 Principal findings
In Chapter 4, I explored TB epidemiology in England with a focus on BCG vaccination and data completeness. I found that there was some evidence that negative outcomes were more frequent in Tuberculosis (TB) cases not BCG vaccinated than in those that were. I also found that missingness in routine surveillance sources of TB data was associated with multiple risk factors. In Chapter 6, I used logistic regression to estimate associations between BCG vaccination and TB outcomes. I found supporting evidence that BCG vaccination was associated with reduced all-cause morality with some evidence that this may have been due to reduced TB mortality. I found little evidence for any other association with TB outcomes, after adjusting for confounding. In Chapter 5, I explored some of the modelling evidence that was used by policy makers to assess the impact of ending school-age universal BCG vaccination. I found that the previous approach was methodologically flawed and had underestimated the amount of uncertainty surrounding the effect estimates. Using newly available data, I also found that ending universal school-age BCG vaccination was projected to result in greater number of notifications in the UK born than previously thought. These findings were confirmed in Chapter 7, where I evaluated the evidence in the surveillance data that the change in policy had impacted TB incidence rates in the target populations, using Possion and negative binomial models. However, in this chapter, I found that any increase in TB notifications in the UK born was likely far outweighed by reductions in the number of notifications in the non-UK born. Using this approach, I was unable to rule out an unrelated policy change as the cause of this reduction. Finally, in Chapter 10, I forecast the impact of various vaccination scenarios using a dynamic TB model that was developed and fitted in Chapter 8 and Chapter 9. I found that the BCG schools scheme was projected to reduce UK born TB incidence compared to both neonatal and no vaccination over a range of time horizons. However, neonatal vaccination reduced incidence in children compared to any other scenario, although it had little impact in any other age group. No vaccination programme evaluated had an impact on incidence in older adults. The results from Chapter 10 are preliminary as the model on which they were based was a very poor fit to the observed data (Chapter 9).
11.2 Strengths and limitations
This thesis has used multiple methods, and data sources, to explore the impact of the 2005 change in BCG policy. This multi-method approach allows for more certainty in the findings than if only a single approach had been used. A limitation of the work in this thesis is that all results were based on a single surveillance dataset. Surveillance data is subject to multiple bias issues (see Chapter 4 for details). Ideally, multiple different data types would have been used to more effectively triangulate the impact of the change in policy. Datasets that would have added value include: notification data from other countries that also changed BCG policy, regional datasets with more reliable data on BCG status and year of BCG, and data on BCG coverage and eligibility. However, to my knowledge, no similarly thorough use of an equivalent data source exists for TB. Another limitation of the work in this thesis is the very poor fit of the dynamic model, developed in Chapter 8, to the observed data. This made drawing conclusions from it difficult. On the other hand, the model fitting presented in Chapter 9 represented one of the only open source examples of fitting a very complex model to data using robust bayesian, plug and play, approaches. The lack of success is itself a useful result from which much can be learned about the usability of the fitting tools and the upper limits on model complexity. A major strength of the work in this thesis is the attention that has been paid to make it both open and reproducible. Hopefully, this will allow these findings to be more easily validated, and built upon, by others. Finally, the work in this thesis generated several tools as a by-product of the main research question.
11.3 Implications for policy makers
This thesis has highlighted the trade-off between vaccinating those at school-age and neonates in a setting where the waning of BCG effectiveness when given later in life is minimal. Whilst policy makers were previously aware of this trade-off, their was little quantitative evidence exploring it explicitly. Globally, BCG policy does not account for areas where the BCG vaccine may be equally effective regardless of when it is given.[3] Future BCG policy should consider these factors. In addition, new TB vaccines are in development that may be less susceptible to waning effectiveness when given later in life over a greater geographic area.[22] The findings from this thesis may be applicable to these new vaccines in areas where the BCG vaccine is currently known to be ineffective when given later in life. This may mean that these newly developed vaccines may be better targeted at those at school-age, rather than neonates, depending on the duration of protection that they provide. The work from Chapter 8 and Chapter 9 may be particularly suitable to adaption for this use case. This thesis has also explored the potential benefits of BCG vaccination on TB outcomes. The evidence of a reduction in all-cause mortality in TB cases may add additional weight to the argument that wider vaccination maybe more cost effective than previously thought in low incidence countries. These findings may also be used to drive vaccine uptake as they provide additional incentives for vaccination. Finally, this thesis has shown that the impact of the BCG policy has varied depending on UK birth status. This may strengthen the case for varying vaccination policies depending on the country of origin of the target of vaccination policy, and their immediate families country of origin.
Policy makers in the UK should consider the implications of the results from this thesis for BCG vaccination policy. Of particular importance is the finding that the previously published approach for estimating the impact of ending BCG vaccination in the UK was methodologically flawed, spuriously precise, and, when updated with newly available data, produced much larger estimates of the impact of ending BCG vaccination than previously thought. This finding highlights the importance of independent validation of modelling studies by subject area experts. To counterbalance this, the finding that overall the change in BCG vaccination policy was associated with decreased incidence rates indicate that the change in policy may have been justified. However, the benefit of the policy change was focussed on the non-UK born population who may have been impacted by other policy changes. Considering UK born cases alone, this thesis provides evidence that TB cases increased, with cases decreasing in young children. The modelling evidence, although preliminary, supported the finding that the change in policy would lead to increased overall incidence rates in the UK born population but reduce incidence rates in young children compared to school-age vaccination. On top of these specific findings, the points made in the previous paragraph also apply to the UK.
Unfortunately, definite recommendations cannot be made to UK policy makers based on the findings from this thesis. This is due to several limitations of the evidence and due to the inherent complexity of both TB and the BCG vaccine. A particular limitation is the difficulty in identifying who - post the change in BCG vaccination policy - should have been vaccinated and who should not have been. This meant that targeted high-risk neonatal vaccination could not be modelled and therefore could not be compared to school-age BCG vaccination. It also meant that population-level studies had to be used when studying the impact of change in policy. As the impact of the targeted scheme was likely focussed on the groups it targeted this may have diluted the impact of this policy. An additional issue is that whilst a trade-off was identified between reducing the number of TB cases in young children at the cost of a larger number of adult TB cases the impact of TB symptoms on children compared to adults was not considered. To be able to make more precise recommendations additional data sources are required. The first of these is a measure of the impact of TB on individuals at various ages, such as a quality-adjusted life-year (QALY). This would ideally be arrived at by assessing expert opinion and could then be used to identify what level of trade-off between childhood and adult TB cases results in the lowest overall QALY burden. Finally, data is required on who should have been vaccinated under the targeted high-risk neonatal vaccination scheme and of these individuals how many of them then went on to develop TB. This data would allow more precise statistical estimates to be made of the overall impact of the change in policy and would also enable targeted high-risk neonatal vaccination to be included in the dynamic TB model. Inclusion in the dynamic model would then allow the impact of various targeted vaccination scenarios to be considered over an extended time-frame and compared to universal vaccination scenarios.
11.4 Open reproducible research
Open reproducible research has been a primary focus of this thesis. A version controlled archive of this thesis is available from GitHub41, with a formatted version available on my personal site42. This thesis relies on data from the Enhanced TB Surveillance system and the Labour Force Survey. The cleaning and munging of this data has been standardised as an R package, tbinenglanddataclean43, and is available for download. All chapters that contain analysis are linked to their own GitHub repositories, each of which is fully reproducible (discounting the raw data which cannot be released due confidentiality reasons). Literate coding was used to link analysis code with documentation using the R tool chain. An R package, prettypublisher44, was developed to augment these tools. Where possible open source tooling has been used to provide a working analytical environment for each chapter. All chapters that have been peer reviewed, or are undergoing peer review, have been preprinted. The model developed in Chapter 8 has been released as an R package along with the fitting pipeline developed in Chapter 9. Tools used to develop the figures in Chapter 2, using World Health Organization data, were expanded into an R package (see Chapter 3). Tooling developed alongside this thesis follows open source best practices. See Chapter 1 for details of the open source projects developed as part of this thesis.
11.5 Public engagement
Public engagement has been a constant theme throughout my doctoral work. This is closely linked with the previous aim of open and reproducible research. Effort has been taken so that all peer reviewed content is available for the wider public with Twitter used to disseminate findings. Where appropriate, interactive applications have been developed that seek to explore some of the key findings of this thesis, as well as teaching more theoretical concepts used throughout (see Chapter 1). Numerous case studies have also been produced that outline these theoretical concepts using some of the open source tools developed alongside this thesis45. These tools were themselves developed to lower the barrier of entry to infectious disease research. One of these tools, idmodelr, has been released to CRAN. Finally, in 2017 I spent a week at the Green Man Festival exploring the mathematics of vaccination with the general public. This made use of several simple games, as well as an interactive online tool46.
11.6 Future research
The finding that BCG vaccination may reduce mortality in TB cases from Chapter 6 require validation in other data sources and settings. A larger sample size may be required in order to unpick the association between BCG vaccination and the cause of mortality. These findings could also be included in a cost effectiveness study of the BCG vaccine. The dynamic model developed in Chapter 8 and fitted in 9 did not fit the observed data well. Additional compute time is needed to diagnosis whether this is a limitation of the fitting technique used, if the model itself was a poor fit for the data, or if the model was overly complex. Additional strategies for exploring this issue are discussed in Chapter 9. In addition this model does not currently include targeted vaccination of neonates and is only fitted to data up to 2004. In order to be able to more accurately explore current, and future, vaccination policy the extension of the model to the present data would be required. This would potentially be of great use for policy makers. An alternative would be to develop a comparable model in different settings. This would allow the generalisability of the findings to be explored. The dynamic model could also be further generalised to include hypothetical future vaccines with differing characteristics. This would allow vaccine characteristics and optimal deployment strategies to be explored, via simulations, ahead of further development. Both getTBinR and idmodelr have active user bases and further developments are planned. This includes: additional tooling, documentation, and case studies. Further development of several of the interactive tools discussed in Chapter 1 is also planned.
11.7 Conclusions
This thesis has provided new evidence regarding the use of BCG vaccination in England. A simulation study that was used as part of the quantitative evidence for the change in policy was recreated, corrected, and updated. The results from this updated model suggested that the change in policy was likely to have a greater impact on the UK born, at school-age, than previously thought. This finding was supported by a regression modelling study on the impact on TB incidence rates from the policy change. However, this study also found that the change in policy was associated with some benefits in UK born neonates and a much larger reduction in TB incidence rates in both non-UK born neonates and those at school-age. An additional regression study looking at the possible link between BCG vaccination and improved TB outcomes found some evidence that BCG vaccination was associated with reduced all-cause mortality, with little evidence of any other benefits. This result strengthens the case for wider vaccination. Additionally, a dynamic model of TB transmission was developed to provide a more detailed tool for evaluating the impact of the change in policy. Although this model was a poor fit to the observed data there was some evidence that continuing school-age vaccination would have resulted in fewer UK born TB cases but that neonatal vaccination reduced TB incidence in UK born children compared to continuing school-age vaccination. There was also some evidence that none of the vaccination strategies evaluated impacted incidence rates in older adults. These findings suggest a stronger case for the use of the BCG vaccine in school-age populations; in areas with an equivalent level of TB transmission to England; and where the effectiveness of the BCG vaccine has been shown to not reduce with age. They also indicate that a future vaccine, without the reduced effectiveness observed in some geographic areas, may be more effectively targeted at those at school-age than at neonates. However, this depends on the potential duration of protection conferred by vaccination. In addition, they highlight the trade-off between neonatal and school-age vaccination with school-age vaccination reducing overall TB incidence but neonatal vaccination reducing incidence in young children where more severe outcomes are more common. The findings from this thesis may be of use to policy-makers to inform vaccine usage both in the UK and globally. As a by-product of the work conducted in this thesis several open source tools have been developed. These tools maybe used as learning resources, for public engagement, and as part of other research projects.
References
3 The World Health Organization. BCG vaccine:WHO position paper. Weekly epidemiological record 2018;1–24.
22 Luca S, Mihaescu T. History of bcg vaccine. Maedica 2013;8.http://europepmc.org/articles/PMC3749764
Thesis GitHub: https://github.com/seabbs/thesis↩
Thesis website: https://www.samabbott.co.uk/thesis↩
tbinenglanddataclean: https://www.samabbott.co.uk/tbinenglanddataclean/↩prettypublisher: https://github.com/seabbs/prettypublisher↩See my personal site: https://www.samabbott.co.uk↩
Available here: https://github.com/seabbs/pebblegame↩