Causal Bayesian reasoning in critical decision making
Bayesian networks (BNs) are causal probabilistic models that combine data and expert knowledge to quantify uncertainty, providing the most rigorous and rational basis for critical decision-making. Yet, theoretical and technical impediments restricted widespread use of BNs; decision-makers who most needed them (in finance, law, medicine, transport and defence) were unable to build and use the models required.
We made BNs widely accessible by extending the theory and technology, which was incorporated into a software system (AgenaRisk) that enables non-statisticians to develop and use BN-based systems for high stakes decision-making. AgenaRisk now has 8000 commercial and research users (6,000 new since 2008).
The research has had an economic impact and an impact on public policy and services. The economic impact has been the result of the methodological research on building causal BN models for risk assessment being ‘brought to market’ through a combination of direct collaborative consultancy projects and public dissemination. Implementation of the models has been made possible by the fact that Agena has incorporated much of the published algorithmic and usability research into its AgenaRisk toolset. A free version of AgenaRisk can be downloaded from www.agenarisk.com. Since 2008 AgenaRisk has added over 6000 registered users, including several hundred commercial users with paid licenses.
Impact on Economy
There has been economic benefit to both Agena and its end-users who include many of the world’s leading technology, defence and transport organisations. Since most sales of AgenaRisk have been to organisations outside of the UK there has also been a direct economic benefit to “UK plc”. For many technology companies, the defining feature of the application was the development of genuine causal models (taking account of expert knowledge) compared to pure statistical or machine learning models that had been previously used, leading to decision-support systems that improved profitability, accuracy, reliability etc. For example, one leading technology company, known around the world for innovation and leadership in wireless and broadband communications, stated in a communication with Agena that they achieved annual savings of over $5m by using AgenaRisk to calculate the reliability and increase the quality of new electronic products.
The most recent and exciting developments have been in finance and the law.
In finance, our models of operational risk assessment (see Neil, M. and Hager, D., "Modeling Operational Risk in Financial Institutions using Hybrid Dynamic Bayesian Networks". Journal of Operational Risk, 4(1), 2009; Neil, M., Marquez, D., and Fenton, N., Using Bayesian Networks to Model the Operational Risk to Information Technology Infrastructure in Financial Institutions. Journal of Financial Transformation, 2008. 22: p. 131-138) have been used by banks in the UK, Ireland, South Africa and Norway.
One international bank uses our models for routine identification of risk in new and on-going maintenance projects. Another has used the methodology for legal, treasury and retail banking risk evaluation. A company responsible for national financial infrastructure uses our models for risk and vulnerability evaluation, while a leading reinsurance company used them for comparing accuracy of competing models for predicting insurance losses arising from catastrophic flood events. A world leading actuarial firm has adopted our approach worldwide for financial risk assessment and regulatory compliance.
Impact on Public Policy and Services
In the legal domain, the impact has been in public policy and services. Lawyers and forensic scientists have been incorporating our approach to using BNs for evidence evaluation, while the Crown Prosecution Service is considering it to help determine whether or not a prosecution should proceed. The research has led to a demand for our services as expert witnesses in major criminal and civil cases including:
- a major case on software quality and risk assessment involving safety critical software in the rail industry.
- a medical negligence case against the NHS in which our approach enabled lawyers to properly quantify the risks of different treatment pathways (see: Fenton, N. and M. Neil (2010). "Comparing risks of alternative medical diagnosis using Bayesian arguments." Journal of Biomedical Informatics 43: 485-495 and Fenton, N. E. (2011). "Science and law: Improve statistics in court." Nature 479: 36-37);
- the trial of Levi Bellfield at the Old Bailey in February 2008 (see Fenton, N. E. and M. Neil (2011). "Avoiding Legal Fallacies in Practice Using Bayesian Networks." Australian Journal of Legal Philosophy 36: 114-150);
- the retrial of R v D (one of the most infamous murder cases in British legal history) 2011-2012;
- the case of R v K, 2012 (Croydon riots of the summer of 2012);
- an ongoing case (R v LW) – where our analysis exposed inconsistencies and flaws in the presentation of DNA evidence.
We also recently exposed flaws in the legal reasoning central to the successful appeal of Barry George who had been convicted of the murder of TV presenter Jill Dando (see Fenton, N. E., et al, (2013). "When ‘neutral’ evidence still has probative value (with implications from the Barry George Case)", Science and Justice, dx.doi.org/10.1016/j.scijus.2013.07.002). A new paper Fenton, N. E., Neil, M., & Hsu, A. (2013). "Calculating and understanding the value of any type of match evidence when there are potential testing errors". Artificial Intelligence and Law, provides important guidelines to forensic scientists when presenting evidence, based on our experiences of working with forensic scientists and lawyers.
Since June 2011 our approach has been the focus of an international consortium aimed at addressing the problems of expert witness presentation of Bayesian analyses. The consortium now numbers 85 experts and has been the subject of feature articles in the Guardian ("A formula for justice", 3 October 2011, http://www.theguardian.com/law/2011/oct/02/formula-justice-bayes-theorem-miscarriage) and Nature ("Science and law: Improve statistics in court." Nature 479: 36-37, 2011). Prof Fenton was one of 15 UK academics invited to contribute to the Home Office workshop on “Forecasting rare and extreme criminal events”, 12 June 2009.
Hover mouse over image to enlarge.
In addition to the normal academic publishing route, public dissemination of our research methods has been achieved through a number of popular websites promoting improved public understanding of BN methods (see, e.g. http://probabilityandlaw.blogspot.co.uk/).
One site (www.pi-football.com/ ) maintained by one of our former PhD students publishes the predictions each week of premiership football matches based on ‘bookie beating’ BN models arising from our published research in this area (a full list of these publications is here: http://constantinou.info/publications.aspx).
In terms of impact on understanding BNs and risk assessment, the publication of the book (Fenton, N. E. and M. Neil (2012). Risk Assessment and Decision Analysis with Bayesian Networks. CRC Press) is having a game-changing impact. The book, which has a foreword written by the 2011 Turing prize winner Judea Pearl, is the first to really bring BN technology to non-academics. It has received outstanding endorsements from many leading professionals (see reviews at www.crcpress.com/product/isbn/9781439809105). On amazon.com and amazon.co there are already 21 reader reviews of which 19 are 5* and 2 are 4*; none of the other dozens of books written on Bayesian networks over many years have accumulated even 10 reviews in total. Moreover, despite supply problems, sales are already over 2000 and the book is currently CRC Press’s top Kindle seller in maths and stats.
Typical mathematical approaches to risk assessment and decision analysis rely on statistical modelling based on historical data. But there is an increasing concern with risks that are rare or novel (e.g. catastrophic banking failures or terrorist attacks). For these there is insufficient (or no) relevant data. Moreover, even when dealing with common risks with extensive historical data (e.g. in medicine and finance), crucial data on key causal and explanatory factors is typically missing. Traditional mathematical and statistical approaches do not work in these situations. We have pioneered the use of causal probabilistic models (called Bayesian networks) to provide improved quantified risk assessment and decision-making in such situations.
A Bayesian network (BN) can capture complex interdependencies between risk factors and effectively combine data with expert judgement to provide rigorous risk quantification and decision support for risk management. Although BNs have been around since the late 1980’s, they are most typically used in a machine learning context, where it is assumed the causal structure and relationships are learnt from data. But for risk assessment this approach suffers from the exact limitations of the statistical models described above. Even for highly constrained problems, with small numbers of variables, it only works with very large volumes of relevant data, and it suffers from the same problem of unobserved factors. In other words, most BN applications are not truly causal models of the problem.
We first pioneered a genuine causal BN modelling approach, incorporating expert knowledge and data, in [R1] to the problem of defect prediction; this work formed the foundation for subsequent advanced modelling of project risk as described [R6]. The underlying philosophy has also led to a methodology for a) measuring and predicting resilience and vulnerability in complex systems such as in banking; b) modelling and evaluation of sensor technologies in real world settings by accounting for hard and soft factors (linking to psychology, privacy and the law);
Our early modelling work identified limitations of existing BN tools and methods – it was generally too difficult for people without statistical and Bayesian expertise to develop and use BNs to solve real world risk problems. Consequently, the focus of our most important research has since been on addressing the limitations of BN tools and methods with respect to usability and accuracy. In particular, we have:
- Developed breakthrough BN algorithms [R4] that enable modellers to use continuous variables alongside discrete ones, without going through the painful (and intrinsically inaccurate) process of static discretisation. This makes the development of realistic BNs drastically simpler and enables a broader class of risk assessment problems to be solved with much greater accuracy [R3] .
- Developed a range of novel elicitation and modelling techniques that make it much easier for domain experts (who are not trained in Bayesian methods or statistics) to build and use models. This includes breakthrough work on BN idioms [R2] (which provide common templates for building BNs); and ranked nodes in [R5] (which massively reduces the burden on defining probability tables required for BN models).
- [R1] Fenton NE and Neil M, ''A Critique of Software Defect Prediction Models'', 25(5) IEEE Transactions on Software Engineering, 675-689,1999 doi.ieeecomputersociety.org/10.1109/32.815326 (753 citations)
- [R2] Neil, M., N. Fenton, Nielsen, L. (2000). "Building large-scale Bayesian Networks." The Knowledge Engineering Review, 15(3) 15(3): 257-284. doi.10.1017/S0269888900003039 (186 citations)
- [R3] Neil, M., D. Marquez and N. E. Fenton (2010). "Improved Reliability Modeling using Bayesian Networks and Dynamic Discretization." Reliability Engineering & System Safety 95(4): 412-425. DOI: 10.1016/j.ress.2009.11.012 (31 citations)
- [R4] Neil M. Tailor M. Marquez D. Inference in Bayesian Networks using Dynamic Discretisation. Statistics and Computing 17:3 September 2007. 219-233, DOI: 10.1007/s11222-007-9018-y (54 citations)
- [R5] Fenton NE, Neil M, and Caballero JG, "Using Ranked nodes to model qualitative judgements in Bayesian Networks" IEEE TKDE 19(10), 1420-1432, Oct 2007 (74 citations)
- [R6] Fenton NE, Neil M, Hearty P, Marsh W, Marquez D, Krause P, Mishra R, "Predicting Software Defects in Varying Development Lifecycles using Bayesian Nets", Information & Software Technology, Vol 49, pp 32-43, Jan 2007 DOI: 10.1016/j.infsof.2006.09.001 (121 citations)