-
Calculating the Likelihood Ratio for Multiple Pieces of Evidence
Authors:
Norman Fenton,
Martin Neil
Abstract:
When presenting forensic evidence, such as a DNA match, experts often use the Likelihood ratio (LR) to explain the impact of evidence . The LR measures the probative value of the evidence with respect to a single hypothesis such as 'DNA comes from the suspect', and is defined as the probability of the evidence if the hypothesis is true divided by the probability of the evidence if the hypothesis i…
▽ More
When presenting forensic evidence, such as a DNA match, experts often use the Likelihood ratio (LR) to explain the impact of evidence . The LR measures the probative value of the evidence with respect to a single hypothesis such as 'DNA comes from the suspect', and is defined as the probability of the evidence if the hypothesis is true divided by the probability of the evidence if the hypothesis is false. The LR is a valid measure of probative value because, by Bayes Theorem, the higher the LR is, the more our belief in the probability the hypothesis is true increases after observing the evidence. The LR is popular because it measures the probative value of evidence without having to make any explicit assumptions about the prior probability of the hypothesis. However, whereas the LR can in principle be easily calculated for a distinct single piece of evidence that relates directly to a specific hypothesis, in most realistic situations 'the evidence' is made up of multiple dependent components that impact multiple different hypotheses. In such situations the LR cannot be calculated . However, once the multiple pieces of evidence and hypotheses are modelled as a causal Bayesian network (BN), any relevant LR can be automatically derived using any BN software application.
△ Less
Submitted 9 June, 2021;
originally announced June 2021.
-
Positive results from UK single gene testing for SARS-COV-2 may be inconclusive, negative or detecting past infections
Authors:
Martin Neil
Abstract:
The UK Office for National Statistics (ONS) publish a regular infection survey that reports data on positive RT-PCR test results for SARS-COV-2 virus. This survey reports that a large proportion of positive test results may be based on the detection of a single target gene rather than on two or more target genes as required in the manufacturer instructions for use, and by the WHO in their emergenc…
▽ More
The UK Office for National Statistics (ONS) publish a regular infection survey that reports data on positive RT-PCR test results for SARS-COV-2 virus. This survey reports that a large proportion of positive test results may be based on the detection of a single target gene rather than on two or more target genes as required in the manufacturer instructions for use, and by the WHO in their emergency use assessment. Without diagnostic validation, for both the original virus and any variants, it is not clear what can be concluded from a positive test resulting from a single target gene call, especially if there was no confirmatory testing. Given this, many of the reported positive results may be inconclusive, negative or from people who suffered past infection for SARS-COV-2.
△ Less
Submitted 18 March, 2021; v1 submitted 23 February, 2021;
originally announced February 2021.
-
Product risk assessment: a Bayesian network approach
Authors:
Joshua Hunte,
Martin Neil,
Norman Fenton
Abstract:
Product risk assessment is the overall process of determining whether a product, which could be anything from a type of washing machine to a type of teddy bear, is judged safe for consumers to use. There are several methods used for product risk assessment, including RAPEX, which is the primary method used by regulators in the UK and EU. However, despite its widespread use, we identify several lim…
▽ More
Product risk assessment is the overall process of determining whether a product, which could be anything from a type of washing machine to a type of teddy bear, is judged safe for consumers to use. There are several methods used for product risk assessment, including RAPEX, which is the primary method used by regulators in the UK and EU. However, despite its widespread use, we identify several limitations of RAPEX including a limited approach to handling uncertainty and the inability to incorporate causal explanations for using and interpreting test data. In contrast, Bayesian Networks (BNs) are a rigorous, normative method for modelling uncertainty and causality which are already used for risk assessment in domains such as medicine and finance, as well as critical systems generally. This article proposes a BN model that provides an improved systematic method for product risk assessment that resolves the identified limitations with RAPEX. We use our proposed method to demonstrate risk assessments for a teddy bear and a new uncertified kettle for which there is no testing data and the number of product instances is unknown. We show that, while we can replicate the results of the RAPEX method, the BN approach is more powerful and flexible.
△ Less
Submitted 9 October, 2020;
originally announced October 2020.
-
On the limitations of probabilistic claims about the probative value of mixed DNA profile evidence
Authors:
Norman Fenton,
Allan Jamieson,
Sara Gomes,
Martin Neil
Abstract:
The likelihood ratio (LR) is a commonly used measure for determining the strength of forensic match evidence. When a forensic expert determines a high LR for DNA found at a crime scene matching the DNA profile of a suspect they typically report that 'this provides strong support for the prosecution hypothesis that the DNA comes from the suspect'. However, even with a high LR, the evidence might no…
▽ More
The likelihood ratio (LR) is a commonly used measure for determining the strength of forensic match evidence. When a forensic expert determines a high LR for DNA found at a crime scene matching the DNA profile of a suspect they typically report that 'this provides strong support for the prosecution hypothesis that the DNA comes from the suspect'. However, even with a high LR, the evidence might not support the prosecution hypothesis if the defence hypothesis used to determine the LR is not the negation of the prosecution hypothesis (such as when the alternative is 'DNA comes from a person unrelated to the defendant' instead of 'DNA does not come from the suspect'). For DNA mixture profiles, especially low template DNA (LTDNA), the value of a high LR for a 'match' - typically computed from probabilistic genotyping software - can be especially questionable. But this is not just because of the use of non-exhaustive hypotheses in such cases. In contrast to single profile DNA 'matches', where the only residual uncertainty is whether a person other than the suspect has the same matching DNA profile, it is possible for all the genotypes of the suspect's DNA profile to appear at each locus of a DNA mixture, even though none of the contributors has that DNA profile. In fact, in the absence of other evidence, we show it is possible to have a very high LR for the hypothesis 'suspect is included in the mixture' even though the posterior probability that the suspect is included is very low. Yet, in such cases a forensic expert will generally still report a high LR as 'strong support for the suspect being a contributor'. Our observations suggest that, in certain circumstances, the use of the LR may have led lawyers and jurors into grossly overestimating the probative value of a LTDNA mixed profile 'match'
△ Less
Submitted 18 September, 2020;
originally announced September 2020.
-
The role of collider bias in understanding statistics on racially biased policing
Authors:
Norman Fenton,
Martin Neil,
Steven Frazier
Abstract:
Contradictory conclusions have been made about whether unarmed blacks are more likely to be shot by police than unarmed whites using the same data. The problem is that, by relying only on data of 'police encounters', there is the possibility that genuine bias can be hidden. We provide a causal Bayesian network model to explain this bias, which is called collider bias or Berkson's paradox, and show…
▽ More
Contradictory conclusions have been made about whether unarmed blacks are more likely to be shot by police than unarmed whites using the same data. The problem is that, by relying only on data of 'police encounters', there is the possibility that genuine bias can be hidden. We provide a causal Bayesian network model to explain this bias, which is called collider bias or Berkson's paradox, and show how the different conclusions arise from the same model and data. We also show that causal Bayesian networks provide the ideal formalism for considering alternative hypotheses and explanations of bias.
△ Less
Submitted 16 July, 2020;
originally announced July 2020.
-
Simpson's Paradox and the implications for medical trials
Authors:
Norman Fenton,
Martin Neil,
Anthony Constantinou
Abstract:
This paper describes Simpson's paradox, and explains its serious implications for randomised control trials. In particular, we show that for any number of variables we can simulate the result of a controlled trial which uniformly points to one conclusion (such as 'drug is effective') for every possible combination of the variable states, but when a previously unobserved confounding variable is inc…
▽ More
This paper describes Simpson's paradox, and explains its serious implications for randomised control trials. In particular, we show that for any number of variables we can simulate the result of a controlled trial which uniformly points to one conclusion (such as 'drug is effective') for every possible combination of the variable states, but when a previously unobserved confounding variable is included every possible combination of the variables state points to the opposite conclusion ('drug is not effective'). In other words no matter how many variables are considered, and no matter how 'conclusive' the result, one cannot conclude the result is truly 'valid' since there is theoretically an unobserved confounding variable that could completely reverse the result.
△ Less
Submitted 3 December, 2019;
originally announced December 2019.
-
Modelling Competing Legal Arguments using Bayesian Model Comparison and Averaging
Authors:
Martin Neil,
Norman Fenton,
David Lagnado,
Richard D. Gill
Abstract:
Bayesian models of legal arguments generally aim to produce a single integrated model, combining each of the legal arguments under consideration. This combined approach implicitly assumes that variables and their relationships can be represented without any contradiction or misalignment, and in a way that makes sense with respect to the competing argument narratives. This paper describes a novel a…
▽ More
Bayesian models of legal arguments generally aim to produce a single integrated model, combining each of the legal arguments under consideration. This combined approach implicitly assumes that variables and their relationships can be represented without any contradiction or misalignment, and in a way that makes sense with respect to the competing argument narratives. This paper describes a novel approach to compare and 'average' Bayesian models of legal arguments that have been built independently and with no attempt to make them consistent in terms of variables, causal assumptions or parametrisation. The approach involves assessing whether competing models of legal arguments are explained or predict facts uncovered before or during the trial process. Those models that are more heavily disconfirmed by the facts are given lower weight, as model plausibility measures, in the Bayesian model comparison and averaging framework adopted. In this way a plurality of arguments is allowed yet a single judgement based on all arguments is possible and rational.
△ Less
Submitted 30 January, 2020; v1 submitted 7 March, 2019;
originally announced March 2019.
-
Comment: Expert Elicitation for Reliable System Design
Authors:
Norman Fenton,
Martin Neil
Abstract:
Comment: Expert Elicitation for Reliable System Design [arXiv:0708.0279]
Comment: Expert Elicitation for Reliable System Design [arXiv:0708.0279]
△ Less
Submitted 2 August, 2007;
originally announced August 2007.
