Showing 1–2 of 2 results for author: Greenland, S
-
Divergence vs. Decision P-values: A Distinction Worth Making in Theory and Keeping in Practice
Authors:
Sander Greenland
Abstract:
There are two distinct definitions of 'P-value' for evaluating a proposed hypothesis or model for the process generating an observed dataset. The original definition starts with a measure of the divergence of the dataset from what was expected under the model, such as a sum of squares or a deviance statistic. A P-value is then the ordinal location of the measure in a reference distribution compute…
▽ More
There are two distinct definitions of 'P-value' for evaluating a proposed hypothesis or model for the process generating an observed dataset. The original definition starts with a measure of the divergence of the dataset from what was expected under the model, such as a sum of squares or a deviance statistic. A P-value is then the ordinal location of the measure in a reference distribution computed from the model and the data, and is treated as a unit-scaled index of compatibility between the data and the model. In the other definition, a P-value is a random variable on the unit interval whose realizations can be compared to a cutoff alpha to generate a decision rule with known error rates under the model and specific alternatives. It is commonly assumed that realizations of such decision P-values always correspond to divergence P-values. But this need not be so: Decision P-values can violate intuitive single-sample coherence criteria where divergence P-values do not. It is thus argued that divergence and decision P-values should be carefully distinguished in teaching, and that divergence P-values are the relevant choice when the analysis goal is to summarize evidence rather than implement a decision rule.
△ Less
Submitted 21 September, 2023; v1 submitted 6 January, 2023;
originally announced January 2023.
-
There are natural scores: Full comment on Shafer, "Testing by betting: A strategy for statistical and scientific communication"
Authors:
Sander Greenland
Abstract:
Shafer (2021) offers a betting perspective on statistical testing which may be useful for foundational debates, given that disputes over such testing continue to be intense. To be helpful for researchers, however, this perspective will need more elaboration using real examples in which (a) the betting score has a justification and interpretation in terms of study goals that distinguishes it from t…
▽ More
Shafer (2021) offers a betting perspective on statistical testing which may be useful for foundational debates, given that disputes over such testing continue to be intense. To be helpful for researchers, however, this perspective will need more elaboration using real examples in which (a) the betting score has a justification and interpretation in terms of study goals that distinguishes it from the uncountable mathematical possibilities, and (b) the assumptions in the sampling model are uncertain. On justification, Shafer says 'No one has made a convincing case for any particular choice' of a score derived from a P-value and then states that 'the choice is fundamentally arbitrary'. Yet some (but not most) scores can be motivated by study goals (e.g., information measurement; decision making). The one I have seen repeatedly in information statistics and data mining is the surprisal, logworth or S-value s = -log(p), where the log base determines the scale. The present comment explains the rationale for this choice.
△ Less
Submitted 10 February, 2021;
originally announced February 2021.
