The 95%-CI decision rule in psychological assessment from a decision
theoretic perspective
Authors: Florian Pargent, David Goretzko, Benedikt Friemelt
Psychological assessment sometimes requires a dichotomous decision,
whether a person is “below the norm” in some psychological domain. One
common decision rule is whether the upper boundary of a 95% confidence
interval for the person’s true value lies below the 16% quantile (one
standard deviation below the mean of a normal distribution) based on the
distribution of the true values in the population.
We demonstrate how the 95%-CI decision rule can be interpreted from the
perspective of (Bayesian) decision theory: it is equivalent with
minimizing expected loss when considering the type I error as 39 times
more severe than the type II error. Many practitioners might not be
aware of this implicit assumption. Thus, mindlessly applying any default
rule can lead to diagnostical decisions which are not in line with the
diagnostician’s judgement of the possible negative consequences of
decision errors in the concrete setting. In fact, “rational” optimal
decisions would require an explicit weighting of those errors. To stress
this point, we present a small survey of clinical neuropsychologists,
who had to report different representations of their internal loss
function for a fictitious diagnostical scenario. We look at how much
these judgements differ between practitioners and how much decisions
based on practitioners’ “true” loss functions differ from the 95%-CI
rule.