Frequentist versus Subjective view of Uncertainty
The
first fallacy of probability is one that needs to be understood before
we even attempt to define what probability means (that's why our
definition of probability is left until the next article). It is the fallacy that there is one and only one valid way to measure uncertainty.
Consider the following two statements:
- "There is a 50% chance of tossing a
Head on a fair coin"
- "There is a 0.0000001% chance of Martians landing on earth this year"
Each of these is a
statement that attempts to quantify our uncertainty about some unkown
event. But, although the statements are superficially similar
there are fundamental differences between them.
Statement 1 can be explained by a 'frequentist' arguments: if you toss a fair coin many times it will land as a Head 50% of the times.
Statement 2 has no such frequentist argument. We cannot 'play' this
year over and over again counting the number of times in which Martians
land. We can only provide a subjective measure of uncertainty based on
our current state of knowledge.
Some people (including even clever ones) feel comfortable with the
frequentist approach but so uncomfortable with the subjective approach
that they reject it is invalid. Their primary objections are that:
- the subjective measure cannot be validated
- different experts will give different subjective measures
The problem with these objections is
that they apply just as much to the frequentist approach. Even in the
coin tossing example, if we toss a coin 10,000 times it is almost
certain that Heads will NOT come up on exactly 5,000 occasions.
Moreover, different 'experts' running different sequences of
10,000 tosses would alsomt certainly arrive at different numbers of
Heads. Does that make the 50% figure invalid? Moreover, in less extreme
examples than statements 1 and 2, it is impossible to argue that the
frequentist approach is superior. Consider, for example, the following
two statements:
- "There is a 50.9% chance that a baby born in the UK is a girl"
- "There is a 5% chance of Spurs
winning the FA Cup this year"
There is no doubt that statement 3 is explained by a frequentist argument: Over the last 100 years 50.9% of all births recorded in the UK have been girls.
There is also no doubt that statement 4 has no such frequentist
explanation (and hence must be subjective) since there is only one FA
Cup this year and we cannot somehow play the tournament many times
and count the number of occasions on which Spurs win.
But if we dig a little deeper here, things get rather murky. The 50.9%
figure in statement 3 is actually based on many years of data that may
disguise crucial trend information. Suppose we discover that the
percentage of girls born is increasing; say a hundred years ago 48.5%
of babies were girls compared with 51.2% last year. Then surely the
probability of a randomly selected new born baby being a girl now is
higher than 50.9% (and higher than 51.2% if the figures have been
steadily increasing). And what exactly do we mean by a 'randomly'
selected baby. Surely what we are most interested in are specific
babies such as "the next baby born to Mrs Roberts of 213 White Hart
Land, London N17". In that case the frequency data may need to be
'adjusted' to take account of specific factors relevant to Mrs Roberts.
Both the general trend adjustments and the case specific adjustments
here clearly require the subjective judgment of relevant experts. But
that means, according to the frequentists, that their own approach is
no longer valid since, as we saw above:
- the measure cannot be validated
- different experts will give different subjective measures
Now look at statement 4 in
comparison. Although it is true that we cannot play the FA Cup more
than once this season, we can nevertheless consider the number of times
Spurs won the FA Cup in the last 100 years as a key factor informing
our subjective judgement. Of course past form (especially of the
distant past) is not a strong indicator of current form, but
can we say with true certainty that the situation was any different for
the past 'form' of babies born? It is not infeasible that drastic
changes in national figures could result from sudden environmental
changes. And just as Spurs might invest in the world's greatest
players to increase their chances of winning the FA Cup this year, so a
particular mother might apply a range of techniques to dramatically
increase or decrease the chances of having a girl.
Whatever anybody's objection to subjective measures, like it or not, they are used
so extensively that the fabric of modern society would break down
without them. Hence bookies will provide 'odds' on events (such as
Spurs winning the FA Cup) based on subjective measures, while insurance
companies will do the same in determining policy premiums and
governments the same when determing economic policies.
The frequentist approach for
measuring uncertainty is all well and good providing that we have been
able to record accurate information about many past instances of the
event. However, most uncertain events of interest do not have
such historical databases associated with them, and even where
relevant historical data does exist it must still usually be informed
by subjective judgements before it can be used for measuring
uncertainty. Hence, generally we cannot rely on the frequentist
approach to measure uncertainty.
The
subjective approach accepts unreservedly that different people
(even experts) may have vastly different beliefs about the uncertainty
of the same event. Hence Norman's belief about the chances of Spurs
winning the FA Cup this year may be very different from Daniel's. Norman,
using only his knowledge of the current team and past achievements may
rate the chances at 10%. Daniel, on the other hand, may rate the
chances as 2% based on some inside knowledge he has about key
players having to be sold in the next two months.
Hence the subjective approach is always based on some prior body of
knowledge. In this sense subjective measures of uncertainty are
always conditional on this prior knowledge. The subjective approach is also
called the Bayesian approach, because only in the Bayesian approach is
there a rigorous way of reasoning about such conditional knowledge.
Norman
Fenton
Return to Main Page
Making Sense of Probability: Fallacies, Myths and Puzzles