Bayesian approach to probability

 The frequentist approach for defining the probability of an uncertain event is all well and good providing that we have been able to record accurate information about many past instances of the event. However, if no such historical database exists, then we have to consider a different approach. Suppose, for example, we want to know the probability that a newly developed flight control system contains a critical fault. Since there are no previous instances of such systems we cannot use the frequentist approach to define our degree of belief in this uncertain event.

Bayesian probability is a formalism that allows us to reason about beliefs under conditions of uncertainty. If we have observed that a particular event has happened, such as Spurs winning the FA Cup in 1991, then there is no uncertainty about it. However, suppose a is the statement

"Spurs win the FA Cup in the year 2011"

Since this is a statement about a future event, nobody can state with any certainty whether or not it is true. Different people may have different beliefs in the statement depending on their specific knowledge of factors that might effect its likelihood. For example, Norman may have a strong belief in the statement a based on his knowledge of the current team and past achievements. Alan, on the other hand, may have a much weaker belief in the statement based on some inside knowledge about the status of Spurs; for example, he might know that the club is going to have to sell all of its best players in the year 2009.

Thus, in general, a person's subjective belief in a statement a will depend on some body of knowledge K. We write this as P(a|K). Norman's belief in a is different from Alan's because they are using different K's. However, even if they were using the same K they might still have different beliefs in a.

The expression P(a|K) thus represents a belief measure. Sometimes, for simplicity, when K remains constant we just write P(a), but you must be aware that this is a simplification.