Bayes’ Theorem

In its general form, Bayes' theorem deals with specific events, such as A₁, A₂,...., A_k, that have prior probabilities. These events are mutually exclusive events that cover the entire sample space. Each prior probability is already known to the decision maker, and these probabilities have the following form: P(A₁), P(A₂),...., P(A_k). The events with prior probabilities produce, cause, or precede another event, say B.
A conditional probability relation exists between events A₁, A₂, ....., A_k and event B. The conditional probabilities are P(B|A₁), P(B|A₂), ..., P(B|A_k).

Bayes' formula allows us to calculate the probability of an event, say, A1 occurring given that event B has already occurred with a known probability, P(B). The probability of A₁ occurring given that B has already occurred is the posterior (or revised) probability. It is denoted by P(A₁|B). Thus, we are given P(A₁) and the P(B|A₁) which we use to calculate P(A₁|B).

For any event A_i, Bayes' theorem has the form

 The probability that A₁ and B occur simultaneously is equal to the probability that A₁ occurs multiplied by the probability that B occurs given A₁. Thus, we have

P(A1 and B) = P(A1) P(B|A1)

Since A₁, A₂, . . . . , Ak form a partition of the entire sample space when event B occurs, only one of the events in the partition occurs. Thus, we have

P(B) = P(A₁ and B) + P(A₂  and B) + .... + P(A_k and B)

We already know that for any event Ai,

P(Ai and B) = P(Ai) P(B|Ai)

When we substitute the formula for P(Ai and B) into the equation for P(B) we obtain

P(B) = P(A₁) P(B|A₁) + P(A₂) P(B|A₂) +...+ P(A_k) P(B|A_k)

If we then substitute P(B) and P(Ai and B) into the conditional probability, i.e. P(A|B) =      we obtain the generalized version of Bayes' formula, which is shown in the box.

Bayes' Theorem P(Ai | B)  =

Example 

Suppose that a personnel administrator wishes to hire one person from among a number of job applicants for a clerical position. The job to be filled is fairly simple. On the basis of past experience, the personnel director feels that there is a 0.80 probability of an applicant being able to fill the position. This probability is the prior probability.

A personnel administrator usually interviews or tests each applicant, rather than select one at random. This procedure supplies additional direct information about the applicant. In light of this additional information, the personnel director may revise the prior probability about an applicant's chances for success or failure at the job. The revised probability is the posterior probability.

The terms prior and posterior refer to the time when information is collected. Before information is obtained, we have prior probabilities. Bayes' theorem provides a means of calculating posterior probabilities from prior probabilities. The next example illustrates the use of Bayes' theorem.