Total probability and Bayes' theorem

A partition of the sample space is a set of events E₁, E₂, ..., Eₙ that do not overlap and together cover everything, each with a positive chance. Think of them as the possible causes or routes: which bag was chosen, which machine made the item.

Theorem of total probability: the chance of a result A is the sum over all routes:

P(A) = P(E₁)P(A|E₁) + P(E₂)P(A|E₂) + ... + P(Eₙ)P(A|Eₙ)

A tree diagram makes this easy: one branch for each Eᵢ, then the chance of A along each branch. Multiply along each branch, and add the branches that end in A.

Bayes' theorem goes backwards: you have seen the result A, and you ask which route it probably came from.

P(Eᵢ|A) = P(Eᵢ)P(A|Eᵢ) / P(A): your branch ÷ all the branches that lead to A.

Example: Bag I has 2 white and 3 red balls; Bag II has 4 white and 1 red. A coin picks the bag, and a ball is drawn.

• P(white) = ½ × 2/5 + ½ × 4/5 = 1/5 + 2/5 = 3/5.
• The ball was white. P(it came from Bag II) = (2/5) ÷ (3/5) = 2/3.

The theorem is named after Thomas Bayes, an English mathematician; his result was published in 1763.