Probability

1. Check this before you revise anything

The introduction promises four topics the chapter never delivers. Its opening paragraph states: "We shall also learn an important concept of random variable and its probability distribution and also the mean and variance of a probability distribution. In the last section of the chapter, we shall study an important discrete probability distribution called Binomial distribution."

None of that is taught. The actual section list stops at 13.5:

There is no section 13.6 and no section 13.7 — a full-text search of the 33-page chapter returns zero hits for either. Checking the promised topics one by one against the whole chapter:

• "variance" appears exactly once in the entire chapter, and that single occurrence is the sentence promising it. It is never defined, never computed, never examined.
• "mean of a random variable" returns zero hits.
• "Bernoulli" and "binomial" appear only in the Historical Note, describing Jacob Bernoulli's 1713 book Ars Conjectendi, plus that one introductory sentence. There is no formula, no worked example, no exercise.
• "random variable" survives only as a single Remark placed after the Bayes' theorem material: "A random variable is a real valued function whose domain is the sample space of a random experiment," followed by one coin-toss example. Exercise 13.3 then begins immediately. There is no probability distribution table anywhere.

The old stub taught precisely the removed material — it carried "Random Variables and Probability Distributions" and "Binomial Distribution" as two of its own full sections, along with a comparison table of "Classical vs Conditional vs Binomial Probability." All of that is gone from this rebuild.

One exercise question still requires the deleted topic. Miscellaneous Exercise Q4 asks for "the probability that at most 6 of a random sample of 10 people are right-handed," which is a binomial calculation with $n=10$ and $p=0.9$. The chapter provides no tool for it. It is solved in full on the exercise page, with the gap noted.

The old stub's exercise list was invented. Its meta declared three exercises named "Conditional probability and independence" (10 questions), "Bayes' theorem" (8) and "Distributions" (10). None of those names or counts exist in the book. The real structure is:

62 questions in total, none of which the old stub had a solutions file for.

A genuine flaw survives in Exercise 13.2 Q10. It gives $P(A)=\frac{1}{2}$, $P(B)=\frac{7}{12}$ and $P(\text{not }A\text{ or not }B)=\frac{1}{4}$. By De Morgan's law the last forces $P(A\cap B)=\frac{3}{4}$ — which is larger than $P(A)=\frac{1}{2}$, and an intersection can never be more probable than an event containing it. The data is impossible as printed. The expected answer, "not independent," still follows from the product-rule test, and the solution notes the inconsistency rather than silently working around it.

2. Conditional Probability (Textbook 13.2)

Sometimes an event is known to have occurred, and that knowledge changes the probability of another event. The conditional probability of $E$ given that $F$ has occurred is written $P(E|F)$.

$P(E|F)=\frac{P(E\cap F)}{P(F)},P(F)\ne 0$

Why the denominator changes. Knowing $F$ has occurred means the sample space effectively shrinks to $F$. The favourable outcomes are those in both $E$ and $F$, and they are now measured against $F$ rather than against the whole space. Dividing by $P(F)$ rescales the restricted space so its total probability is $1$ again.

The condition $P(F)\ne 0$ is not decorative. If $P(F)=0$, the quotient is undefined — not zero. Exercise 13.1 Q16 tests exactly this, and "not defined" is the expected answer.

For equally likely outcomes the formula reduces to counting: $P(E|F)$ is the number of outcomes in $E\cap F$ divided by the number in $F$. Most of Exercise 13.1 is solved this way, by listing a small sample space and counting.

Properties (13.2.1). Writing $S$ for the sample space:

$P(S|F)=P(F|F)=1$

$P((A\cup B)|F)=P(A|F)+P(B|F)-P((A\cap B)|F)$

$P(E^{\prime }|F)=1-P(E|F)$

The middle one is the ordinary addition rule holding inside the conditioned space, and the last is the complement rule doing the same. Conditional probability behaves like an ordinary probability once the space has been restricted.

Conditional probability is not symmetric. $P(E|F)$ and $P(F|E)$ share a numerator but have different denominators, so they are generally different numbers. Exercise 13.1 Q1 makes the point immediately, giving $\frac{2}{3}$ and $\frac{1}{3}$. Treating them as interchangeable is the most common conceptual error in the chapter.

3. Multiplication Theorem on Probability (Textbook 13.3)

Rearranging the definition of conditional probability gives a way to compute the probability that two events both occur.

$P(E\cap F)=P(E)P(F|E)=P(F)P(E|F)$

Both forms are valid, provided the conditioning event has non-zero probability. Choose whichever conditional probability is easier to read off the problem.

Why this matters for drawing without replacement. When two balls are drawn one after another without replacement, the second draw depends on the first. The multiplication rule handles it directly: the probability of two black balls from an urn of 10 black and 5 white is $\frac{10}{15}\times \frac{9}{14}$, where the second factor reflects the urn after the first ball is gone.

Extension to three or more events:

$P(E\cap F\cap G)=P(E)P(F|E)P(G|(E\cap F))$

Each new factor is conditioned on everything already assumed to have happened. The pattern extends to any number of events, and Exercise 13.2 Q3 applies it to three draws in a row.

4. Independent Events (Textbook 13.4)

Two events are independent when the occurrence of one does not change the probability of the other:

$P(E\cap F)=P(E)\cdot P(F)$

Equivalently $P(E|F)=P(E)$ and $P(F|E)=P(F)$, whenever those conditional probabilities are defined.

Testing for independence always means computing all three quantities — $P(E)$, $P(F)$ and $P(E\cap F)$ — and checking whether the product rule holds exactly. Several Exercise 13.2 questions produce values that are close but unequal, such as $\frac{9}{50}$ against $\frac{10}{50}$, so the comparison must be done with fractions rather than decimals.

Independent is not the same as mutually exclusive. This is the single most confused pair in the chapter:

Two mutually exclusive events with non-zero probabilities are in fact strongly dependent — knowing one occurred tells you the other definitely did not. They are independent only in the degenerate case where one has probability zero.

Independence survives complementation. If $A$ and $B$ are independent, then so are $A$ and $B^{\prime }$, $A^{\prime }$ and $B$, and $A^{\prime }$ and $B^{\prime }$. This is what makes "neither occurs" computations quick: $P(A^{\prime }\cap B^{\prime })=P(A^{\prime })P(B^{\prime })$. Exercise 13.2 Q18 tests exactly this equivalence.

Three or more events are called mutually independent only if the product rule holds for every subset, not just for pairs.

5. Partition, Total Probability, and Bayes' Theorem (Textbook 13.5)

Partition of a sample space (13.5.1). A collection of events $E_1,E_2,\dots ,E_n$ partitions $S$ if they are pairwise disjoint, each has non-zero probability, and together they exhaust $S$. Exactly one of them occurs on any trial.

Theorem of total probability (13.5.2). For any event $A$ and any partition $\{E_i\}$:

$P(A)={\sum }_{i=1}^{n}P(E_i)P(A|E_i)$

The idea is to split $A$ across the mutually exclusive cases, compute its probability within each, and weight each by how likely that case was. Exercise 13.3 Q1 uses it on its own: the second ball's colour depends on which colour was drawn first, so the two branches are computed separately and combined.

Bayes' theorem. This runs the reasoning backwards. Total probability goes from causes to an effect; Bayes goes from an observed effect back to the probability of each cause:

$P(E_i|A)=\frac{P(E_i)P(A|E_i)}{{\sum }_{j=1}^{n}P(E_j)P(A|E_j)}$

The denominator is exactly $P(A)$ from the total probability theorem, so the two results are used together as a matter of course.

The vocabulary. $P(E_i)$ are the prior probabilities, known before the evidence. $P(A|E_i)$ are the likelihoods. $P(E_i|A)$ are the posterior probabilities, revised in light of the evidence.

Setting a Bayes problem up. The mechanical part is easy; identifying which quantity is which is where marks are lost.

1. Name the partition — the competing causes, hypotheses or sources.
2. Write down the priors, which must sum to $1$.
3. Write down the likelihood of the observed evidence under each cause.
4. The numerator is the term for the cause you are asked about.
5. The denominator is the sum of all such terms.

Reading the question carefully matters. Exercise 13.3 Q8 gives machine A a larger share of production but asks about machine B, so B's term belongs in the numerator. Putting the wrong cause on top is a routine error.

When the priors are equal they cancel. If every $P(E_i)$ is the same, the common factor divides out of numerator and denominator, and the answer is just one likelihood over the sum of the likelihoods. Questions 6 and 12 of Exercise 13.3 both simplify this way.

Worked, mirroring the textbook's own technique. A blood test is 99% effective when the disease is present, gives a false positive for 0.5% of healthy people, and 0.1% of the population has the disease. Find the probability that a person with a positive test actually has it.

Let $D$ be 'has the disease', so $P(D)=0.001$ and $P(D^{\prime })=0.999$. Let $T$ be 'tests positive', with $P(T|D)=0.99$ and $P(T|D^{\prime })=0.005$.

$P(D|T)=\frac{0.001\times 0.99}{0.001\times 0.99+0.999\times 0.005}=\frac{0.00099}{0.005985}=\frac{22}{133}\approx 0.165$

The answer is only about 17%, which routinely surprises students. Because the disease is rare, the small false-positive rate applied to the very large healthy population produces more false positives than there are true positives.

Summary

• $P(E|F)=\frac{P(E\cap F)}{P(F)}$ with $P(F)\ne 0$; the denominator rescales the sample space, which has effectively shrunk to $F$.
• If $P(F)=0$ the conditional probability is not defined — not zero.
• Conditional probability is not symmetric: $P(E|F)$ and $P(F|E)$ share a numerator but differ in denominator.
• Inside a conditioned space the usual rules still hold: $P(S|F)=1$, the addition rule, and $P(E^{\prime }|F)=1-P(E|F)$.
• Multiplication theorem: $P(E\cap F)=P(E)P(F|E)=P(F)P(E|F)$, extending to $P(E)P(F|E)P(G|E\cap F)$ for three events.
• Drawing without replacement is the standard use of the multiplication rule, since each draw changes what remains.
• Independent means $P(E\cap F)=P(E)P(F)$, equivalently $P(E|F)=P(E)$; test it with exact fractions, not decimals.
• Mutually exclusive is not independent — disjoint events with non-zero probability are strongly dependent.
• Independence is preserved under complementation, so $P(A^{\prime }\cap B^{\prime })=P(A^{\prime })P(B^{\prime })$.
• Total probability: $P(A)=\sum P(E_i)P(A|E_i)$ over a partition — split into cases, then weight.
• Bayes' theorem reverses it, giving $P(E_i|A)$ from priors and likelihoods, with $P(A)$ as the denominator.
• Identify the partition, the priors and the likelihoods before computing, and put the cause the question asks about in the numerator.
• Equal priors cancel out, reducing Bayes to one likelihood over the sum of the likelihoods.
• A rare condition plus a small false-positive rate gives a low posterior even after positive evidence — the base rate dominates.
• Random variables, probability distributions, mean, variance and the Binomial distribution are not taught in the current edition, despite the introduction promising all of them. "Variance" occurs once in the whole chapter, in that promise.
• Miscellaneous Exercise Q4 nonetheless requires the binomial distribution; it is solved in full on the exercise page.