Mathematics

Conditional Probability

Conditional probability is a fundamental concept in probability theory that describes the probability of an event occurring given that another event has already occurred. It is a measure of the likelihood of one event in the context of some other event. it is frequently used in quantitative finance to calculate the likelihood of events occurring given certain other events have occurred.

Random Variables I Random Variables II The Covariance Matrix Multiple Random Variables Moments Combinatorics Conditional Probability Markov Chains Random Processes Stochastic Calculus

Our perception of event probabilities change as we learn new information. Imagine that you predict the probability that it will rain in London today. After you have made the prediction, I tell you that there are currently dark clouds above London. You then can update your prediction accordingly. Clearly, this new information should make you increase your predicted probability. Considering probabilities in this way is known as conditional probability - the probability of an event occurring conditioned on something else being true. Your initial prediction is known as your prior probability, whereas your updated prediction given new information is known as your posterior probability. Consider two random variables $X$ and $Y$. We saw in the last section that if the random variables are independent then $$ P(X \cap Y) = P(X) P(Y). $$ If the variables are not independent then the outcome of one affects the outcome of the other. We can define the outcome of $X$ given an outcome $Y$ as $$ P(X|Y) = \frac{P(X \cap Y)}{P(Y)} $$ where $\cap$ denotes the intersection of $X$ and $Y$.

We calculate the probability of rolling a 6, conditioned on the fact that $X>3$. $$ P(X=6|X>3) = \frac{P(X=6 \cap X>3)}{P(X>3)} = \frac{1}{6} \cdot \frac{1}{0.5} = \frac{1}{3}. $$

Imagine we have a complete set of mutually exclusive potential events ${Y_k}$ which span the full sample space. A consequence of this definition is that only one of the ${Y_k}$ can occur at any time. Then, it is the case that $$ P(X) = \sum_{k=1}^n P(X | Y_k) P(Y_k). $$ The above equation is known as the law of total probability.

We can use the law of total probability. $$ \begin{align*} P(\text{white}) = P(\text{white} | \text{drawer 1}) P(\text{drawer 1}) + P(\text{white} | \text{drawer 2}) P(\text{drawer 2})\\ = (0 \times 0.5) + (0.5 \times 0.5)\\ =\frac{1}{4}. \end{align*} $$

Technical interviews questions will frequently use the law of total probability. However, with some practice it's use will become second nature. Now let's introduce perhaps the most famous theorem in conditional probability - Bayes' Theorem. $$ P(A|B) = \frac{P(A)}{P(B)} \cdot P(B|A) $$ Bayes' Theorem is a fundamental concept in probability theory that describes the likelihood of an event based on prior knowledge of conditions related to the event. It provides a mathematical framework for updating the probability of a hypothesis (like a market trend or the value of an asset) in light of new evidence or information.

The definition of conditional probability is $$ P(A|B) = \frac{P(A \cap B)}{P(B)} $$ so equivalently, $$ P(B|A) = \frac{P(B \cap A)}{P(A)}. $$ Since $P(A \cap B) = P(B \cap A)$, then it follows that $$ P(A|B)P(B) = P(B|A) P(A) $$ and rearranging this equation gives us the result.

The information we have is $$ P(\text{test pos} | \text{not diseased}) = 0.05 $$ $$ P(\text{test neg} | \text{diseased}) = 0.10 $$ $$ P(\text{diseased}) = 0.02 $$ We can infer that $$ P(\text{test neg} | \text{not diseased}) = 0.95 $$ and $$ P(\text{test pos} | \text{diseased}) = 0.90 $$ because with probability 1 you will test either positive or negative. The formula we need to solve is $$ P(\text{diseased} | \text{test pos}) = \frac{P(\text{diseased})}{P(\text{test pos})} \cdot P(\text{test pos} | \text{diseased}) $$ We can use the law of total probability to calculate $P(\text{test pos})$. $$ \begin{align*} P(\text{test pos}) = P(\text{test pos} | \text{diseased}) P(\text{diseased}) + P(\text{test pos} | \text{not diseased}) P(\text{not diseased})\\ = (0.9 \times 0.02) + (0.05 \times 0.98) = 0.269 \end{align*} $$ The probability of being diseased given that you test positive is only $26.9\%$.