Mathematics

Random Variables I

This course is designed for those who are interested in understanding and applying discrete random variables in the context of quantitative finance. The second part of the series, Random Variables II, will cover continuous random variables and will build upon the intuition developed in this course. The concept of random variables underpin much of modern finance, and as such a deep working knowledge of random variables is of paramount importance.

Random Variables I Random Variables II The Covariance Matrix Multiple Random Variables Moments Combinatorics Conditional Probability Markov Chains Random Processes Stochastic Calculus

The random variable is a concept that lies at the heart of quantitative finance. A random variable is a quantity that can take on different values with certain probabilities. For example, a die roll is a random variable with outcomes $\{1,2,3,4,5,6\}$. For a fair die, all outcomes are equally likely with probability $\frac{1}{6}$. The mathematical framework of random variables gives us the toolset to analyse uncertain events, make predictions and assess the likelihood of certain events occurring. Let's dive into some mathematics. Let us define a random variable, $X$. Measuring $X$ gives us an outcome $X_k$ with probability $p_k$. An equivalent way of denoting $p_k$ is to write $P(X = X_k)$, the probability that $X$ takes on the specific value $X_k$. $P$ is known as the probability mass function, which is a function of the space that the outcomes lie on - in the case of a die roll, it is the natural numbers. You can draw a probability mass function by plotting the probabilities on the y-axis, and outcomes on the x-axis. An example of the probability mass function is displayed below.

The probability mass function for a single die roll.

The probability mass function for a single die roll.

The expected value of a random variable is defined as $$ \text{E}[X] = \sum_{k=1}^{N} p_k \cdot X_k. $$ $\text{E}[X]$ can be read as "expectation of $X$". The expected value is equivalent to the mean of the random variable. "Expected value of $X$", "mean of $X$", and "expectation of $X$" are used interchangeably.

The expected value of a die roll is $$ \begin{align*} \text{E}[X] = \sum_{k=1}^{N} p_k \cdot X_k \\ = \sum_{k=1}^{6} \frac{1}{6} \cdot k\\ = \frac{1}{6} \sum_{k=1}^{6} k \\ = \frac{21}{6} \\ = 3.5. \\ \end{align*} $$

An important property of expected value is linearity. Linearity of expectation means that $$\text{E}[X+Y] = \text{E}[X] + \text{E}[Y]$$ for two random variables X and Y. This property will often simplify calculations significantly, so it is definitely worth remembering.

On first glance, this could be a tedious question where we calculate a sum of 36 terms, corresponding to every outcome of two dice rolls. Fortunately, we can use the linearity of expectation to simplify things. We know from the previous question that $E[X] = 3.5$. Then $$ \begin{align*} \text{E}[2X] = 2\text{E}[X] \\ = 2 \times 3.5 \\ = 7. \end{align*} $$

The variance of a random variable is a quantitative measure of how much a random variable tends to deviate from its expected value. Random variables with higher variance are more spread out from their expected value. The formal definition is $$ \text{Var}(X) = \sum_{k=1}^{N} p_k \cdot (X_k - \text{E}[X])^2. $$ We can see that the definition of variance is actually an expectation: the expectation of $(X - E[X])^2$. As a result, it is often written as $$ \text{Var}(X) = \text{E} \hspace{-1mm} \left[ (X - \text{E}[X])^2 \right]. $$

We use our previous result for the expected value of a die roll: $\text{E}[X] = 3.5$. $$ \begin{align*} \text{Var}(X) = \sum_{k=1}^{N} p_k \cdot (X_k - \text{E}[X])^2\\ = \sum_{i=1}^{6} \frac{1}{6} \cdot (k - 3.5)^2\\ = \frac{35}{12}. \end{align*} $$

The standard deviation, $\sigma$, is defined as the square root of the variance. $$\text{Var}(X) = \sigma^2$$ Why would we need to define such a quantity? Well, standard deviation has an attractive property: it is expressed in the same units as the mean. As a result, it's the most commonly used measure of the deviation of a random variable. The final two properties we will cover in this section are covariance and correlation. The covariance measures the joint variability of several random variables. The covariance between two random variables $X$ and $Y$ is defined as $$ \text{Cov}(X,Y) = \text{E}\left[ \left( X - \text{E}[X] \right) \left( Y - \text{E}[Y] \right) \right] $$ The correlation coefficient, $\rho$ of two random variables $X$ and $Y$ is the covariance normalised by the product of the standard deviations $X$ and $Y$, $\sigma_X$ and $\sigma_Y$ respectively. $$ \rho_{XY} = \frac{\text{Cov}(X,Y)}{\sigma_X \sigma_Y}. $$ To finish, let's look at an example. The table below contains five joint measurements of $X$ and $Y$.

$X$ $Y$
2 3
2 2
3 4
3 5
5 6

Let us first calculate the respective expectations of $X$ and $Y$. $$ \text{E}[X] = \frac{1}{5} \left( 2 + 2 + 3 + 3 + 5 \right) = 3 $$ $$ \text{E}[Y] = \frac{1}{5} \left( 3 + 2 + 4 + 5 + 6 \right) = 4 $$ Plugging in the sample values into $$ \text{Cov}(X,Y) = \text{E}\left[ \left( X - \text{E}[X] \right) \left( Y - \text{E}[Y] \right) \right] $$ gives us $$ \text{Cov}(X,Y) = \frac{1}{5} \left( (-1)(-1) + (-1)(-2) + 0 + 0 + (2)(2) \right) = \frac{7}{5}. $$

Given that the definition of the correlation coefficient is $$ \rho_{XY} = \frac{\text{Cov}(X,Y)}{\sigma_X \sigma_Y} $$ and we have already calculated $\text{Cov}(X,Y)$, we only need to calculate $\sigma_X$ and $\sigma_Y$. $$ \text{Var}(X) = \frac{1}{5} \left( (-1)^2 + (-1)^2 + 0 + 0 + 2^2 \right) = \frac{6}{5} $$ $$ \sigma_X = \sqrt{\frac{6}{5}} $$ $$ \text{Var}(Y) = \frac{1}{5} \left( (-1)^2 + (-2)^2 + 0 + 1^2 + 2^2 \right) = 2 $$ $$ \sigma_Y = \sqrt{2} $$ Therefore, $$ \rho_{XY} = \frac{7}{5} \cdot \sqrt{\frac{5}{12}} \approx 90\%. $$
Example: the Poisson distribution

The Poisson distribution describes the probability of a number of events occurring in a given time interval, if the events are independently occurring with a mean rate $\lambda$. The Poisson distribution is often used in finance to model events, such as incoming updates from an exchange. The probability of $k$ events occurring in a time interval is given by $$ f(k; \lambda) = P(X=k) = \frac{\lambda^k e^{-\lambda}}{k!}. $$ Since there cannot be a fractional number of events occurring in an interval, the Poisson distribution is a discrete distribution. The probability mass function of the Poisson distribution is shown in the image below.

The probability mass function of the Poisson distribution, with mean event rate 5 (top) and 10 (bottom).

The probability mass function of the Poisson distribution

The expected value of the Poisson distribution is $$ E[X] = \lambda $$ and the variance is $$ \text{Var}(X) = \lambda. $$ Once again, it is worth proving these relationships for yourself. You will need the definitions of expected value and variance for discrete random variables in the previous section.

Exercises

There are four possible outcomes, all occurring with equal probability. Let H be the outcome of a head, and T be the outcome of a tail. The probabilities are $$ P(HH) = \frac{1}{4}, $$ $$ P(HT) = \frac{1}{4}, $$ $$ P(TH) = \frac{1}{4}, $$ $$ P(TT) = \frac{1}{4}. $$

The set of possible outcomes is the set of positive integers $$ S = \{1,2,3,4,...,\infty\}. $$ Let $n$ be the number of coin tosses until a head is tossed. The probability of $n=1$ occurring is the probability that we throw a head on the first toss $P(H) = \frac{1}{2}$. The probability of $n=2$ occurring is the probability of tossing the sequence $TH$ and is equal to $P(TH) = \frac{1}{4}$. Generally, the probability of $n=k$ is equal to the probability that we toss $(k-1)$ $T$s and then a $H$ on the $k$th toss, and since each throw occurs with probability $\frac{1}{2}$, we get $$ P(n=k) = \frac{1}{2^k}. $$

The expected value of $Y$ is given by $$ \text{E}[Y] = \sum_{k=1}^{\infty} k \times P(n=k). $$ Substituting our previous result for $P(n=k)$, we get $$ \text{E}[Y] = \sum_{k=1}^{\infty} \frac{k}{2^k}. $$ To solve this sum, recall the definition of a geometric series $$ \sum_{k=0}^{\infty} x^k = \frac{1}{1-x}. $$ Since $x^0 = 1$, let us define $$ f(x) = \sum_{k=1}^{\infty} x^k = \frac{1}{1-x} - 1. $$ We want to calculate $$ xf'(x) = \sum_{k=1}^{\infty} k x^k. $$ Evaluating this and substituting $x=\frac{1}{2}$ gives us the result $$ xf'(x) = \frac{x}{(1-x)^2} = 2. $$