9 Continuous Random Variables

A continuous random variable takes values on an interval of the real line \(R\).

Examples of continuous random variables include:

The height \(H\) of a student: in cm, \(0\le H \le 300\).
The temperature \(T\) of a room: in Celsius, \(-273.15 \le T \le \infty\).
The time \(T\) it takes for a light bulb to burn out: in seconds, \(0 \le T \le \infty\).

9.1 Cumulative Distribution Function

The distribution function, also known as the cumulative distribution function (CDF), of a random variable \(X\) provides a comprehensive description of its probability distribution. It is defined as follows:

The distribution function \(F\) of the random variable \(X\) is the function

\[F(x) = P(X\le x)= P(\omega : X(\omega) \le x), \qquad x \in R\]

In simpler terms, the CDF, denoted as \(F(x)\) or \(F_X(x)\) to emphasize the random variable, gives the probability that the random variable \(X\) takes on a value less than or equal to a given value \(x\).

9.1.1 Examples:

Uniform Distribution, \(U(a,b)\)

\[ F(x) = \begin{cases} 0 & \; & x\le a \\ \frac{x-a}{b-a} & \; & a<x<b \\ 1 & \; & b\le x \end{cases} \]

Exponential Distribution, \(Exp(\lambda)\)

\[ F(x) = \begin{cases} 0 & \; & x \le 0 \\ \\ 1- e^{-\lambda x} & \; & 0 < x \; \lambda > 0 \end{cases} \]

Normal Distribution, \(N(\mu,\sigma)\)

\[ F(x) = \frac{1}{\sqrt{2\pi\sigma^2}}\int_{-\infty}^{x} e^{\frac{(u-\mu)^2}{2\sigma^2}}du \] We do not know how to solve that integral using simple functions (polynomials, trig function, exp, logs, algebraic functions, etc.), so we have to keep as it is. Maybe this is a suggestion to write all \(F(x)\) as integrals, and use the integrated function (\(\frac{1}{\sqrt{2\pi}}e^{\frac{u^2}{2}}\)) instead?

Binomial distribution, Binomial\((n,p)\)

9.1.2 Properties of the Cumulative Distribution Function

The CDF possesses several key properties:

Boundedness: \(0 \le F(x) \le 1\) for all \(x\).
Non-decreasing: \(P(x < X \le y) = F(y) - F(x) \ge 0\), for all \(x \le y\).
- Graphical Example for X with Uniform(0, 1)

\[ P(.6 < X < .8) = F(.8) - F(.6) = .8 - .6 = .2 \]

Right-continuous: If \(h>0\), then \[\lim_{h \to 0} F(x+h) = F(x).\]
Limits at Infinity: \[\lim_{x \to \infty} F(x) = 1 \text{ and } \lim_{x \to -\infty} F(x) = 0.\]

Any function satisfying these properties can be the distribution function of some random variable.

9.1.3 Probability of a Single Point

An important consequence of the continuity of the CDF is the following result:

If \(X\) has a continuous distribution function \(F\), then \[P(X = x) = 0 \qquad \text{for all} \quad x\].

Check:

We can express the probability of \(X\) taking a specific value \(x\) as a limit of probabilities over shrinking intervals:

\[P(X = x) = \lim_{n \to \infty} P \left( x - \frac{1}{n} < X \le x \right)\]

Using the properties of the CDF from Theorem (6), this can be rewritten as:

\[= \lim_{n \to \infty} \left( F(x) - F \left( x - \frac{1}{n} \right) \right)\]

Since \(F\) is continuous, the limit as \(n\) approaches infinity becomes:

\[ = F(x) - F(x) = 0.\]

This result highlights a crucial distinction between discrete and continuous random variables. While for discrete variables the probability of individual points can be non-zero, for continuous variables, it is always zero.

9.1.4 Mystery CDF

Questions:

What can we learn from the CDF?
What is going on in the regions that are flatter?
What is going on in the regions that are steeper?
Where is the median?
Where is the 25th percentile? 75th percentile?

Answers:

We can identify quantiles and outcomes.
Values in flatter intervals are less likely to occur than values in steeper intervals. The derivative (a measure of flatness/steepness) of the CDF (if it exists) is the relative likelihood, also called the probability density function.
See previous answer.
To find the median, draw the horizonal line \(y = 0.5\). The x value where that line intersects the CDF is the median.
Use the same procedure as the previous answer with \(y=0.25\) and \(y=0.75\).

9.2 Density Function

The density function provides a more intuitive way to understand the distribution of a continuous random variable. It is linked to the CDF through differentiation:

Definition: Let \(X\) have distribution function \(F\). If the derivative

\[\frac{dF}{dx} = F'(x)\]

exists at all but a finite number of points, and the function \(f\) defined by

\[f(x) = \begin{cases} F'(x), & \text{where } F'(x) \text{ exists} \\ 0, & \text{elsewhere} \end{cases}\]

satisfies

\[F(x) = \int_{-\infty}^x f(v) dv,\]

then \(X\) is said to be a continuous random variable with density \(f(x)\).

\[ F(1) = \int_{-\infty}^1 f(x) dx \] Similarly, we can write \[ P(0<X<2) = \int_{0}^2 f(x) dx = \int_{-\infty}^2 f(x) dx - \int_{-\infty}^1 f(x) dx \]

The density function, \(f(x)\), represents the probability density at a given point \(x\). It’s important to remember that \(f(x)\) itself is not a probability but a measure of how dense the probability is around \(x\).

9.2.1 Key Rule for Probabilities with Density Functions

If \(X\) has density \(f(x)\), then for a set \(C \subseteq R\), the probability of \(X\) falling within \(C\) is obtained by integrating the density function over that set:

\[P(X \in C) = \int_C f(x) dx.\]

9.3 Distributions

9.3.1 Uniform

\[ F(x) = \begin{cases} 0 & \; & x< a \\ \frac{x-a}{b-a} & \; & a\le x \le b \\ 1 & \; & b < x \end{cases} \qquad \qquad f(x) = \begin{cases} 0 & x < a\\ \frac{1}{b-a}, & a \le x \le b \\ 0, & b < x \end{cases} \]

Two parameters:
- \(a\), left end point
- \(b\), right end point
All values between \(a\) and \(b\) are equally likely to occur.
Applications
- Generation of random variables from other distributions. (How might one generate a Bernoulli random variable from a uniform random variable?)
- Maximum uncertainty models

9.3.2 Exponential

The exponential random variable is frequently used to model the time until an event occurs, such as the time until a machine breaks down or the time between customer arrivals. Its density function is defined as:

\[ F(x) = \begin{cases} 0 & \; & x < 0 \\ \\ 1- e^{-\lambda x} & \; & 0 \le x \end{cases} \qquad \qquad f(x) = \begin{cases} 0, & x < 0\\ \\ \lambda e^{-\lambda x} & 0 \le x \end{cases} \]

where \(\lambda > 0\) is the rate parameter.

9.3.3 Normal

The normal random variable, also known as the Gaussian random variable, is ubiquitous in statistics due to its central role in the central limit theorem. Its density function has the characteristic bell shape

\[ F(x) = \frac{1}{\sqrt{2\pi \sigma^2}} \int_{-\infty}^x \exp \left( -\frac{(t - \mu)^2}{2\sigma^2} \right) dt, \qquad \qquad f(x) = \frac{1}{\sqrt{2\pi \sigma^2}} \exp \left( -\frac{(x - \mu)^2}{2\sigma^2} \right) \]

where \(\mu\) is the mean and \(\sigma^2\) is the variance.

9.3.4 Gamma

The gamma random variable is a generalization of the exponential random variable and can model a wider range of phenomena. Its density function is given by:

\[f(x) = \frac{c \lambda^\alpha x^{\alpha - 1} e^{-\lambda x}}{\Gamma(\alpha)},\]

where \(\alpha > 0\) is the shape parameter, \(\lambda > 0\) is the rate parameter, and \(\Gamma(\alpha)\) is the gamma function.

9.3.5 Beta

The beta random variable can model proportions and percentages, for instance the proportion of students left-handed at UVA.

\[ \frac{\Gamma(\alpha + \beta)}{\Gamma(\alpha)\Gamma(\beta)} t^{\alpha-1}(1-t)^{\beta-1} \] where \(0 < t < 1\), and \(\alpha\) and \(\beta\) are parameters of the distribution.

[With \(\alpha=\beta=1\) we get the Uniform distribution U(0,1)]

9.3.6 Mixture distributions

9.4 Expectation and Variance

For continuous random variables, the expected value, also known as the mean, is calculated using integration instead of summation, which is used for discrete random variables.

\[E[X] = \int_{-\infty}^{\infty} x f_X(x) dx \]

where: \(f_X(x)\) is the probability density function (PDF) of the random variable X.

This formula essentially weighs each possible value x of the random variable by its corresponding probability density \(f_X(x)\), and then integrates over the entire support of the distribution to arrive at the expected value.

For a function g of the random variable X, we get a new random variable \(Y = g(X)\). The expectation of \(Y\) can be calculated as:

\[E[Y] = E[g(X)] = \int_{-\infty}^{\infty} g(x) f_X(x) dx\]

This formula allows for calculating the expected value of any function of the random variable, such as \(X^2\) or \(e^X\), by simply substituting the function g into the formula.

Similarly, the variance of a continuous random variable is also defined using integration.

\[Var[X] = E[(X-\mu)^2] = \int_{-\infty}^{\infty} (x-\mu)^2 f_X(x) dx\]

where \(\mu = E[X]\) is the expected value of X.

This formula measures the average squared deviation of the random variable X from its mean. It provides a measure of the spread or dispersion of the probability distribution.

An alternative way to express the variance:

\[Var[X] = E[X^2] - \mu^2\]

9.5 Joint Distribution of continuous random variables

9.5.1 Motivating example: Bivariate Normal

9.5.2 Univariate vs Bivariate

Univariate	Bivariate
\(F_X(x) = P(X\leq x)\)	\(F_{X,Y}(x,y) = P(X\leq x\ \text{and}\ Y\leq y)\)
\(f_X(x) = \frac{d}{dx} F_X(x)\)	\(f_{X,Y}(x,y) = \frac{\partial^2}{\partial x \partial y} F_{X,Y}(x,y)\)