Discrete Probability Distributions

Recap: Probability Distribution

A probability distribution is a function or rule that describes how probability is spread across the possible values of a random variable.

A discrete probability distribution is a probability distribution for a discrete random variable, meaning the variable can take only a countable set of distinct values

Probability Mass Function (PMF)

This is the mathematical tool or function used to define that distribution.

Definition:

A Probability Mass Function (PMF) is a probability distribution function or rule that defines the probability of occurrence of each value of discrete random variable in sample space ( $Ω$ )

Mathematically, it gives the probability that a discrete random variable ( $X$ ) is exactly equal a particular value $P (X = x)$ . It is frequently denoted by $f (x)$

Properties:

Non-negativity: $0 \leq P (X = k) \leq 1$ for all possible values of $k$ in the state space.
Normalization: The sum of the probabilities for all possible values must equal 1:

\sum_{a l l k} P (X = k) = 1

Support: $P (X = k) > 0$ only for $k$ in the set of possible values (the support) of $X$ .

Example: Rolling Two Fair Die

Experiment: Roll two fair six-sided dice.
Sample Space: $Ω = {(d_{1}, d_{2}) : d_{1}, d_{2} \in {1, \dots, 6}}$
- Number of outcomes: $| Ω | = 6 \times 6 = 36$
Random Variable ( $X$ ): Sum of the two dice, $X = d_{1} + d_{2}$
- Possible values: $X \in {2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12}$
Let $N (x)$ = Number of pairs $(d_{1}, d_{2})$ such that $d_{1} + d_{2} = x$
- e.g., $N (6) = 5$ (5 pairs: (1,5), (2,4), (3,3), (4,2), (5,1))
PMF:

P (X = x) = \frac{N (x)}{36}

e.g., $P (X = 6) = \frac{5}{36}$

★ Common Discrete Distributions

1. The Bernoulli Distribution

The Bernoulli distribution is the simplest of all. It models an experiment with only two possible outcomes, which we typically label as "success" and "failure."

The Experiment: A single trial.
The Outcomes:
- Success (e.g., getting heads, finding a defective item, a customer clicking an ad).
- Failure (e.g., getting tails, finding a good item, a customer not clicking).
The Random Variable:
- $X = 1$ for a success.
- $X = 0$ for a failure.
The Parameter:
- $p$ : the probability of success.
The PMF:

P (X = x) = {\begin{matrix} p & when x = 1 \\ 1 - p & when x = 0 \end{matrix}

Example:
Probability of getting the Job offer. Let's say the probability of "yes" is $p = 0.7$ .

$P (Job offer) = P (X = 1) = 0.7$ .
$P (Rejection) = P (X = 0) = 1 - 0.7 = 0.3$ .

2. The Binomial Distribution

What happens if we repeat a Bernoulli trial multiple times? That's where the Binomial distribution comes in. It models the number of successes in a fixed number of independent Bernoulli trials.

The Experiment: Performing $n$ independent Bernoulli trials.
The Random Variable ( $X$ ): The total number of successes in the $n$ trials.
The Parameters:
- $n$ : the total number of trials.
- $p$ : the probability of success on any single trial.
The PMF: The probability of getting exactly $k$ successes in $n$ trials is:$$P(X=k) = \binom{n}{k} p^k (1-p)^{n-k}$$
Where $(\binom{n}{k})$ is the combination "n choose k," which counts the number of ways to arrange the $k$ successes among the $n$ trials.

Example:
You flip a fair coin ( $p = 0.5$ ) 5 times ( $n = 5$ ). What is the probability of getting exactly 3 heads ( $k = 3$ )?

$P (X = 3) = (\binom{5}{3}) (0.5)^{3} (1 - 0.5)^{5 - 3}$
$P (X = 3) = 10 \times (0.125) \times (0.25) = 0.3125$
So, there is a 31.25% chance of getting exactly 3 heads in 5 flips.

3. The Poisson Distribution

The Poisson distribution models the number of times an event occurs over a fixed interval of time or space, when we know the average rate at which the event occurs.

The Experiment: Counting the number of occurrences of an event in a fixed interval.
The Random Variable ( $X$ ): The number of events in that interval.
The Parameter:
- $λ$ (lambda): the average number of events per interval.
The PMF: The probability of observing exactly $k$ events is: $P (X = k) = \frac{λ^{k} e^{- λ}}{k!}$ Where $e$ is the base of the natural logarithm ( $e \approx 2.71828$ ).

Example:
A call center receives an average of 10 calls per hour ( $λ = 10$ ). What is the probability that they receive exactly 7 calls in a given hour ( $k = 7$ )?

$P (X = 7) = \frac{10^{7} e^{- 10}}{7!}$
$P (X = 7) = \frac{10, 000, 000 \times 0.0000454}{5040} \approx 0.09$
So, there is about a 9% chance of receiving exactly 7 calls in that hour.

4. The Discrete Uniform Distribution

Discrete Uniform Distribution is a Probability Distribution that describes the likelihood of outcomes when each outcome in a finite set is equally likely.
It's characterized by a constant probability mass function (PMF) over a finite range of values.

P (X = x_{i} ​) = \frac{1}{n} \dots for i=1,2,…,n

Example: Rolling a Fair Die

When rolling a fair six-sided die, each face (1, 2, 3, 4, 5, 6) has an equal probability of $\frac{1}{6}$ of landing face up.