📊 Odds, Log-Odds, and Odds Ratio

Part I: Foundations

1. Probability vs Odds

Key Question

Why do statisticians use "odds" when probability seems simpler and more intuitive?

This section answers that question by building from basic definitions to the motivation for using odds.

1.1 What is Probability?

Definition: Probability measures the likelihood of an event occurring as a fraction of all possible outcomes.

P (event) = \frac{Number of favorable outcomes}{Total number of all possible outcomes}

Properties:

Range: $0 \leq P \leq 1$ (or 0% to 100%)
$P (event) + P (not event) = 1$
Always expressed as a fraction of the total

Example: Rolling a die

$P (rolling a 6) = \frac{1}{6} \approx 0.167$ or 16.7%
$P (not rolling a 6) = \frac{5}{6} \approx 0.833$ or 83.3%

1.2 What are Odds?

Definition: Odds compare the number of times an event occurs to the number of times it does NOT occur.

Odds (event) = \frac{Number of times event occurs}{Number of times event does NOT occur}

Properties:

Range: $0 \leq Odds \leq \infty$ (unbounded on the upper end)
Expressed as a ratio (e.g., "5 to 3" or "5:3")
Can be greater than 1 (unlike probability)

Example: Sports Betting

"The odds in favor of my team winning are 5 to 3"

This means:

Out of 8 similar scenarios, team wins 5 times and loses 3 times
${Odds}_{win} = \frac{5}{3} = 1.67$
$P (win) = \frac{5}{8} = 0.625$ (62.5%)
$P (loss) = \frac{3}{8} = 0.375$ (37.5%)

1.3 Key Difference: Probability vs Odds

Aspect	Probability	Odds
Definition	Favorable / Total	Favorable / Unfavorable
Range	$[0, 1]$	$[0, \infty)$
Example (Win 5, Lose 3)	$\frac{5}{8} = 0.625$	$\frac{5}{3} = 1.67$
Interpretation	"62.5% chance of winning"	"5 to 3 odds in favor"
Use Case	General understanding	Betting, logistic regression

Critical Distinction

🚫 Odds ≠ Probability

Probability = $\frac{count of event}{count of ALL events}$ (denominator includes event)
Odds = $\frac{count of event}{count of NOT event}$ (denominator excludes event)

1.4 Converting Between Probability and Odds

From Probability to Odds:

Odds = \frac{p}{1 - p}

where $p$ is the probability.

Example: Sports Betting (Contd)

As seen above: $P (win) = \frac{5}{8}; P (loss) = \frac{3}{8}; {Odds}_{win} = \frac{5}{3}$
Probability of loss is overall probability minus probability of winning.
- $P (loss) = 1 - P (win) = 1 - \frac{5}{8} = \frac{3}{8}$
$\frac{p}{1 - p} = \frac{P (w i n)}{P (l o s s)} = \frac{\frac{5}{8}}{\frac{3}{8}} = \frac{5}{3} = O d d s_{w i n}$

This proves ➛ $Odds = \frac{p}{1 - p}$

From Odds to Probability:

p = \frac{Odds}{1 + Odds}

Example Conversions:

Probability $p$	Odds Calculation	Odds Value	Interpretation
0.5	$\frac{0.5}{1 - 0.5} = \frac{0.5}{0.5}$	1	Even odds (1:1)
0.75	$\frac{0.75}{1 - 0.75} = \frac{0.75}{0.25}$	3	3 to 1 in favor
0.25	$\frac{0.25}{1 - 0.25} = \frac{0.25}{0.75}$	0.33	1 to 3 against
0.8	$\frac{0.8}{1 - 0.8} = \frac{0.8}{0.2}$	4	4 to 1 in favor
0.2	$\frac{0.2}{1 - 0.2} = \frac{0.2}{0.8}$	0.25	1 to 4 against

Verification Example:

Given: $P (win) = \frac{5}{8} = 0.625$
Calculate: $Odds = \frac{0.625}{1 - 0.625} = \frac{0.625}{0.375} = 1.67 = \frac{5}{3}$ ✓

1.5 Why Use Odds Instead of Probability?

Problem with Probability: Probabilities are bounded between 0 and 1, which creates asymmetry:

$P = 0.9$ (strong favor) is only 0.4 units from $P = 0.5$ (neutral)
$P = 0.1$ (strong against) is also 0.4 units from $P = 0.5$ (neutral)
But 90% vs 10% feels much more extreme than the numbers suggest

Advantage of Odds: Odds are symmetric around 1:

Odds = 9 (9:1 in favor) means 9 times more likely to occur
Odds = 1/9 = 0.11 (1:9 against) means 9 times more likely NOT to occur
These are reciprocals.

However, Odds Still Have Asymmetry:

Odds in favor: 1 to 6 → Odds = $\frac{1}{6} = 0.17$
Odds in favor: 6 to 1 → Odds = $\frac{6}{1} = 6$
Distance from neutral (1): $| 0.17 - 1 | = 0.83$ vs $| 6 - 1 | = 5$ ❌ Not symmetric!

Solution: Use Log-Odds to achieve perfect symmetry! � See Section 2

2. Log-Odds (Logit Function)

2.1 Definition

Log-Odds (also called logit) is the natural logarithm of the odds.

Log-Odds = \log (Odds) = \log (\frac{p}{1 - p})

This is called the logit function: $logit (p) = \log (\frac{p}{1 - p})$

2.2 Why Log-Odds? The Symmetry Solution

Taking the logarithm of odds creates perfect symmetry:

Example:

Odds against (1 to 6): Odds = $\frac{1}{6} = 0.17$
- $\log (Odds) = \log (0.17) = - 1.79$
Odds in favor (6 to 1): Odds = $\frac{6}{1} = 6$
- $\log (Odds) = \log (6) = 1.79$

Key Insight: The distance from the origin (0) is now exactly the same (1.79) for both scenarios! This symmetry is crucial for statistical modeling.

2.3 Properties of Log-Odds

Property	Description
Range	$(- \infty, + \infty)$ (unbounded in both directions)
Neutral Point	0 (when Odds = 1, meaning $p = 0.5$ )
Symmetry	$\log ({Odds}_{1}) = - \log ({Odds}_{2})$ when odds are reciprocals
Additivity	Log-odds differences correspond to odds ratios
Linearity	Perfect for linear modeling (logistic regression)

Explain "Symmetry" of Log-Odds

Property: Log-odds differences correspond to multiplication (odds ratios).

This property means that switching your perspective from "winning" to "losing" changes the sign of the log-odds, but the absolute value stays exactly the same.

Team wins 5 times and loses 3 times
- ${Log-odds}_{w i n} = l o g (\frac{5}{3}) \approx 0.221$
- ${Log-odds}_{l o s s} = l o g (\frac{3}{5}) \approx - 0.221$

Explain "Additivity" of Log-Odds

Property: Log-odds differences correspond to multiplication (odds ratios).

Logarithms turn multiplication into addition, and division into subtraction. In statistics, when you want to compare the odds of two different groups, you look at the Odds Ratio. Because of additivity, subtracting two log-odds values is mathematically identical to taking the logarithm of their odds ratio.

Let's introduce a second team to see this in action:

Your original Team (Team A): Wins 5, Loses 3.

\begin{aligned} {Odds}_{A} & = \frac{5}{3} \\ {Log-Odds}_{A} & = \log (\frac{5}{3}) = 0.2218 \end{aligned}

A rival Team (Team B): Wins 7, Loses 1.

\begin{aligned} {Odds}_{B} & = \frac{7}{1} \\ {Log-Odds}_{B} & = \log (7) = 0.8450 \end{aligned}

If we want to know how much better Team B is compared to Team A, we can do it two ways:

Using Odds (Division): Find the Odds Ratio.

Odds Ratio = \frac{O d d s_{B}}{O d d s_{A}} = \frac{7}{\frac{5}{3}} = 4.2

Team B's odds are $4.2$ times higher than Team A's odds.

Using Log-odds (Subtraction): Subtract Team A's log-odds from Team B's log-odds.

\begin{aligned} Log(Odds Ratio) & = L o g (\frac{O d d s_{B}}{O d d s_{A}}) = \underset{Log’s Quotient Rule}{\underset{⏟}{{Log-Odds}_{B} - {Log-Odds}_{A}}} \\ = 0.8450 - 0.2218 = 0.6232 \\ ∴ Difference & = 0.6232 \end{aligned}

To see the additivity connection, if you take the logarithm of the Odds Ratio (4.2), you get the exact same number: $$log(4.2)=0.6232$$Why this matters:
In logistic regression models, this property allows us to change an outcome by simply adding coefficients (e.g., $0.6232$ to the log-odds), which represents multiplying the real-world odds of the event happening (e.g., multiplying the odds by $4.2$ ).

2.4 Comprehensive Conversion Table

Probability $p$	Odds	Log-Odds (logit)	Interpretation
0.01	0.0101	-4.60	Extremely unlikely
0.1	0.111	-2.20	Very unlikely
0.25	0.333	-1.10	Unlikely
0.5	1.0	0	Neutral (50-50)
0.75	3.0	1.10	Likely
0.9	9.0	2.20	Very likely
0.99	99.0	4.60	Extremely likely

Visual Pattern: Notice how log-odds are symmetric around 0:

$p = 0.25$ → logit = -1.10
$p = 0.75$ → logit = +1.10

2.5 Calculating Log-Odds (Example)

Given: Team wins 5 times, loses 3 times (from earlier example)

Step 1: Calculate probability

p = \frac{5}{5 + 3} = \frac{5}{8} = 0.625

Step 2: Calculate odds

Odds = \frac{5}{3} = 1.67

Or alternatively:

Odds = \frac{p}{1 - p} = \frac{0.625}{1 - 0.625} = \frac{0.625}{0.375} = 1.67

Step 3: Calculate log-odds

Log-Odds = \log (1.67) = 0.51

Interpretation:

A log-odds of 0.51 indicates the event is more likely to occur than not (positive value)
The event is $e^{0.51} = 1.67$ times more likely to occur than not occur

2.6 The Logit Function in Context

Mathematical Overview of Logit functions

Input range: $(0, 1)$ representing standard probabilities.
Output range: $(- \infty, \infty)$ , spanning all real numbers.
Symmetry: It passes exactly through $(0.5, 0)$ where the odds are even $(1, 1)$ .
Asymptotes: As $p$ approaches $0$ , the logit approaches $- \infty$ as $p$ approaches $1$ , it approaches $\infty$ .

The logit function is the foundation of logistic regression:

logit (p) = \log (\frac{p}{1 - p}) = β_{0} + β_{1} x_{1} + β_{2} x_{2} + \dots + β_{n} x_{n}

Why this matters:

Left side: Log-odds (unbounded: $- \infty$ to $+ \infty$ )
Right side: Linear combination (also unbounded)
This allows us to model probabilities using linear equations!

Inverse (Sigmoid Function):

p = \frac{1}{1 + e^{- (β_{0} + β_{1} x_{1} + \dots)}} = \frac{e^{log-odds}}{1 + e^{log-odds}}

This is why the sigmoid function appears in logistic regression!

3. Odds Ratio

Transition to Comparisons

Now that we understand what odds are and why log-odds are useful, we need a way to compare odds between two groups. This is where the odds ratio comes in.

3.1 Definition

Odds Ratio (OR) compares the odds of an event occurring in two different groups.

Odds Ratio = \frac{Odds in Group 1}{Odds in Group 2}

Purpose: Measure the strength of association between an exposure (e.g., treatment, risk factor) and an outcome (e.g., disease, success).

3.2 Interpreting Odds Ratios

Odds Ratio	Interpretation
OR = 1	No association (exposure doesn't affect outcome)
OR > 1	Positive association (exposure increases odds of outcome)
OR < 1	Negative association (exposure decreases odds of outcome)
OR = 2	Exposure doubles the odds of outcome
OR = 0.5	Exposure halves the odds of outcome
OR = 3	Exposure triples the odds of outcome

3.3 Example: Smoking and Lung Cancer

Scenario: Study of 1000 people

	Lung Cancer	No Lung Cancer	Total
Smokers	80	120	200
Non-smokers	20	780	800
Total	100	900	1000

Step 1: Calculate odds for each group

Odds (Smokers): (Odds of person having Cancer, Given he is Smoker) ${Odds}_{smokers} = \frac{Cancer}{No Cancer} = \frac{80}{120} = 0.667$
Odds (Non-smokers): (Odds of person having Cancer, Given he is Non-Smoker) ${Odds}_{non-smokers} = \frac{20}{780} = 0.0256$

Step 2: Calculate Odds Ratio

OR = \frac{{Odds}_{smokers}}{{Odds}_{non-smokers}} = \frac{0.667}{0.0256} = 26.0

Interpretation: Smokers have 26 times higher odds of developing lung cancer compared to non-smokers.

3.4 Odds Ratio in Case-Control Studies

Odds ratios are particularly useful in retrospective studies (case-control studies) where you:

Start with cases (people with disease) and controls (people without)
Look backwards to see exposure rates

Why not use Relative Risk?

In case-control studies, you can't calculate disease incidence (you artificially selected cases)
Odds ratio approximates relative risk when disease is rare
Odds ratio is always calculable from 2×2 tables

4. Log-Odds Ratio

Final Transformation

Just as we transformed odds to log-odds for better mathematical properties, we do the same for odds ratios.

4.1 Definition

Log-Odds Ratio (also called log OR) is the natural logarithm of the odds ratio.

Log-Odds Ratio = \log (OR) = \log (\frac{{Odds}_{1}}{{Odds}_{2}})

Equivalently:

Log-OR = \log ({Odds}_{1}) - \log ({Odds}_{2}) = {Log-Odds}_{1} - {Log-Odds}_{2}

4.2 Why Use Log-Odds Ratio?

Advantages over regular Odds Ratio

Symmetry:
- OR = 2 means doubling risk (Case: Odds of 2 wins and 1 loss)
- OR = 0.5 means halving risk (Case: Odds of 1 wins and 2 loss)
- But 2 and 0.5 are NOT symmetric around 1
- $\log (2) = 0.69$ and $\log (0.5) = - 0.69$ ARE symmetric around 0!
Additivity:
- Combining multiple effects: add log-odds ratios
- Example: Effect A (log-OR = 0.5) + Effect B (log-OR = 0.3) = Combined (log-OR = 0.8)
Statistical Properties:
- Log-OR is approximately normally distributed (good for statistical tests)
- Confidence intervals are symmetric on log scale
- Easier to work with in regression models
Effect Size Interpretation:
- Positive log-OR: increased odds
- Negative log-OR: decreased odds
- Zero log-OR: no effect

4.3 Interpreting Log-Odds Ratios

OR	Log-OR	Interpretation
0	$- \infty$	Impossible in exposed group
0.14	-2.0	86% reduction in odds
0.37	-1.0	63% reduction in odds
0.5	-0.69	Halves the odds
1	0	No effect
2	0.69	Doubles the odds
2.72	1.0	172% increase in odds
7.39	2.0	639% increase in odds
$\infty$	$+ \infty$	Certain in exposed group

4.4 Example Calculation

Using our smoking example where OR = 26:

Log-OR = \log (26) = 3.26

Interpretation:

A log-odds ratio of 3.26 indicates a very strong positive association
The effect is $e^{3.26} = 26$ times
This is 3.26 standard deviations away from no effect (log-OR = 0)

Summary

Key Takeaways

Probability vs Odds: Different ways to express likelihood
- Probability: fraction of total
- Odds: ratio of favorable to unfavorable
Log-Odds (Logit): Transforms [0,1] to (-∞,+∞)
- Creates symmetry
- Foundation of logistic regression
- Makes effects additive
Odds Ratio: Measures association strength
- OR = 1: No association
- OR > 1: Positive association
- OR < 1: Negative association