Introduction to Bayesian Statistics

Murray Logan

17 February 2025

Frequentist vs Bayesian

Frequentist

P(D|H)

long-run frequency

relatively simple analytical methods to solve roots

conclusions pertain to data, not parameters or hypotheses

compared to theoretical distribution when NULL hypothesis is true

probability of obtaining observed data or MORE EXTREME data

Frequentist

P-value
- probabulity of rejecting NULL
- NOT a measure of the magnitude of an effect or degree of significance!
- measure of whether the sample size is large enough
95% CI
- NOT about the parameter it is about the interval
- does not tell you the range of values likely to contain the true mean

Frequentist vs Bayesian

	Frequentist	Bayesian
Probability	Long-run frequency \(P(D\|H)\)	Degree of belief \(P(H\|D)\)
Parameters	Fixed, true	Random, distribution
Obs. data	One possible	Fixed, true
Inferences	Data	Parameters

Frequentist vs Bayesian

n: 10
Slope: -0.1022
t: -2.3252
p: 0.0485

n: 10
Slope: -10.2318
t: -2.2115
p: 0.0579

n: 100
Slope: -10.4713
t: -6.6457
p: 1.7101362^{-9}

Frequentist vs Bayesian

	Population A	Population B
Percentage change	0.46	45.46
Prob. >5% decline	0	0.86

Bayesian Statistics

Bayesian

Bayes rule

\[ \begin{aligned} P(H\mid D) &= \frac{P(D\mid H) \times P(H)}{P(D)}\\[1em] \mathsf{posterior\\belief\\(probability)} &= \frac{likelihood \times \mathsf{prior~probability}}{\mathsf{normalizing~constant}} \end{aligned} \]

Bayesian

Bayes rule

\[ \begin{aligned} P(H\mid D) &= \frac{P(D\mid H) \times P(H)}{P(D)}\\ \mathsf{posterior\\belief\\(probability)} &= \frac{likelihood \times \mathsf{prior~probability}}{\mathsf{normalizing~constant}} \end{aligned} \]

The normalizing constant is required for probability - turn a frequency distribution into a probability distribution

Estimation: Likelihood

\(P(D\mid H)\)

Bayesian

conclusions pertain to hypotheses

computationally robust (sample size,balance,collinearity)

inferential flexibility - derive any number of inferences