Introduction to Bayesian Statistics

Murray Logan

15 September 2024

Frequentist vs Bayesian

Frequentist

P(D|H)
long-run frequency
relatively simple analytical methods to solve roots
conclusions pertain to data, not parameters or hypotheses
compared to theoretical distribution when NULL hypothesis is true
probability of obtaining observed data or MORE EXTREME data

Frequentist

P-value
- probabulity of rejecting NULL
- NOT a measure of the magnitude of an effect or degree of significance!
- measure of whether the sample size is large enough
95% CI
- NOT about the parameter it is about the interval
- does not tell you the range of values likely to contain the true mean

Frequentist vs Bayesian

	Frequentist	Bayesian
Probability	Long-run frequency \(P(D\|H)\)	Degree of belief \(P(H\|D)\)
Parameters	Fixed, true	Random, distribution
Obs. data	One possible	Fixed, true
Inferences	Data	Parameters

Frequentist vs Bayesian

n: 10
Slope: -0.1022
t: -2.3252
p: 0.0485

n: 10
Slope: -10.2318
t: -2.2115
p: 0.0579

n: 100
Slope: -10.4713
t: -6.6457
p: 1.7101362^{-9}

Frequentist vs Bayesian

	Population A	Population B
Percentage change	0.46	45.46
Prob. >5% decline	0	0.86

Bayesian Statistics

Bayesian

Bayes rule

\[ \begin{aligned} P(H\mid D) &= \frac{P(D\mid H) \times P(H)}{P(D)}\\[1em] \mathsf{posterior\\belief\\(probability)} &= \frac{likelihood \times \mathsf{prior~probability}}{\mathsf{normalizing~constant}} \end{aligned} \]

Bayesian

Bayes rule

\[ \begin{aligned} P(H\mid D) &= \frac{P(D\mid H) \times P(H)}{P(D)}\\ \mathsf{posterior\\belief\\(probability)} &= \frac{likelihood \times \mathsf{prior~probability}}{\mathsf{normalizing~constant}} \end{aligned} \]

The normalizing constant is required for probability - turn a frequency distribution into a probability distribution

Estimation: Likelihood

\(P(D\mid H)\)

Bayesian

conclusions pertain to hypotheses
computationally robust (sample size,balance,collinearity)
inferential flexibility - derive any number of inferences