A compact reference to the book’s key results.
The core identities
| Quantity | Formula |
|---|---|
| Bayes’ theorem | \(P(\Hyp\mid D)=\dfrac{P(D\mid\Hyp)\,P(\Hyp)}{P(D)}\) |
| Working form | \(p(\theta\mid D)\propto p(D\mid\theta)\,p(\theta)\) |
| Evidence (marginal likelihood) | \(p(D)=\int p(D\mid\theta)\,p(\theta)\,d\theta\) |
| Posterior predictive | \(p(\tilde x\mid D)=\int p(\tilde x\mid\theta)\,p(\theta\mid D)\,d\theta\) |
| MAP estimate | \(\hat\theta=\argmax_\theta\big[\log p(D\mid\theta)+\log p(\theta)\big]\) |
| Bayes action | \(a^\star=\argmin_a \E_{\theta\mid D}[L(\theta,a)]\) |
| Bayes factor | \(\mathrm{BF}_{10}=p(D\mid M_1)/p(D\mid M_0)\) |
| ELBO | \(\E_q[\log p(D,\theta)]-\E_q[\log q(\theta)]\le \log p(D)\) |
| Naive Bayes | \(\hat C=\argmax_C P(C)\prod_j P(x_j\mid C)\) |
| Laplace smoothing | \(P(w\mid C)=\dfrac{\text{count}(w,C)+\alpha}{\text{count}(C)+\alpha V}\) |
Conjugate prior cheat sheet
| Likelihood | Parameter | Conjugate prior | Posterior update |
|---|---|---|---|
| Bernoulli / Binomial | prob. \(\theta\) | \(\Betad(\alpha,\beta)\) | \(\Betad(\alpha+h,\ \beta+n-h)\) |
| Poisson | rate \(\lambda\) | \(\Gammad(\alpha,\beta)\) | \(\Gammad(\alpha+\textstyle\sum x,\ \beta+n)\) |
| Normal (known \(\sigma^2\)) | mean \(\mu\) | \(\Normal(\mu_0,\tau_0^2)\) | precision-weighted (see Ch. [ch:conjugate]) |
| Multinomial | probs. \(\boldsymbol\theta\) | Dirichlet\((\boldsymbol\alpha)\) | Dirichlet\((\boldsymbol\alpha+\text{counts})\) |
| Normal (known \(\mu\)) | variance \(\sigma^2\) | Inverse-Gamma | Inverse-Gamma (updated) |
| Exponential | rate \(\lambda\) | \(\Gammad(\alpha,\beta)\) | \(\Gammad(\alpha+n,\ \beta+\textstyle\sum x)\) |
Choosing a computational method
| Situation | Recommended approach |
|---|---|
| Conjugate model | closed-form update (Ch. [ch:conjugate]) |
| Low-dim, want exactness | MCMC: HMC / NUTS (Ch. [ch:mcmc]) |
| Conjugate full conditionals | Gibbs sampling |
| Large model / dataset, speed | variational inference (ELBO) |
| Deep network uncertainty | MC dropout, deep ensembles, Laplace, SG-MCMC |
| Discrete graphical model | variable elimination / belief propagation (Ch. [ch:networks]) |
| Expensive black-box tuning | Bayesian optimization (GP surrogate) |
| Many related groups | hierarchical model + partial pooling (Ch. [ch:hierarchical]) |
Workflow checklist
State the model: likelihood (data-generating story) and priors on all parameters.
Run a prior predictive check; revise priors that imply absurd data.
Compute the posterior (closed form, MCMC, or VI); for MCMC check \(\hat R\), ESS, and trace plots.
Run a posterior predictive check: can the model reproduce the data?
Summarize with means/credible intervals; make decisions by minimizing expected loss.
Compare models with LOO/WAIC; do a prior sensitivity analysis; consider model averaging.
Symbols
| Symbol | Meaning |
|---|---|
| \(\theta,\ \vw\) | parameter(s) / network weights |
| \(D,\ \Ldata\) | observed data |
| \(\Hyp,\ M\) | hypothesis; model |
| \(p(\theta)\) / \(p(\theta\mid D)\) | prior / posterior |
| \(p(D\mid\theta)\) | likelihood |
| \(p(D)\) | evidence (marginal likelihood) |
| \(\alpha,\beta\) | prior hyperparameters (pseudo-counts) |
| \(q_\phi(\theta)\) | variational approximation |
| \(\KL(q\,\|\,p)\) | Kullback–Leibler divergence |