Now we put the machinery to work on the models practitioners use every day. Bayesian regression replaces point-estimate coefficients with full posteriors, delivering principled uncertainty on every prediction. Hierarchical (multilevel) models go further, sharing information across related groups through a shared prior — the elegant idea of “partial pooling” that underlies much of modern applied Bayesian statistics.

Bayesian linear regression

Place a prior on the regression coefficients and infer their posterior. With a Gaussian prior \(\vbeta\sim\Normal(\mathbf 0,\tau^2\mathbf I)\) and Gaussian noise, the posterior over \(\vbeta\) is Gaussian (conjugate), and predictions use the posterior predictive \[p(\tilde y\mid \tilde{\vx},D)=\int p(\tilde y\mid \tilde{\vx},\vbeta)\,p(\vbeta\mid D)\,d\vbeta.\]

Bayesian linear regression returns a distribution over lines, not one line. Two consequences follow for free: the posterior mean coefficients equal the ridge estimate (the Gaussian prior is the \(L_2\) penalty of Chapter [ch:priors]), and the predictive variance grows where data is sparse — the model is appropriately unsure when extrapolating. You get regularization and calibrated uncertainty from one coherent recipe.

Hierarchical models and partial pooling

A hierarchical (multilevel) model gives each group \(g\) its own parameters \(\theta_g\), drawn from a shared population prior whose hyperparameters are themselves estimated: \[\theta_g\mid \mu,\tau \sim \Normal(\mu,\tau^2),\qquad \mu,\tau\sim\text{hyperprior}.\]

This formalizes partial pooling, the middle path between two bad extremes. Complete pooling ignores groups (one estimate for all — too rigid); no pooling fits each group alone (noisy, overfit for small groups). The hierarchical model learns how much to pool from the data: groups with little data are pulled (shrunk) toward the population mean, groups with abundant data keep their own estimate. It borrows strength across groups automatically.

This is why a baseball player with 5 at-bats should not be projected as a \(.800\) hitter: the hierarchical model shrinks that noisy estimate toward the league average, by an amount the data dictates. The same logic stabilizes estimates for small schools, rare diseases, new products — anywhere you have many related but unevenly-sampled groups. Hierarchical shrinkage is the principled version of “regularize the small-sample estimates.”

Bayesian logistic and generalized models

The same recipe extends to Bayesian logistic regression and other GLMs: keep the link and likelihood, put priors on the coefficients, and infer the posterior (no longer conjugate, so use MCMC or VI from Chapter [ch:mcmc]). The reward is posterior distributions over odds ratios and calibrated predictive probabilities with honest uncertainty — valuable in medicine, risk, and any setting where overconfidence is costly.

Hierarchical models have characteristic traps. The funnel: when a group has little data, the geometry of \(\theta_g\) versus \(\tau\) becomes pathological and samplers struggle — a non-centered reparameterization usually fixes it. Choosing the hyperprior on the group variance \(\tau\) matters: too tight forces over-pooling, too loose allows over-fitting; half-Normal or half-Cauchy priors are common defaults. And always run posterior predictive checks (Chapter [ch:modelcomp]) to confirm the model can reproduce the data’s structure.

Hierarchical Bayes is the statistical ancestor of ideas everywhere in ML. Shrinkage toward a shared prior is exactly transfer learning and multi-task learning, where related tasks share parameters or a prior. Empirical Bayes — estimating the prior from data — underlies modern meta-learning (“learning to learn,” e.g. MAML as hierarchical Bayes) and automatic relevance determination. Partial pooling is the principled cousin of regularization and is used in large-scale recommendation and experimentation platforms to stabilize millions of per-item or per-user estimates. And Bayesian regression with predictive uncertainty is the backbone of Bayesian optimization and active learning.

import numpy as np
# James-Stein-style shrinkage: pull noisy group means toward the grand mean
rng = np.random.default_rng(0)
true = rng.normal(0, 1, 8)                  # 8 groups' true effects
obs  = true + rng.normal(0, 1, 8)           # noisy observations (n=1 each)
grand = obs.mean()
tau2, s2 = obs.var(), 1.0                    # pop variance vs noise variance
w = tau2 / (tau2 + s2)                       # data-driven pooling weight
shrunk = grand + w*(obs - grand)             # partial pooling
print("MSE no-pool:", round(((obs-true)**2).mean(),3),
      " MSE shrunk:", round(((shrunk-true)**2).mean(),3))   # shrinkage usually wins

Chapter summary

  • Bayesian regression infers a posterior over coefficients; the mean is the ridge estimate and predictive variance grows where data is sparse.

  • Predictions use the posterior predictive (marginalize over coefficients) for calibrated uncertainty.

  • Hierarchical models draw group parameters from a shared prior, giving partial pooling: small groups shrink toward the population mean.

  • Watch the funnel (use non-centered parameterization) and the variance hyperprior; check with posterior predictive checks.

  • Hierarchical/empirical Bayes underlies transfer and multi-task learning, meta-learning, and large-scale shrinkage in ML.