The posterior is a full distribution, but people need answers: a single estimate, an interval, a yes/no decision. This chapter shows how to summarize a posterior responsibly (point estimates and credible intervals) and how Bayesian decision theory converts beliefs into optimal actions by minimizing expected loss — the principled bridge from “what do I believe?” to “what should I do?”
Point estimates
Three common single-number summaries of a posterior \(p(\theta\mid D)\):
Posterior mean \(\E[\theta\mid D]\) — minimizes expected squared error.
Posterior median — minimizes expected absolute error; robust to skew.
Posterior mode (MAP) \(\argmax_\theta p(\theta\mid D)\) — the single most probable value.
Which to report depends on the loss you care about (next section) and the posterior’s shape. For a symmetric, unimodal posterior all three coincide. For a skewed posterior they diverge, and the mean can sit in a low-density region — so always look at the whole distribution before collapsing it. The MAP is the cheapest to compute (just optimize), which is why it dominates large-scale ML, but it ignores the posterior’s width.
Credible intervals
A \(95\%\) credible interval is a range containing \(95\%\) of the posterior probability. Two common forms: the equal-tailed interval (2.5% in each tail) and the highest posterior density (HPD) interval — the shortest interval containing 95% of the mass.
A credible interval means exactly what people wish a confidence interval meant: “given the data and prior, there is a 95% probability the parameter lies in this range.” That is a direct probability statement about \(\theta\) — legitimate because, to a Bayesian, \(\theta\) has a distribution. Contrast this with the frequentist confidence interval (Chapter [ch:vsfreq]), whose 95% refers to the long-run coverage of the procedure, not to your one interval.
Bayesian decision theory
A loss function \(L(\theta,a)\) gives the cost of taking action \(a\) when the truth is \(\theta\). The Bayes action minimizes the posterior expected loss: \[a^\star=\argmin_a\ \E_{\theta\mid D}\big[L(\theta,a)\big]=\argmin_a\int L(\theta,a)\,p(\theta\mid D)\,d\theta.\]
This is the payoff of being Bayesian: a single, coherent recipe for optimal action under uncertainty — average the loss over everything you don’t know, then pick the best action. Different losses recover different estimators automatically: squared-error loss \(\Rightarrow\) posterior mean; absolute-error loss \(\Rightarrow\) posterior median; \(0/1\) loss \(\Rightarrow\) posterior mode. The estimate is not arbitrary; it is dictated by what mistakes cost you.
Decisions should depend on consequences, not just beliefs. If missing a disease is far worse than a false alarm, the optimal action treats at a posterior probability well below 50%. Bayesian decision theory makes this asymmetry explicit through the loss function, rather than hiding it in an arbitrary threshold — the same logic that sets classification thresholds in ML.
Beware reporting the MAP as if it summarized the posterior: in high dimensions the mode can be wildly unrepresentative of where the mass actually is (the “mean is not the mode” problem, severe for skewed or multimodal posteriors). Also, a credible interval is only as trustworthy as the model and prior that produced it — 95% credibility under a bad model is false comfort. And do not confuse credible intervals (Bayesian, about \(\theta\)) with confidence intervals (frequentist, about the procedure); they answer different questions.
Decision theory is the hidden backbone of applied ML. Choosing a classification threshold to balance precision and recall is minimizing expected loss under an asymmetric cost matrix. Bayes-optimal prediction — the theoretical best any classifier can do — is exactly the Bayes action under \(0/1\) loss. Active learning and Bayesian optimization choose the next query to minimize expected future loss (maximize expected information or improvement). And in reinforcement learning, acting to maximize expected return is decision theory with the posterior over outcomes. Whenever an AI system “decides,” it is (ideally) computing a Bayes action.
import numpy as np
from scipy.stats import beta
post = beta(9, 4) # example posterior over theta
print("mean:", round(post.mean(), 3), " median:", round(post.median(), 3))
lo, hi = post.ppf([0.025, 0.975]) # 95% equal-tailed credible interval
print("95% CI:", (round(lo,3), round(hi,3)))
# Decision: treat if expected loss of not-treating exceeds that of treating
c_fn, c_fp = 10.0, 1.0 # cost of false negative vs false positive
p_disease = 1 - post.cdf(0.5) # posterior prob theta>0.5, say
print("treat?", c_fn*p_disease > c_fp*(1-p_disease))Chapter summary
Summarize a posterior with the mean (squared-error optimal), median (absolute-error optimal), or mode/MAP (\(0/1\) optimal).
A credible interval is a direct probability statement about \(\theta\) — what a confidence interval is often mistaken for.
Bayesian decision theory: the optimal action minimizes posterior expected loss; the loss determines the estimator.
Asymmetric losses justify thresholds away from 50% — decisions follow consequences, not just beliefs.
Beware the MAP in high dimensions and credibility under a bad model; this machinery underlies thresholds, Bayes-optimal prediction, and active learning.