Preface: how to use this book
There is one equation behind this entire book, and it fits on a single line: \[P(\Hyp\mid D)=\frac{P(D\mid \Hyp)\,P(\Hyp)}{P(D)}\,.\] Bayes’ theorem is a recipe for learning from evidence: start with what you believe (the prior), see how well each possibility explains the data (the likelihood), and end with an updated belief (the posterior). That is all of Bayesian inference — and, remarkably, it is also a lens through which much of modern machine learning comes into focus. Regularization is a prior. The loss is a negative log-likelihood. Uncertainty is a posterior. A spam filter is Bayes’ theorem with an independence shortcut. This book builds the idea from a coin flip all the way to Bayesian neural networks.
Who this is for
This book starts from the very beginning — what a probability means — and climbs to conjugate analysis, Naive Bayes classifiers, Bayesian networks, Markov chain Monte Carlo, variational inference, hierarchical models, and Bayesian model comparison. It is for the self-learner who wants both fluency (“I can set up a prior, compute a posterior, and make a decision”) and understanding (“I know why the posterior is a compromise, when the prior matters, and how this connects to the rest of statistics and ML”). Every major idea gets a picture, a worked example, and a forward link to machine learning.
The central idea
All of Bayesian reasoning orbits one move and its consequences:
Probability is degree of belief. We assign probabilities to hypotheses, not just to repeatable events — so we can talk about the probability that a parameter lies in a range, or that a model is true.
Learning is conditioning. New data updates belief by Bayes’ theorem: \(\post\propto \text{likelihood}\times\prior\). Yesterday’s posterior is today’s prior.
Answers are distributions, not points. The output is a full posterior, carrying its own uncertainty — from which we extract estimates, intervals, predictions, and decisions.
The structure
Part I — Foundations (beginner): Bayesian thinking; Bayes’ theorem and its classic puzzles.
Part II — The Bayesian Engine (core): prior, likelihood, and posterior; conjugate updating; choosing priors; summarizing the posterior and making decisions.
Part III — Classification & Comparison (advanced): Naive Bayes classifiers; Bayesian versus frequentist inference.
Part IV — Structure & Computation (advanced/expert): Bayesian networks; MCMC and variational inference; Bayesian regression and hierarchical models.
Part V — Frontier: model comparison and Occam’s razor; and a capstone on Bayes in ML/DL/AI.
How to read it
Read with a pen and a computer. When you meet a Definition, restate it; when you meet an Example, work it before reading on; when you meet a Try it in code box, run it. The colored boxes recur throughout:
Key idea — the one sentence to remember.
Intuition — the picture or analogy behind the math.
Common pitfall — the mistakes that bite everyone.
How this powers ML / AI — the forward link to applications.
Try it in code — a hands-on check in Python (NumPy / SciPy / scikit-learn).
Hold one picture above all others: a Bayesian starts with a belief, sees data, and ends with a sharper belief. The prior is where you begin, the likelihood is what the data says, and the posterior is the negotiated result — always a compromise between the two, tilting toward the data as evidence accumulates. Everything else — conjugacy, Naive Bayes, MCMC, Bayesian deep learning — is machinery for carrying out this one update when the math gets hard.