Preface: how to use this book
If linear algebra is the language of machine learning and calculus is its engine, then probability and statistics are its worldview. Machine learning is, at its core, the discipline of drawing reliable conclusions from noisy, incomplete data — which is exactly what statistics was invented to do. Every prediction a model makes is a probabilistic statement; every loss function it minimizes is, on closer inspection, a likelihood; every claim that a model “generalizes” is a statistical claim about unseen data. This book builds that worldview from first principles and carries it to the frontier of modern AI.
Who this is for
This book starts from the very beginning — what a probability is — and climbs to maximum likelihood, Bayesian inference, Markov chain Monte Carlo, and information theory. It is written for the self-learner who wants both fluency (“I can compute the posterior, run the test, fit the model”) and understanding (“I know what these numbers mean and when they lie”). Every major idea is motivated with a picture, a worked example, and a forward link to how it is used in machine learning.
The two halves
The subject has two complementary halves, and you need both:
Probability (Parts I–III): given a model of the world, what data should we expect? This is the forward direction — from assumptions to predictions.
Statistics (Parts IV–V): given the data, what model of the world is plausible? This is the inverse direction — from observations back to the underlying truth. It is the direction machine learning lives in.
The structure
Part I — Foundations of Probability (beginner): the rules of probability; conditioning and Bayes’ theorem.
Part II — Random Variables & Distributions (core): random variables and expectation; the common distributions; joint distributions and the multivariate Gaussian.
Part III — Limit Theorems (core): inequalities, the law of large numbers, and the central limit theorem.
Part IV — Statistical Inference (advanced): estimation and maximum likelihood; confidence intervals and hypothesis testing; Bayesian inference; regression.
Part V — Advanced & Computational: stochastic processes, Markov chains and Monte Carlo; information theory; and a capstone on probability and statistics in ML/DL/AI.
How to read it
Read with a pen and a computer. When you meet a Definition, restate it; when you meet an Example, try it before reading the solution; when you meet a Try it in code box, run it. The colored boxes recur throughout:
Key idea — the one sentence to remember.
Intuition — the picture behind the formula.
Common pitfall — the mistakes that bite everyone.
How this powers ML / AI — the forward link to applications.
Try it in code — a hands-on check in Python/NumPy/SciPy.
Hold one idea above all others: learning from data is inference under uncertainty. A model never knows the truth; it maintains beliefs and updates them as evidence arrives. Probability is the calculus of those beliefs, and statistics is the practice of forming them from data. Master this and the entire training loop of any model — from a coin-flip estimator to a diffusion transformer — becomes one coherent story.