Random Variables and Expectation
So far events have been qualitative (“it rained”). A random variable attaches numbers to outcomes, letting us do arithmetic with uncertainty: average it, measure its spread, add independent copies. Random variables and t
阅读文章 →Common Probability Distributions
A handful of distributions describe most of the randomness we ever model. Each arises from a simple, recurring story — a coin flip, a count of rare events, a sum of many small effects — and learning those stories lets yo
阅读文章 →Joint Distributions, Covariance, and the Multivariate Gaussian
Real data is rarely one number; it is vectors of many interrelated quantities — pixels, features, tokens. To model them we need joint distributions that capture how variables vary together, the covariance that quantifies
阅读文章 →Inequalities and Limit Theorems
Why does averaging more data give a better answer? Why are so many things bell-shaped? Why can we estimate an expectation by a sample mean — the move that justifies mini-batch training? The limit theorems answer these qu
阅读文章 →Estimation and Maximum Likelihood
We now cross from probability into statistics — from “given the model, predict the data” to “given the data, infer the model.” The central task is estimation: using a sample to guess an unknown parameter. The dominant me
阅读文章 →Confidence Intervals and Hypothesis Testing
A point estimate alone is dangerous: it hides its own uncertainty. This chapter adds the error bars. Confidence intervals express how precise an estimate is; hypothesis tests decide whether an observed effect is real or
阅读文章 →Bayesian Inference
Frequentist statistics treats parameters as fixed unknowns and data as random. Bayesian inference flips the emphasis: it treats parameters as random variables with probability distributions that encode our beliefs, and u
阅读文章 →Statistical Models and Regression
Regression is where probability and statistics become predictive machine learning. A statistical model describes how outputs depend on inputs plus randomness; regression fits that dependence to data. Linear and logistic
阅读文章 →Stochastic Processes, Markov Chains, and Monte Carlo
So far our randomness has been static. A stochastic process adds time: a sequence of random variables that evolve and depend on one another. The most important kind — the Markov chain — forgets everything but the present
阅读文章 →Information Theory
Information theory, born from Claude Shannon's 1948 work on communication, turns probability into a measure of information and surprise. Its quantities — entropy, cross-entropy, KL divergence, mutual information — are no
阅读文章 →Probability and Statistics in Machine Learning, Deep Learning, and AI
Here is where everything converges. Machine learning is applied probability and statistics at scale: it posits a probabilistic model of data, estimates its parameters by (penalized) maximum likelihood, quantifies uncerta
阅读文章 →Formulas and Method Reference
A compact reference to the book's key formulas and a decision guide for choosing a method.
阅读文章 →What Is Regression?
Regression answers a deceptively simple question: given some inputs, what number should I predict for the output, and how are they related? Predict a house's price from its size; a patient's risk from their labs; tomorro
阅读文章 →Simple Linear Regression
We begin where regression itself began: fitting a straight line through points. Simple linear regression — one predictor, one response — is the entire subject in miniature. Every concept you will need later (a model, a l
阅读文章 →Multiple Linear Regression and Its Geometry
Real problems have many predictors at once: price depends on size and location and age. Multiple linear regression extends the line to a plane, then a hyperplane, and — crucially — introduces the matrix algebra and geome
阅读文章 →Assumptions, Diagnostics, and Inference
A fitted model always produces coefficients — but whether you can trust them, predict with them, or attach uncertainty to them depends on assumptions. This chapter states what OLS assumes, shows how to check it with resi
阅读文章 →Feature Engineering and Model Building
Linear regression is linear in the parameters, not necessarily in the raw inputs. That loophole is enormous: by transforming and combining predictors we can model curves, interactions, categories, and saturating effects
阅读文章 →The Bias–Variance Tradeoff and Model Validation
A model that fits the training data perfectly is often useless on new data. This chapter explains why — the bias–variance tradeoff, the most important concept in all of predictive modeling — and gives the tools to measur
阅读文章 →Regularization: Ridge, Lasso, and Elastic Net
When predictors are many or correlated, ordinary least squares overfits: coefficients explode, variance soars, and predictions on new data suffer. Regularization fixes this by adding a penalty that discourages large coef
阅读文章 →Logistic Regression and Classification
What if the response is a category — spam or not, disease or healthy, click or no click? Fitting a line to 0/1 labels gives nonsensical probabilities outside [0,1]. Logistic regression fixes this elegantly: it runs a lin
阅读文章 →Generalized Linear Models
Linear regression assumes Gaussian, constant-variance noise; logistic regression handles binary outcomes. Generalized linear models (GLMs) reveal these as two members of one family, unifying regression for continuous, bi
阅读文章 →Nonlinear and Nonparametric Regression
When the relationship is genuinely curved and you do not know its form, you need methods that let the data dictate the shape. This chapter surveys the flexible end of regression: polynomials and splines, local and kernel
阅读文章 →Robust, Quantile, and Bayesian Regression
Ordinary least squares is optimal under Gaussian noise — but real data has outliers, heavy tails, and asymmetric costs, and sometimes we want full probability distributions over our answers rather than point estimates. T
阅读文章 →Evaluation, Pitfalls, and Causality
A regression can be technically correct and practically misleading. This chapter is the practitioner's survival guide: how to measure performance with the right metric, the classic traps that fool even experts, and the b
阅读文章 →