Random Variables and Expectation
So far events have been qualitative (``it rained''). A attaches numbers to outcomes, letting us do arithmetic with uncertainty: average it, measure its spread, add independent copies. Random variables and their two summa
閱讀文章 →Common Probability Distributions
A handful of distributions describe most of the randomness we ever model. Each arises from a simple, recurring story — a coin flip, a count of rare events, a sum of many small effects — and learning those stories lets yo
閱讀文章 →Joint Distributions, Covariance, and the Multivariate Gaussian
Real data is rarely one number; it is vectors of many interrelated quantities — pixels, features, tokens. To model them we need that capture how variables vary , the that quantifies their linear relationship, and the , t
閱讀文章 →Inequalities and Limit Theorems
Why does averaging more data give a better answer? Why are so many things bell-shaped? Why can we estimate an expectation by a sample mean — the move that justifies mini-batch training? The answer these questions. They a
閱讀文章 →Estimation and Maximum Likelihood
We now cross from probability into statistics — from ``given the model, predict the data'' to ``given the data, infer the model.'' The central task is : using a sample to guess an unknown parameter. The dominant method,
閱讀文章 →Confidence Intervals and Hypothesis Testing
A point estimate alone is dangerous: it hides its own uncertainty. This chapter adds the error bars. express how precise an estimate is; decide whether an observed effect is real or could be chance. These frequentist too
閱讀文章 →Bayesian Inference
Frequentist statistics treats parameters as fixed unknowns and data as random. flips the emphasis: it treats parameters as random variables with probability distributions that encode our beliefs, and updates those belief
閱讀文章 →Statistical Models and Regression
Regression is where probability and statistics become predictive machine learning. A describes how outputs depend on inputs plus randomness; fits that dependence to data. Linear and logistic regression are the simplest s
閱讀文章 →Stochastic Processes, Markov Chains, and Monte Carlo
So far our randomness has been static. A adds time: a sequence of random variables that evolve and depend on one another. The most important kind — the — forgets everything but the present, yet is rich enough to model la
閱讀文章 →Information Theory
Information theory, born from Claude Shannon's 1948 work on communication, turns probability into a measure of and . Its quantities — entropy, cross-entropy, KL divergence, mutual information — are not exotic add-ons to
閱讀文章 →Probability and Statistics in Machine Learning, Deep Learning, and AI
Here is where everything converges. Machine learning is applied probability and statistics at scale: it posits a probabilistic model of data, estimates its parameters by (penalized) maximum likelihood, quantifies uncerta
閱讀文章 →Formulas and Method Reference
A compact reference to the book's key formulas and a decision guide for choosing a method.
閱讀文章 →What Is Regression?
Regression answers a deceptively simple question: Predict a house's price from its size; a patient's risk from their labs; tomorrow's demand from today's. This chapter sets up the vocabulary — response and predictors, th
閱讀文章 →Simple Linear Regression
We begin where regression itself began: fitting a straight line through points. — one predictor, one response — is the entire subject in miniature. Every concept you will need later (a model, a loss, least-squares fittin
閱讀文章 →Multiple Linear Regression and Its Geometry
Real problems have many predictors at once: price depends on size location age. extends the line to a plane, then a hyperplane, and — crucially — introduces the matrix algebra and geometric picture that unify all of line
閱讀文章 →Assumptions, Diagnostics, and Inference
A fitted model always produces coefficients — but whether you can them, predict with them, or attach uncertainty to them depends on assumptions. This chapter states what OLS assumes, shows how to check it with residual p
閱讀文章 →Feature Engineering and Model Building
Linear regression is linear , not necessarily in the raw inputs. That loophole is enormous: by transforming and combining predictors we can model curves, interactions, categories, and saturating effects — all while keepi
閱讀文章 →The Bias–Variance Tradeoff and Model Validation
A model that fits the training data perfectly is often useless on new data. This chapter explains why — the , the most important concept in all of predictive modeling — and gives the tools to measure generalization hones
閱讀文章 →Regularization: Ridge, Lasso, and Elastic Net
When predictors are many or correlated, ordinary least squares overfits: coefficients explode, variance soars, and predictions on new data suffer. fixes this by adding a penalty that discourages large coefficients — deli
閱讀文章 →Logistic Regression and Classification
What if the response is a category — spam or not, disease or healthy, click or no click? Fitting a line to 0/1 labels gives nonsensical probabilities outside [0,1]. fixes this elegantly: it runs a linear model through a
閱讀文章 →Generalized Linear Models
Linear regression assumes Gaussian, constant-variance noise; logistic regression handles binary outcomes. (GLMs) reveal these as two members of one family, unifying regression for continuous, binary, count, and positive
閱讀文章 →Nonlinear and Nonparametric Regression
When the relationship is genuinely curved and you do not know its form, you need methods that let the dictate the shape. This chapter surveys the flexible end of regression: polynomials and splines, local and kernel regr
閱讀文章 →Robust, Quantile, and Bayesian Regression
Ordinary least squares is optimal under Gaussian noise — but real data has outliers, heavy tails, and asymmetric costs, and sometimes we want full probability distributions over our answers rather than point estimates. T
閱讀文章 →Evaluation, Pitfalls, and Causality
A regression can be technically correct and practically misleading. This chapter is the practitioner's survival guide: how to measure performance with the right metric, the classic traps that fool even experts, and the b
閱讀文章 →