Stochastic Processes, Markov Chains, and Monte Carlo
So far our randomness has been static. A stochastic process adds time: a sequence of random variables that evolve and depend on one another. The most important kind — the Markov chain — forgets everything but the present
記事を読む →Information Theory
Information theory, born from Claude Shannon's 1948 work on communication, turns probability into a measure of information and surprise. Its quantities — entropy, cross-entropy, KL divergence, mutual information — are no
記事を読む →Probability and Statistics in Machine Learning, Deep Learning, and AI
Here is where everything converges. Machine learning is applied probability and statistics at scale: it posits a probabilistic model of data, estimates its parameters by (penalized) maximum likelihood, quantifies uncerta
記事を読む →Formulas and Method Reference
A compact reference to the book's key formulas and a decision guide for choosing a method.
記事を読む →What Is Regression?
Regression answers a deceptively simple question: given some inputs, what number should I predict for the output, and how are they related? Predict a house's price from its size; a patient's risk from their labs; tomorro
記事を読む →Simple Linear Regression
We begin where regression itself began: fitting a straight line through points. Simple linear regression — one predictor, one response — is the entire subject in miniature. Every concept you will need later (a model, a l
記事を読む →Multiple Linear Regression and Its Geometry
Real problems have many predictors at once: price depends on size and location and age. Multiple linear regression extends the line to a plane, then a hyperplane, and — crucially — introduces the matrix algebra and geome
記事を読む →Assumptions, Diagnostics, and Inference
A fitted model always produces coefficients — but whether you can trust them, predict with them, or attach uncertainty to them depends on assumptions. This chapter states what OLS assumes, shows how to check it with resi
記事を読む →Feature Engineering and Model Building
Linear regression is linear in the parameters, not necessarily in the raw inputs. That loophole is enormous: by transforming and combining predictors we can model curves, interactions, categories, and saturating effects
記事を読む →The Bias–Variance Tradeoff and Model Validation
A model that fits the training data perfectly is often useless on new data. This chapter explains why — the bias–variance tradeoff, the most important concept in all of predictive modeling — and gives the tools to measur
記事を読む →Regularization: Ridge, Lasso, and Elastic Net
When predictors are many or correlated, ordinary least squares overfits: coefficients explode, variance soars, and predictions on new data suffer. Regularization fixes this by adding a penalty that discourages large coef
記事を読む →Logistic Regression and Classification
What if the response is a category — spam or not, disease or healthy, click or no click? Fitting a line to 0/1 labels gives nonsensical probabilities outside [0,1]. Logistic regression fixes this elegantly: it runs a lin
記事を読む →Generalized Linear Models
Linear regression assumes Gaussian, constant-variance noise; logistic regression handles binary outcomes. Generalized linear models (GLMs) reveal these as two members of one family, unifying regression for continuous, bi
記事を読む →Nonlinear and Nonparametric Regression
When the relationship is genuinely curved and you do not know its form, you need methods that let the data dictate the shape. This chapter surveys the flexible end of regression: polynomials and splines, local and kernel
記事を読む →Robust, Quantile, and Bayesian Regression
Ordinary least squares is optimal under Gaussian noise — but real data has outliers, heavy tails, and asymmetric costs, and sometimes we want full probability distributions over our answers rather than point estimates. T
記事を読む →Evaluation, Pitfalls, and Causality
A regression can be technically correct and practically misleading. This chapter is the practitioner's survival guide: how to measure performance with the right metric, the classic traps that fool even experts, and the b
記事を読む →Regression in Machine Learning, Deep Learning, and AI
This capstone ties the book together by arguing a single thesis: regression is not just one tool in machine learning — it is the conceptual skeleton of the entire field. Supervised learning is regression and classificati
記事を読む →Regression — Preface: how to use this book
Regression is the oldest and most useful idea in data analysis: predict a number from other numbers, and understand the relationship between them. Two centuries after Gauss and Legendre fit lines to astronomical data, re
記事を読む →Support Vector Machines — Preface: how to use this book
For most of the 1990s and 2000s, if you wanted the best off-the-shelf classifier, you reached for a Support Vector Machine. SVMs combined a beautiful geometric idea — separate the classes with the widest possible street
記事を読む →Formulas, Kernels, and Hyperparameter Reference
A compact reference for the formulas, kernels, and knobs used throughout the book.
記事を読む →What Are Support Vector Machines?
Imagine two groups of points on a page and a ruler you must lay down to separate them. Many positions work — but which is best? A Support Vector Machine answers: the line that leaves the widest empty street between the g
記事を読む →The Maximal Margin Classifier
Now we make the widest-street idea precise. For data that can be perfectly separated by a line, the maximal margin classifier (the hard-margin SVM) is the cleanest version of the story: a little geometry turns “make the
記事を読む →Soft Margins and Slack
Real data overlaps, contains noise, and is rarely perfectly separable. The soft-margin SVM — the model people actually use — relaxes the hard constraints by allowing a controlled budget of margin violations. This single
記事を読む →Hinge Loss and the Primal Problem
We ended the last chapter with a striking reformulation: the soft-margin SVM is just L_2 regularization plus a special loss. That loss is the hinge loss, and viewing the SVM as “minimize regularized hinge loss” (the prim
記事を読む →