Stochastic Processes, Markov Chains, and Monte Carlo
So far our randomness has been static. A adds time: a sequence of random variables that evolve and depend on one another. The most important kind — the — forgets everything but the present, yet is rich enough to model la
記事を読む →Information Theory
Information theory, born from Claude Shannon's 1948 work on communication, turns probability into a measure of and . Its quantities — entropy, cross-entropy, KL divergence, mutual information — are not exotic add-ons to
記事を読む →Probability and Statistics in Machine Learning, Deep Learning, and AI
Here is where everything converges. Machine learning is applied probability and statistics at scale: it posits a probabilistic model of data, estimates its parameters by (penalized) maximum likelihood, quantifies uncerta
記事を読む →Formulas and Method Reference
A compact reference to the book's key formulas and a decision guide for choosing a method.
記事を読む →What Is Regression?
Regression answers a deceptively simple question: Predict a house's price from its size; a patient's risk from their labs; tomorrow's demand from today's. This chapter sets up the vocabulary — response and predictors, th
記事を読む →Simple Linear Regression
We begin where regression itself began: fitting a straight line through points. — one predictor, one response — is the entire subject in miniature. Every concept you will need later (a model, a loss, least-squares fittin
記事を読む →Multiple Linear Regression and Its Geometry
Real problems have many predictors at once: price depends on size location age. extends the line to a plane, then a hyperplane, and — crucially — introduces the matrix algebra and geometric picture that unify all of line
記事を読む →Assumptions, Diagnostics, and Inference
A fitted model always produces coefficients — but whether you can them, predict with them, or attach uncertainty to them depends on assumptions. This chapter states what OLS assumes, shows how to check it with residual p
記事を読む →Feature Engineering and Model Building
Linear regression is linear , not necessarily in the raw inputs. That loophole is enormous: by transforming and combining predictors we can model curves, interactions, categories, and saturating effects — all while keepi
記事を読む →The Bias–Variance Tradeoff and Model Validation
A model that fits the training data perfectly is often useless on new data. This chapter explains why — the , the most important concept in all of predictive modeling — and gives the tools to measure generalization hones
記事を読む →Regularization: Ridge, Lasso, and Elastic Net
When predictors are many or correlated, ordinary least squares overfits: coefficients explode, variance soars, and predictions on new data suffer. fixes this by adding a penalty that discourages large coefficients — deli
記事を読む →Logistic Regression and Classification
What if the response is a category — spam or not, disease or healthy, click or no click? Fitting a line to 0/1 labels gives nonsensical probabilities outside [0,1]. fixes this elegantly: it runs a linear model through a
記事を読む →Generalized Linear Models
Linear regression assumes Gaussian, constant-variance noise; logistic regression handles binary outcomes. (GLMs) reveal these as two members of one family, unifying regression for continuous, binary, count, and positive
記事を読む →Nonlinear and Nonparametric Regression
When the relationship is genuinely curved and you do not know its form, you need methods that let the dictate the shape. This chapter surveys the flexible end of regression: polynomials and splines, local and kernel regr
記事を読む →Robust, Quantile, and Bayesian Regression
Ordinary least squares is optimal under Gaussian noise — but real data has outliers, heavy tails, and asymmetric costs, and sometimes we want full probability distributions over our answers rather than point estimates. T
記事を読む →Evaluation, Pitfalls, and Causality
A regression can be technically correct and practically misleading. This chapter is the practitioner's survival guide: how to measure performance with the right metric, the classic traps that fool even experts, and the b
記事を読む →Regression in Machine Learning, Deep Learning, and AI
This capstone ties the book together by arguing a single thesis: . Supervised learning is regression and classification at scale. A neural network is a flexible regression function. Its output layer is a linear, logistic
記事を読む →Preface: how to use this book
Regression is the oldest and most useful idea in data analysis: . Two centuries after Gauss and Legendre fit lines to astronomical data, regression is still the workhorse of statistics — and, in a deep sense, the concept
記事を読む →Preface: how to use this book
For most of the 1990s and 2000s, if you wanted the best off-the-shelf classifier, you reached for a Support Vector Machine. SVMs combined a beautiful geometric idea — separate the classes with the — with deep optimizatio
記事を読む →Formulas, Kernels, and Hyperparameter Reference
A compact reference for the formulas, kernels, and knobs used throughout the book.
記事を読む →What Are Support Vector Machines?
Imagine two groups of points on a page and a ruler you must lay down to separate them. Many positions work — but which is ? A Support Vector Machine answers: the line that leaves the between the groups. That single insti
記事を読む →The Maximal Margin Classifier
Now we make the widest-street idea precise. For data that be perfectly separated by a line, the maximal margin classifier (the hard-margin SVM) is the cleanest version of the story: a little geometry turns ``make the str
記事を読む →Soft Margins and Slack
Real data overlaps, contains noise, and is rarely perfectly separable. The — the model people actually use — relaxes the hard constraints by allowing a controlled budget of margin violations. This single change makes SVM
記事を読む →Hinge Loss and the Primal Problem
We ended the last chapter with a striking reformulation: the soft-margin SVM is just L_2 regularization plus a special loss. That loss is the , and viewing the SVM as ``minimize regularized hinge loss'' (the problem) dem
記事を読む →