Regression in Machine Learning, Deep Learning, and AI
This capstone ties the book together by arguing a single thesis: regression is not just one tool in machine learning — it is the conceptual skeleton of the entire field. Supervised learning is regression and classificati
阅读文章 →Regression — Preface: how to use this book
Regression is the oldest and most useful idea in data analysis: predict a number from other numbers, and understand the relationship between them. Two centuries after Gauss and Legendre fit lines to astronomical data, re
阅读文章 →Support Vector Machines — Preface: how to use this book
For most of the 1990s and 2000s, if you wanted the best off-the-shelf classifier, you reached for a Support Vector Machine. SVMs combined a beautiful geometric idea — separate the classes with the widest possible street
阅读文章 →Formulas, Kernels, and Hyperparameter Reference
A compact reference for the formulas, kernels, and knobs used throughout the book.
阅读文章 →What Are Support Vector Machines?
Imagine two groups of points on a page and a ruler you must lay down to separate them. Many positions work — but which is best? A Support Vector Machine answers: the line that leaves the widest empty street between the g
阅读文章 →The Maximal Margin Classifier
Now we make the widest-street idea precise. For data that can be perfectly separated by a line, the maximal margin classifier (the hard-margin SVM) is the cleanest version of the story: a little geometry turns “make the
阅读文章 →Soft Margins and Slack
Real data overlaps, contains noise, and is rarely perfectly separable. The soft-margin SVM — the model people actually use — relaxes the hard constraints by allowing a controlled budget of margin violations. This single
阅读文章 →Hinge Loss and the Primal Problem
We ended the last chapter with a striking reformulation: the soft-margin SVM is just L_2 regularization plus a special loss. That loss is the hinge loss, and viewing the SVM as “minimize regularized hinge loss” (the prim
阅读文章 →Lagrangian Duality and the KKT Conditions
The primal view trains SVMs by gradient descent, but it hides two treasures: why support vectors appear, and how the kernel trick becomes possible. Both emerge when we rewrite the SVM through Lagrangian duality. This is
阅读文章 →The Kernel Trick
Here is the idea that turned a linear classifier into one of the most powerful tools in machine learning. A linear SVM can only draw straight boundaries — useless for data shaped like rings or spirals. The kernel trick l
阅读文章 →Kernels in Depth
The kernel is where you inject knowledge about your data's structure. Choosing and tuning it is the single most consequential decision in applying an SVM. This chapter surveys the standard kernels, explains the crucial p
阅读文章 →Support Vector Regression
The maximum-margin idea is not limited to classification. Support Vector Regression (SVR) flips the logic: instead of keeping points outside a margin, it tries to keep them inside a tube around the prediction, ignoring s
阅读文章 →Solving SVMs: Optimization Algorithms
An SVM is only as useful as our ability to solve its optimization problem efficiently. The dual is a quadratic program with n variables and a dense n× n matrix — naively intractable for large data. This chapter explains
阅读文章 →Multiclass, Probabilities, and Practical SVMs
An SVM is natively a binary, non-probabilistic classifier. Real problems have many classes and often need probabilities. This chapter covers the multiclass schemes, how to extract calibrated probabilities, and the end-to
阅读文章 →Margins, VC Dimension, and Generalization
Why does maximizing the margin actually help on unseen data? The answer is one of the crown jewels of machine learning: statistical learning theory, developed largely by Vapnik alongside the SVM. This chapter explains VC
阅读文章 →SVMs versus the World: When to Use What
When should you reach for an SVM today? This chapter places SVMs honestly in the modern landscape — against logistic regression, tree ensembles, and deep networks — with a clear-eyed account of their strengths, weaknesse
阅读文章 →Support Vector Machines in Machine Learning, Deep Learning, and AI
This capstone gathers the threads. It is tempting to file SVMs under “what we used before deep learning,” but that misses the deeper story: the ideas SVMs crystallized — maximum margin, kernels, convex duality, generaliz
阅读文章 →Algorithm and Hyperparameter Reference
A compact reference for the formulas, algorithms, and knobs used throughout the book.
阅读文章 →What Are Tree-Based Methods?
A decision tree is the most human of machine-learning models: it is a flowchart of yes/no questions, exactly how a doctor triages a patient or a loan officer screens an application. This chapter introduces the core idea
阅读文章 →Decision Trees for Classification
To grow a classification tree we need a precise answer to one question: what makes a split good? The answer is purity — a good split produces child nodes whose samples are as close to a single class as possible. This cha
阅读文章 →Decision Trees for Regression
Trees predict numbers as easily as labels. A regression tree partitions the feature space the same way, but each leaf predicts a constant value — the average target in that region — and splits are chosen to reduce varian
阅读文章 →Growing, Splitting, and Pruning
A tree left to grow freely will keep splitting until every leaf is pure — memorizing the training set, noise and all. The art of a single good tree is controlling its size: how to grow it, when to stop, and how to cut it
阅读文章 →Strengths, Weaknesses, and the Bias–Variance of Trees
Before we multiply trees into forests and boosters, it pays to understand a single tree's character honestly: what it does brilliantly, where it fails, and — most importantly — the bias–variance behavior that motivates e
阅读文章 →Bagging and Random Forests
We now turn high-variance single trees into one of the most robust models in machine learning. The recipe has two ingredients: bagging (train many trees on bootstrap resamples and average them) and the random forest twis
阅读文章 →