Soft Margins and Slack
Real data overlaps, contains noise, and is rarely perfectly separable. The --- the model people actually use --- relaxes the hard constraints by allowing a controlled budget of margin violations. This single change makes
阅读文章 →Hinge Loss and the Primal Problem
We ended the last chapter with a striking reformulation: the soft-margin SVM is just L_2 regularization plus a special loss. That loss is the , and viewing the SVM as ``minimize regularized hinge loss'' (the problem) dem
阅读文章 →Lagrangian Duality and the KKT Conditions
The primal view trains SVMs by gradient descent, but it hides two treasures: support vectors appear, and the kernel trick becomes possible. Both emerge when we rewrite the SVM through . This is the most mathematical chap
阅读文章 →The Kernel Trick
Here is the idea that turned a linear classifier into one of the most powerful tools in machine learning. A linear SVM can only draw straight boundaries --- useless for data shaped like rings or spirals. The lets the ver
阅读文章 →Kernels in Depth
The kernel is where you inject knowledge about your data's structure. Choosing and tuning it is the single most consequential decision in applying an SVM. This chapter surveys the standard kernels, explains the crucial p
阅读文章 →Support Vector Regression
The maximum-margin idea is not limited to classification. flips the logic: instead of keeping points a margin, it tries to keep them a tube around the prediction, ignoring small errors entirely. The result is a flexible,
阅读文章 →Solving SVMs: Optimization Algorithms
An SVM is only as useful as our ability to solve its optimization problem efficiently. The dual is a quadratic program with n variables and a dense n n matrix --- naively intractable for large data. This chapter explains
阅读文章 →Multiclass, Probabilities, and Practical SVMs
An SVM is natively a , classifier. Real problems have many classes and often need probabilities. This chapter covers the multiclass schemes, how to extract calibrated probabilities, and the end-to-end craft --- scaling,
阅读文章 →Margins, VC Dimension, and Generalization
Why does maximizing the margin actually help on data? The answer is one of the crown jewels of machine learning: , developed largely by Vapnik alongside the SVM. This chapter explains VC dimension, the structural-risk-mi
阅读文章 →SVMs versus the World: When to Use What
When should you reach for an SVM today? This chapter places SVMs honestly in the modern landscape --- against logistic regression, tree ensembles, and deep networks --- with a clear-eyed account of their strengths, weakn
阅读文章 →Support Vector Machines in Machine Learning, Deep Learning, and AI
This capstone gathers the threads. It is tempting to file SVMs under ``what we used before deep learning,'' but that misses the deeper story: the ideas SVMs crystallized --- , , , --- are woven through modern machine lea
阅读文章 →Algorithm and Hyperparameter Reference
A compact reference for the formulas, algorithms, and knobs used throughout the book.
阅读文章 →What Are Tree-Based Methods?
A decision tree is the most human of machine-learning models: it is a flowchart of yes/no questions, exactly how a doctor triages a patient or a loan officer screens an application. This chapter introduces the core idea
阅读文章 →Decision Trees for Classification
To grow a classification tree we need a precise answer to one question: The answer is --- a good split produces child nodes whose samples are as close to a single class as possible. This chapter defines the two purity me
阅读文章 →Decision Trees for Regression
Trees predict numbers as easily as labels. A partitions the feature space the same way, but each leaf predicts a --- the average target in that region --- and splits are chosen to reduce rather than class impurity. The r
阅读文章 →Growing, Splitting, and Pruning
A tree left to grow freely will keep splitting until every leaf is pure --- memorizing the training set, noise and all. The art of a single good tree is : how to grow it, when to stop, and how to cut it back. This chapte
阅读文章 →Strengths, Weaknesses, and the Bias--Variance of Trees
Before we multiply trees into forests and boosters, it pays to understand a single tree's character honestly: what it does brilliantly, where it fails, and --- most importantly --- the bias--variance behavior that motiva
阅读文章 →Bagging and Random Forests
We now turn high-variance single trees into one of the most robust models in machine learning. The recipe has two ingredients: (train many trees on bootstrap resamples and average them) and the twist (also randomize the
阅读文章 →Boosting: AdaBoost and Gradient Boosting
Bagging builds trees to cut variance. builds them to cut bias: each new tree focuses on what the ensemble got wrong so far. This chapter develops boosting from AdaBoost (reweight the mistakes) to the more general and pow
阅读文章 →Modern Gradient Boosting: XGBoost, LightGBM, CatBoost
Friedman's gradient boosting is the idea; , , and are the engineering that made it dominate. Each adds algorithmic and systems innovations --- regularized objectives, second-order optimization, histogram binning, clever
阅读文章 →Feature Importance and Interpretability
A single tree is a readable flowchart, but a forest of hundreds or a thousand boosted trees is a black box. The good news: tree ensembles support some of the best interpretability tools in machine learning. This chapter
阅读文章 →Practical Tree Modeling
Knowing the algorithms is half the battle; the other half is the craft of applying them to messy real data. This chapter is a practitioner's checklist: handling categoricals and missing values, dealing with class imbalan
阅读文章 →Specialized and Advanced Trees
Beyond classification, regression, forests, and boosting lies a rich ecosystem of tree variants tuned for special jobs: detecting anomalies, ranking search results, modeling time-to-event, quantifying uncertainty, and sq
阅读文章 →Trees versus the World: When to Use What
When should you reach for a tree ensemble, and when for something else? This chapter places trees in the broader modeling landscape --- against linear models and against deep learning --- and explains the that make trees
阅读文章 →