Regression in Machine Learning, Deep Learning, and AI
This capstone ties the book together by arguing a single thesis: . Supervised learning is regression and classification at scale. A neural network is a flexible regression function. Its output layer is a linear, logistic
閱讀文章 →Preface: how to use this book
Regression is the oldest and most useful idea in data analysis: . Two centuries after Gauss and Legendre fit lines to astronomical data, regression is still the workhorse of statistics — and, in a deep sense, the concept
閱讀文章 →Preface: how to use this book
For most of the 1990s and 2000s, if you wanted the best off-the-shelf classifier, you reached for a Support Vector Machine. SVMs combined a beautiful geometric idea — separate the classes with the — with deep optimizatio
閱讀文章 →Formulas, Kernels, and Hyperparameter Reference
A compact reference for the formulas, kernels, and knobs used throughout the book.
閱讀文章 →What Are Support Vector Machines?
Imagine two groups of points on a page and a ruler you must lay down to separate them. Many positions work — but which is ? A Support Vector Machine answers: the line that leaves the between the groups. That single insti
閱讀文章 →The Maximal Margin Classifier
Now we make the widest-street idea precise. For data that be perfectly separated by a line, the maximal margin classifier (the hard-margin SVM) is the cleanest version of the story: a little geometry turns ``make the str
閱讀文章 →Soft Margins and Slack
Real data overlaps, contains noise, and is rarely perfectly separable. The — the model people actually use — relaxes the hard constraints by allowing a controlled budget of margin violations. This single change makes SVM
閱讀文章 →Hinge Loss and the Primal Problem
We ended the last chapter with a striking reformulation: the soft-margin SVM is just L_2 regularization plus a special loss. That loss is the , and viewing the SVM as ``minimize regularized hinge loss'' (the problem) dem
閱讀文章 →Lagrangian Duality and the KKT Conditions
The primal view trains SVMs by gradient descent, but it hides two treasures: support vectors appear, and the kernel trick becomes possible. Both emerge when we rewrite the SVM through . This is the most mathematical chap
閱讀文章 →The Kernel Trick
Here is the idea that turned a linear classifier into one of the most powerful tools in machine learning. A linear SVM can only draw straight boundaries — useless for data shaped like rings or spirals. The lets the very
閱讀文章 →Kernels in Depth
The kernel is where you inject knowledge about your data's structure. Choosing and tuning it is the single most consequential decision in applying an SVM. This chapter surveys the standard kernels, explains the crucial p
閱讀文章 →Support Vector Regression
The maximum-margin idea is not limited to classification. flips the logic: instead of keeping points a margin, it tries to keep them a tube around the prediction, ignoring small errors entirely. The result is a flexible,
閱讀文章 →Solving SVMs: Optimization Algorithms
An SVM is only as useful as our ability to solve its optimization problem efficiently. The dual is a quadratic program with n variables and a dense n n matrix — naively intractable for large data. This chapter explains t
閱讀文章 →Multiclass, Probabilities, and Practical SVMs
An SVM is natively a , classifier. Real problems have many classes and often need probabilities. This chapter covers the multiclass schemes, how to extract calibrated probabilities, and the end-to-end craft — scaling, tu
閱讀文章 →Margins, VC Dimension, and Generalization
Why does maximizing the margin actually help on data? The answer is one of the crown jewels of machine learning: , developed largely by Vapnik alongside the SVM. This chapter explains VC dimension, the structural-risk-mi
閱讀文章 →SVMs versus the World: When to Use What
When should you reach for an SVM today? This chapter places SVMs honestly in the modern landscape — against logistic regression, tree ensembles, and deep networks — with a clear-eyed account of their strengths, weaknesse
閱讀文章 →Support Vector Machines in Machine Learning, Deep Learning, and AI
This capstone gathers the threads. It is tempting to file SVMs under ``what we used before deep learning,'' but that misses the deeper story: the ideas SVMs crystallized — , , , — are woven through modern machine learnin
閱讀文章 →Algorithm and Hyperparameter Reference
A compact reference for the formulas, algorithms, and knobs used throughout the book.
閱讀文章 →What Are Tree-Based Methods?
A decision tree is the most human of machine-learning models: it is a flowchart of yes/no questions, exactly how a doctor triages a patient or a loan officer screens an application. This chapter introduces the core idea
閱讀文章 →Decision Trees for Classification
To grow a classification tree we need a precise answer to one question: The answer is — a good split produces child nodes whose samples are as close to a single class as possible. This chapter defines the two purity meas
閱讀文章 →Decision Trees for Regression
Trees predict numbers as easily as labels. A partitions the feature space the same way, but each leaf predicts a — the average target in that region — and splits are chosen to reduce rather than class impurity. The resul
閱讀文章 →Growing, Splitting, and Pruning
A tree left to grow freely will keep splitting until every leaf is pure — memorizing the training set, noise and all. The art of a single good tree is : how to grow it, when to stop, and how to cut it back. This chapter
閱讀文章 →Strengths, Weaknesses, and the Bias–Variance of Trees
Before we multiply trees into forests and boosters, it pays to understand a single tree's character honestly: what it does brilliantly, where it fails, and — most importantly — the bias–variance behavior that motivates e
閱讀文章 →Bagging and Random Forests
We now turn high-variance single trees into one of the most robust models in machine learning. The recipe has two ingredients: (train many trees on bootstrap resamples and average them) and the twist (also randomize the
閱讀文章 →