An SVM is natively a binary, non-probabilistic classifier. Real problems have many classes and often need probabilities. This chapter covers the multiclass schemes, how to extract calibrated probabilities, and the end-to-end craft — scaling, tuning, handling imbalance, and building leak-free pipelines — that turns an SVM from a textbook idea into a dependable model.

From binary to multiclass

SVMs handle \(K>2\) classes by combining binary classifiers:

  • One-vs-Rest (OvR): train \(K\) classifiers, each “class \(k\) vs. all others”; predict the highest-scoring. Cheap (\(K\) models) but classes are imbalanced within each problem.

  • One-vs-One (OvO): train \(\binom{K}{2}\) classifiers, one per pair; predict by majority vote. More models but each is small and balanced — this is what LIBSVM/SVC uses.

For large \(K\), OvO’s \(O(K^2)\) count grows, but each model trains on only the two relevant classes, so total cost is often manageable.

Getting probabilities

The raw score \(\vw^\top\vx+b\) is a signed distance, not a probability. To get \(P(y\mid\vx)\), SVMs use Platt scaling: fit a logistic function \(\sigma(a\,f(\vx)+b)\) to the SVM scores on held-out data. scikit-learn does this when you pass probability=True — but be aware it runs an internal cross-validation (slower) and the resulting probabilities can be inconsistent with the decision_function ranking. For ranking/thresholding you often don’t need probabilities at all; use decision_function directly.

The practitioner’s checklist

A reliable SVM workflow:

  1. Standardize features (zero mean, unit variance) — mandatory, especially for RBF.

  2. Choose a kernel: linear for high-dim/sparse (text); RBF as the nonlinear default.

  3. Tune \(C\) (and \(\gamma\) for RBF) on a log grid by cross-validation; they interact, so search jointly.

  4. Handle imbalance with class_weight="balanced" and judge with the right metric (PR-AUC/F1, not accuracy).

  5. Wrap everything in a Pipeline so scaling is fit inside each CV fold (no leakage).

  6. Mind scale: kernel SVC for up to \(\sim10^4\)–\(10^5\) rows; LinearSVC/SGD beyond.

One-class SVM for anomaly detection

A one-class SVM learns the support of “normal” data: it finds a boundary (in kernel feature space) enclosing most training points, flagging anything outside as an anomaly. With no labels needed, it is a classic tool for novelty and outlier detection (fraud, fault monitoring, intrusion). The parameter \(\nu\) sets the expected fraction of outliers. It reappears in Chapter [ch:svmmlai] as the surprising mathematical model for self-supervised contrastive learning.

Production gotchas. (1) Scaling leakage: fitting the scaler on all data before splitting leaks test statistics — always scale inside the pipeline/folds. (2) probability=True surprises: it retrains with CV and can disagree with decision_function; only enable it if you truly need probabilities. (3) Imbalance ignored: a 99/1 problem scores 99% by predicting the majority — set class_weight and use PR-AUC. (4) Default gamma left untuned. (5) Kernel SVM on too-large data, where training silently takes hours — subsample or go linear.

The practical scaffolding here — standardize, pipeline, grid/Bayesian search, calibrate, threshold, judge with the right metric — is identical to MLOps for any model, deep learning included, and uses the same tooling (scikit-learn pipelines, Optuna). Platt scaling, invented to calibrate SVMs, is now applied to neural networks too (alongside temperature scaling) to fix their overconfidence. And the one-class SVM is not just a legacy tool: it is the conceptual template for modern self-supervised representation learning (Chapter [ch:svmmlai]).

from sklearn.svm import SVC
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import GridSearchCV
from sklearn.datasets import load_wine
X, y = load_wine(return_X_y=True)                  # 3 classes
pipe = make_pipeline(StandardScaler(),
                     SVC(kernel="rbf", class_weight="balanced",
                         decision_function_shape="ovo"))
grid = {"svc__C": [0.1, 1, 10], "svc__gamma": ["scale", 0.01, 0.1]}
gs = GridSearchCV(pipe, grid, cv=5).fit(X, y)
print("best:", gs.best_params_, " cv acc:", round(gs.best_score_, 3))

Chapter summary

  • Multiclass via One-vs-Rest (\(K\) models) or One-vs-One (\(\binom{K}{2}\) models, used by SVC).

  • SVMs aren’t probabilistic; get \(P(y\mid\vx)\) via Platt scaling (probability=True) — or use decision_function for ranking.

  • Always standardize, pipeline to avoid leakage, tune \(C,\gamma\) jointly, and handle imbalance with class_weight.

  • One-class SVM (\(\nu\)) detects anomalies without labels — and models contrastive learning later.

  • Use kernel SVC up to \(\sim10^5\) rows; switch to LinearSVC/SGD beyond.