Just as differentiation extended to many variables, so does integration. Multiple integrals accumulate a quantity over a region of two, three, or more dimensions — the natural setting for probabilities over several variables. The change-of-variables formula introduces the Jacobian determinant, which reappears in normalizing flows; and a brief tour of vector calculus names the operators (gradient, divergence, curl) that close out a standard course.
Double and triple integrals
The double integral of \(f(x,y)\) over a region \(R\) is the limit of sums over a grid of small rectangles: \[\iint_R f(x,y)\,\dd A = \lim \sum_{i,j} f(x_i,y_j)\,\Delta x\,\Delta y.\] It measures the (signed) volume under the surface \(z=f(x,y)\) over \(R\).
By Fubini’s theorem, a double integral over a rectangle can be evaluated as an iterated integral — integrate over \(x\) first (holding \(y\) fixed), then over \(y\): \[\iint_R f\,\dd A = \int_c^d\!\!\left(\int_a^b f(x,y)\,\dd x\right)\dd y.\] Triple integrals \(\iiint_E f\,\dd V\) extend this to solid regions, and the pattern continues to any dimension — which is exactly what a probability over many variables requires.
\(\displaystyle\int_0^1\!\!\int_0^2 (x+y)\,\dd x\,\dd y=\int_0^1\Big[\tfrac{x^2}{2}+xy\Big]_0^2\dd y=\int_0^1(2+2y)\,\dd y=[2y+y^2]_0^1=3.\)
Change of variables and the Jacobian
Substitution in many variables rescales the volume element by the absolute value of a determinant.
Under an invertible map \(\vx=\mathbf{T}(\vu)\), \[\iint_R f(\vx)\,\dd A_{\vx} = \iint_{S} f(\mathbf{T}(\vu))\,\big|\det \mathbf{J}_{\mathbf{T}}(\vu)\big|\,\dd A_{\vu},\] where \(\mathbf{J}_{\mathbf{T}}\) is the Jacobian matrix of the transformation. The factor \(|\det\mathbf{J}|\) is the local volume-scaling of the map.
The Jacobian determinant answers “when I bend space with \(\mathbf{T}\), by how much does a tiny volume element stretch or shrink?” Polar, cylindrical, and spherical coordinates are the familiar special cases (e.g. \(\dd A=r\,\dd r\,\dd\theta\), where \(r\) is the Jacobian factor). The very same formula governs how a probability density transforms when you change variables — the foundation of normalizing-flow generative models.
The non-elementary \(\int_{-\infty}^{\infty}e^{-x^2}\dd x\) is found by squaring it into a double integral and switching to polar coordinates (whose Jacobian is \(r\)): \[\left(\int_{-\infty}^{\infty}e^{-x^2}\dd x\right)^2=\iint_{\R^2}e^{-(x^2+y^2)}\dd A=\int_0^{2\pi}\!\!\int_0^\infty e^{-r^2}r\,\dd r\,\dd\theta=\pi,\] so the integral is \(\sqrt{\pi}\). This is the normalizing constant behind the Gaussian distribution.
A glimpse of vector calculus
Vector calculus studies vector fields \(\mathbf{F}:\R^n\to\R^n\) (think: a velocity or force at every point) with three differential operators:
Gradient \(\grad f\): turns a scalar field into the vector field of steepest ascent (Chapter [ch:multivariable]).
Divergence \(\divg\mathbf{F}=\sum_i\pd{F_i}{x_i}\): the net “outflow” per unit volume — a source (\(>0\)) or sink (\(<0\)).
Curl \(\curl\mathbf{F}\): the local rotation of the field.
The capstone theorems of a multivariable course — Green’s, Stokes’, and the Divergence theorem — all say the same profound thing: the total of a derivative over a region equals a boundary measurement. They are the higher-dimensional siblings of the Fundamental Theorem of Calculus (\(\int_a^b F'=F(b)-F(a)\) is exactly “interior derivative = boundary values”). You will not need their full machinery for most of ML, but the divergence shows up in continuous-time generative models, where the change in log-density along a flow is governed by \(\divg\) of the velocity field.
Multivariable integration is how ML reasons about joint distributions. A density over many variables normalizes via a high-dimensional integral; marginalizing a variable integrates it out; and a Bayesian posterior’s evidence \(p(\text{data})=\int p(\text{data}\mid\vtheta)p(\vtheta)\,\dd\vtheta\) is a multidimensional integral — usually intractable, hence variational inference and MCMC. The change-of-variables / Jacobian-determinant formula is the mathematical core of normalizing flows: transform a simple density through an invertible network and track \(\log|\det\mathbf{J}|\). The divergence term appears in continuous normalizing flows and the probability-flow ODE of diffusion models (Chapter [ch:calcmlai]).
import numpy as np
from scipy import integrate
# iterated integral of (x + y) over [0,2] x [0,1] -> 3
val, _ = integrate.dblquad(lambda y, x: x + y, 0, 2, lambda x: 0, lambda x: 1)
print(val) # 3.0
# Gaussian integral over R: sqrt(pi)
print(integrate.quad(lambda x: np.exp(-x**2), -np.inf, np.inf)[0], np.sqrt(np.pi))Chapter summary
Multiple integrals accumulate over 2-D/3-D/\(n\)-D regions; evaluate by iterated (Fubini) integration.
Change of variables multiplies by \(|\det \mathbf{J}|\), the local volume-scaling — the Jacobian determinant.
Polar coordinates give the Gaussian integral \(\int e^{-x^2}=\sqrt\pi\); the same formula transforms probability densities.
Vector calculus (gradient, divergence, curl) and the Green/Stokes/Divergence theorems generalize the FTC.
Joint densities, marginalization, Bayesian evidence, and normalizing flows are all multidimensional integration.