A compact reference for the symbols, rules, and formulas used throughout the book.

Notation

Symbol Meaning
\(\displaystyle\lim_{x\to a}f(x)\) limit of \(f\) as \(x\to a\)
\(f'(x),\ \dydx{y}{x},\ Df\) derivative
\(f''(x),\ \dfrac{\dd^2y}{\dd x^2}\) second derivative (curvature)
\(\displaystyle\int f\,\dd x\) indefinite integral (antiderivative \(+C\))
\(\displaystyle\int_a^b f\,\dd x\) definite integral (signed area)
\(\pd{f}{x_i}\) partial derivative
\(\grad f\) gradient (vector of partials)
\(\mathbf{J},\ \mathbf{H}\) Jacobian, Hessian
\(\E[X],\ \Var[X]\) expectation, variance (integrals)
\(H(p),\ \KL(p\|q)\) entropy, KL divergence
\(\sum_{n} a_n\) infinite series

Differentiation rules

\[(cf)'=cf',\quad (f\pm g)'=f'\pm g',\quad (fg)'=f'g+fg',\quad \left(\frac fg\right)'=\frac{f'g-fg'}{g^2}.\] \[\textbf{Chain rule: }\ \ddx{x}f(g(x))=f'(g(x))\,g'(x).\]

Derivative & integral table

\(f\) \(f'\) \(f\) \(\int f\,\dd x\)
\(x^n\) \(nx^{n-1}\) \(x^n\,(n\neq-1)\) \(\frac{x^{n+1}}{n+1}+C\)
\(e^x\) \(e^x\) \(e^x\) \(e^x+C\)
\(\ln x\) \(1/x\) \(1/x\) \(\ln|x|+C\)
\(\sin x\) \(\cos x\) \(\sin x\) \(-\cos x+C\)
\(\cos x\) \(-\sin x\) \(\cos x\) \(\sin x+C\)
\(\tan x\) \(\sec^2 x\) \(\frac{1}{1+x^2}\) \(\arctan x+C\)
\(\sigma(x)\) \(\sigma(x)(1-\sigma(x))\) \(\tanh x\) —

Integration techniques

\[\textbf{Substitution: }\int f(g(x))g'(x)\,\dd x=\int f(u)\,\dd u,\qquad \textbf{By parts: }\int u\,\dd v=uv-\int v\,\dd u.\] \[\textbf{FTC: }\int_a^b f(x)\,\dd x = F(b)-F(a)\ \text{ where }F'=f;\qquad \ddx{x}\int_a^x f(t)\,\dd t=f(x).\]

Series

\[\sum_{n=0}^\infty r^n=\frac{1}{1-r}\ (|r|<1),\quad e^x=\sum_{n=0}^\infty\frac{x^n}{n!},\quad f(x)=\sum_{n=0}^\infty\frac{f^{(n)}(a)}{n!}(x-a)^n.\]

Multivariable & ML-facing formulas

\[\grad f=\Big[\pd{f}{x_1},\dots,\pd{f}{x_n}\Big]^\top,\qquad \mathbf{H}_{ij}=\frac{\partial^2 f}{\partial x_i\partial x_j},\qquad f(\vx+\Delta)\approx f+\grad f^\top\Delta+\tfrac12\Delta^\top\mathbf{H}\Delta.\] \[\textbf{Gradient descent: }\ \vtheta\leftarrow\vtheta-\eta\,\grad_{\vtheta}L,\qquad \textbf{Newton: }\ \vtheta\leftarrow\vtheta-\mathbf{H}^{-1}\grad_{\vtheta}L.\] \[\grad_{\vx}(\va^\top\vx)=\va,\quad \grad_{\vx}(\vx^\top\mathbf{A}\vx)=(\mathbf{A}+\mathbf{A}^\top)\vx,\quad \text{softmax+CE: }\ \pd{L}{z_i}=p_i-y_i.\] \[\E[g(X)]=\int g(x)p(x)\,\dd x,\qquad \KL(p\|q)=\int p\ln\frac pq\,\dd x,\] \[\text{ELBO}=\E_q[\ln p(x\mid z)]-\KL\big(q(z\mid x)\,\|\,p(z)\big).\]

Minimal NumPy / PyTorch / SciPy reference

Task Call
numeric derivative
symbolic derivative / integral ,
definite integral (numeric)
autodiff gradient ;
Jacobian / Hessian
gradient descent step
optimizers ,

End of book. Derivatives tell AI which way to move; integrals tell it what to optimize.