A compact reference for the notation, distributions, and identities used throughout the book.
Notation
| Symbol | Meaning |
|---|---|
| \(\Omega,\ A,\ A^c\) | sample space, event, complement |
| \(P(A),\ P(A\given B)\) | probability, conditional probability |
| \(X,\ p(x),\ f(x),\ F(x)\) | random variable, PMF, PDF, CDF |
| \(\E[X],\ \Var[X],\ \sigma\) | expectation, variance, standard deviation |
| \(\Cov(X,Y),\ \Corr(X,Y)\) | covariance, correlation |
| \(\boldsymbol\mu,\ \boldsymbol\Sigma\) | mean vector, covariance matrix |
| \(\Normal(\mu,\sigma^2)\) | normal (Gaussian) distribution |
| \(\hat\theta,\ L(\theta)\) | estimator, likelihood |
| \(p(\theta\given\Ldata)\) | posterior distribution |
| \(\Hentropy(p),\ \KL(p\|q),\ \MI(X;Y)\) | entropy, KL divergence, mutual information |
| \(X\indep Y\) | independence |
Core identities
\[P(A\cup B)=P(A)+P(B)-P(A\cap B),\qquad P(A\given B)=\frac{P(A\cap B)}{P(B)}.\] \[\textbf{Bayes: }\ P(H\given E)=\frac{P(E\given H)P(H)}{P(E)},\qquad \textbf{Total prob.: }\ P(E)=\sum_i P(E\given H_i)P(H_i).\] \[\E[aX+bY]=a\E[X]+b\E[Y],\qquad \Var[X]=\E[X^2]-\E[X]^2,\] \[\Var[aX+b]=a^2\Var[X],\qquad \Cov(X,Y)=\E[XY]-\E[X]\E[Y].\] \[\textbf{LLN: }\ \bar X_n\to\mu,\qquad \textbf{CLT: }\ \frac{\bar X_n-\mu}{\sigma/\sqrt n}\to\Normal(0,1).\]
Distribution reference
| Distribution | PMF/PDF | Mean | Variance | Use |
|---|---|---|---|---|
| \(\Bern(p)\) | \(p^x(1-p)^{1-x}\) | \(p\) | \(p(1-p)\) | binary label |
| \(\Binom(n,p)\) | \(\binom nk p^k(1-p)^{n-k}\) | \(np\) | \(np(1-p)\) | success counts |
| \(\Pois(\lambda)\) | \(\frac{\lambda^k e^{-\lambda}}{k!}\) | \(\lambda\) | \(\lambda\) | rare events |
| \(\Unif(a,b)\) | \(\frac{1}{b-a}\) | \(\frac{a+b}{2}\) | \(\frac{(b-a)^2}{12}\) | RNG, priors |
| \(\Expo(\lambda)\) | \(\lambda e^{-\lambda x}\) | \(1/\lambda\) | \(1/\lambda^2\) | waiting time |
| \(\Normal(\mu,\sigma^2)\) | \(\frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{(x-\mu)^2}{2\sigma^2}}\) | \(\mu\) | \(\sigma^2\) | noise, CLT |
| \(\Betad(\alpha,\beta)\) | \(\propto x^{\alpha-1}(1-x)^{\beta-1}\) | \(\frac{\alpha}{\alpha+\beta}\) | — | rate prior |
Inference & information cheat-sheet
\[\hat\theta_{\text{MLE}}=\argmax_\theta\sum_i\log p(x_i\given\theta),\qquad \hat\theta_{\text{MAP}}=\argmax_\theta\Big[\sum_i\log p(x_i\given\theta)+\log p(\theta)\Big].\] \[\text{L2 / weight decay}\ \equiv\ \text{Gaussian prior},\qquad \text{L1 / lasso}\ \equiv\ \text{Laplace prior}.\] \[\MSE(\hat\theta)=\Bias(\hat\theta)^2+\Var(\hat\theta),\qquad \text{95\% CI (mean): }\ \bar X\pm 1.96\,\frac{s}{\sqrt n}.\] \[\Hentropy(p)=-\!\sum p\log p,\quad \Hentropy(p,q)=-\!\sum p\log q,\quad \KL(p\|q)=\Hentropy(p,q)-\Hentropy(p)\ge0.\] \[\text{minimize cross-entropy}\ \Longleftrightarrow\ \text{maximize likelihood}\ \Longleftrightarrow\ \text{minimize }\KL(p\,\|\,q).\]
Minimal NumPy / SciPy reference
| Task | Call |
|---|---|
| sample / pdf / cdf | |
| mean, var, std | , , |
| covariance / correlation | , |
| multivariate normal | |
| \(t\)-test / \(\chi^2\) test | , |
| fit by MLE (optimize NLL) | |
| bootstrap CI | resample with |
End of book. Probability says what to expect; statistics says what to believe.