Measures, Probability and Weights

Everything in 3A.2 Lebesgue Measure and 3A.3 The Lebesgue Integral used only a few properties of Lebesgue measure: it assigns sizes to the sets of a σ-algebra, and it is countably additive. Translation invariance and boxes were needed to build it, but not to integrate against it. So the whole theory works for any countably additive measure on any set. This chapter makes that step, and it pays off at once: sums become integrals, probability becomes measure theory, and changing one measure into another becomes multiplication by a density.

The last idea is the one the guidebook needs most. Perelman never integrates against plain volume. He integrates against the weighted measure $(4\pi\tau)^{-n/2}e^{-f}\,dV$, a probability measure written with its density in exponential form, exactly as physicists write the Boltzmann distribution $e^{-E/kT}$. The reasons are the same in both cases: energy and entropy live in the exponent, and normalising the total mass to $1$ makes the measure a probability. This chapter sets up that language.

By the end of this chapter you will be able to:

• define σ-algebras, measures and integrals on an abstract measure space, and recognise sums as integrals against counting measure;
• compute pushforward measures and use the change-of-variables formula for them;
• work with measures given by densities, state the Radon–Nikodym theorem, and recognise the Dirac mass as a limit of densities;
• translate between probability and measure theory: random variables, laws, expectation;
• reweight a measure by $e^{-f}$, normalise it, and explain why both statistical mechanics and Perelman write densities this way.

4.1Simulating rare events

4.2Measure spaces

The examples that matter:

• Lebesgue measure on $\mathbb{R}^d$ with the Lebesgue measurable sets (3A.2 Lebesgue Measure), or its restriction to the Borel sets.
• Counting measure on any set: $\#(E)$ is the number of elements of $E$ (possibly $\infty$), with every subset measurable.
• The Dirac mass at a point $x_0$: $\delta_{x_0}(E) = 1$ if $x_0 \in E$, and $0$ otherwise. All the mass at one point.
• Restrictions and sums. If $\mu$ is a measure and $A$ is measurable, $\mu\llcorner A(E) = \mu(E \cap A)$ is a measure; positive combinations $\sum_ic_i\mu_i$ of measures are measures.
• Riemannian volume. On a Riemannian manifold, the volume measure $dV$ (9A.1 Riemannian Metrics and Model Spaces), built from Lebesgue measure in coordinate charts.

A function $f : X \to [0, +\infty]$ is measurable if $\{f > t\} \in \mathcal{B}$ for all $t$. The integral $\int f\,d\mu$ is defined exactly as in 3A.3 The Lebesgue Integral: by simple functions, then by supremum, then by splitting into positive and negative parts. The monotone convergence theorem, Fatou's lemma and the dominated convergence theorem hold, with the same proofs; Tao states them in this generality in §1.4.5. From now on "integrable" means $\int|f|\,d\mu < \infty$, and $L^1(\mu)$ is the space of such functions (modulo equality $\mu$-almost everywhere).

4.2.1Sums are integrals

Against counting measure on $\mathbb{N}$, a function is a sequence $a_n$, and

So every theorem about integrals is also a theorem about series. Monotone convergence says that sums of non-negative terms can be computed by increasing truncations. Dominated convergence gives Tannery's theorem: if $a_{n,k} \to a_k$ as $n \to \infty$ for each $k$, and $|a_{n,k}| \leq M_k$ with $\sum_kM_k < \infty$, then $\sum_ka_{n,k} \to \sum_ka_k$. (For example, $\big(1 + \frac xn\big)^n = \sum_k\binom nk\frac{x^k}{n^k} \to \sum_k\frac{x^k}{k!} = e^x$, with $M_k = \frac{|x|^k}{k!}$.) And 3A.5 Product Measures and Change of Variables's Tonelli theorem, applied to counting measure twice, says double series of non-negative terms can be summed in either order (2A.8 Infinite Sets).

4.3Pushforward measures

A measurable map carries a measure from one space to another.

This proof pattern, indicators, then simple functions, then monotone limits, then differences, proves almost every identity about integrals. It is worth recognising on sight.

4.4Densities

The last identity is again proved by indicators, simple functions and monotone limits. Integrating against $\rho\,d\mu$ is the same as weighting the integrand by $\rho$. For probability densities on $\mathbb{R}^d$, $\int\rho\,dx = 1$ and $\nu(E)$ is the probability that a random point lands in $E$.

Which measures have densities? A necessary condition is that $\nu(E) = 0$ whenever $\mu(E) = 0$: a density can't put mass on a set that $\mu$ doesn't see. This is called absolute continuity, $\nu \ll \mu$. The Dirac mass $\delta_0$ is not absolutely continuous with respect to Lebesgue measure ($m(\{0\}) = 0$ but $\delta_0(\{0\}) = 1$), so it has no density, however tempting it is to write "$\delta(x)$". The converse is the theorem.

(σ-finite means the space is a countable union of sets of finite measure, as $\mathbb{R}^d$ is for Lebesgue measure.) The proof, in Tao's An Epsilon of Room I, §1.2, uses either the Hilbert space $L^2$ or a decomposition argument; we use only the statement. In importance sampling, the weight $p/q$ is $\frac{dP}{dQ}$, and the identity that makes the method unbiased is $\int g\,dP = \int g\,\frac{dP}{dQ}\,dQ$.

4.4.1The Dirac mass as a limit of densities

The Dirac mass has no density, but it is a limit of densities in a precise sense.

The rescaled family $\rho_\varepsilon$ is the approximate identity of 2B.5 Uniform Convergence and Arzelà–Ascoli and 2B.7 Fourier Series and the First Heat Equation, now seen as a family of measures converging to a point mass. This kind of convergence, testing against continuous functions, is called weak convergence of measures. The heat kernel at time $t \to 0$ converges to $\delta_0$ in exactly this sense (6A.3 The Heat Equation on ℝⁿ), which is what it means for the heat equation to start from given initial data. In 12A.6 Pseudolocality, Perelman's conjugate heat kernel starts as a Dirac mass at a point and spreads backwards in time.

4.5Probability as measure theory

Kolmogorov's 1933 axioms identify probability with measure theory of total mass $1$.

Probability	Measure theory
sample space $\Omega$, events, probability $\mathbb{P}$	measure space with $\mathbb{P}(\Omega) = 1$
random variable $X$	measurable function $X : \Omega \to \mathbb{R}$
law (distribution) of $X$	pushforward $X_*\mathbb{P}$ on $\mathbb{R}$
density of $X$	Radon–Nikodym derivative $\frac{dX_*\mathbb{P}}{dm}$
expectation $\mathbb{E}[g(X)]$	$\int g\circ X\,d\mathbb{P} = \int g\,d(X_*\mathbb{P})$
almost surely	$\mathbb{P}$-almost everywhere
independence	product measure (3A.5 Product Measures and Change of Variables)

The pushforward formula, Proposition 4.3, is the statement that the expectation of $g(X)$ can be computed either on the sample space or on the real line using the law of $X$. Markov's inequality (3A.3 The Lebesgue Integral) becomes $\mathbb{P}(X \geq t) \leq \mathbb{E}[X]/t$ for $X \geq 0$, and applied to $(X - \mathbb{E}X)^2$ it becomes Chebyshev's inequality, from which the weak law of large numbers follows in a line (Exercise 4.11).

4.6Weighted measures $e^{-f}$

A positive density can always be written as $\rho = e^{-f}$, with $f = -\log\rho$. There are good reasons to do so.

4.7History

Johann Radon proved the density theorem for measures on $\mathbb{R}^n$ in 1913, and Otto Nikodym extended it to abstract measure spaces in 1930. Maurice Fréchet had already defined integrals on abstract sets in 1915. Andrei Kolmogorov's Grundbegriffe der Wahrscheinlichkeitsrechnung (1933) founded probability on measure theory. Ludwig Boltzmann introduced the exponential distribution of energies in the 1860s and 1870s, and J. Willard Gibbs systematised it as the canonical ensemble in his Elementary Principles in Statistical Mechanics (1902). Importance sampling appears in Herman Kahn and Andy Marshall's 1953 paper on reducing sample sizes in Monte Carlo computations.

4.8Exercises

Solution

(a) $4$. (b) $\sum n2^{-n} = 2$. (c) $\tfrac12 + \tfrac12(e - 1) = \tfrac e2$. None of them: each puts positive mass on a Lebesgue-null set of points.

Solution

(a) $\mathbb{P}(-\log U > t) = \mathbb{P}(U < e^{-t}) = e^{-t}$, so the density is $e^{-t}$ on $[0, \infty)$, the exponential distribution. (b) $\frac{1}{\pi\sqrt{1 - y^2}}$, the arcsine density. (c) A uniform random number pushed forward by $-\log$ has exactly the exponential law (inverse transform sampling).

Solution

Expanding the square, the cross terms vanish, leaving $\frac{1}{n^2}\cdot n\sigma^2$. For an indicator of an event of probability $p$, $\sigma^2 = p(1 - p)$, so the relative standard error is $\sqrt{(1-p)/(np)}$; setting it to $0.1$ gives $n \approx 100/p$.

Solution

$\nabla u = -\tfrac12\nabla f\,e^{-f/2}$, so $|\nabla u|^2 = \tfrac14|\nabla f|^2e^{-f}$. Integrate.