Lᵖ Spaces and Jensen’s Inequality

2B.1 Metric Spaces noticed that one space of functions can carry several natural distances: the largest difference, the area between the graphs. Measure theory now gives a whole family of them, one for each exponent $p \geq 1$:

Small $p$ cares about averages and ignores rare large values; large $p$ cares about the largest values and ignores averages. The spaces $L^p$ of functions with $\|f\|_p < \infty$ are the settings for nearly all of the PDE in Courses 4 and 6, and the inequalities proved in this chapter, Hölder's and Minkowski's, are used on almost every page from here on.

The chapter ends with Jensen's inequality: for a convex function, the function of the average is at most the average of the function. It is the source of the inequalities between means, and, applied to the logarithm, it proves that relative entropy is never negative. That is the first link of the entropy thread E, which leads through the log-Sobolev inequality (6A.10 Entropy, Information and Diffusion) to Perelman's $\mathcal{W}$-entropy (12A.3 The 𝓦-Entropy).

By the end of this chapter you will be able to:

• compute $L^p$ norms, and decide which $L^p$ spaces a given function belongs to;
• prove Young's, Hölder's and Minkowski's inequalities, and use Hölder to compare $L^p$ norms;
• prove that $L^p$ is complete, and say which functions are dense in it;
• prove Jensen's inequality and derive from it the AM–GM inequality and Gibbs' inequality;
• choose the right norm for a given practical question.

7.1Which norm is your specification?

7.2$L^p$ spaces

Let $(X, \mu)$ be a measure space and $0 < p < \infty$. For measurable $f$, $\|f\|_p = \big(\int|f|^p\,d\mu\big)^{1/p}$. The essential supremum $\|f\|_\infty$ is the least $M$ with $|f| \leq M$ almost everywhere: the supremum, ignoring null sets.

The identification is what makes $\|f\|_p = 0$ imply $f = 0$ (3A.3 The Lebesgue Integral). The triangle inequality is Minkowski's inequality below; with it, $d(f, g) = \|f - g\|_p$ is a metric for $1 \leq p \leq \infty$. (For $p < 1$ the triangle inequality fails, which is why $p \geq 1$.) The unit balls of the norms $\big(|x_1|^p + |x_2|^p\big)^{1/p}$ on $\mathbb{R}^2$, the $\ell^p$ norms on two points, show the family interpolating between the diamond of $\ell^1$ and the square of $\ell^\infty$ (Figure 7.2, extending 2B.1 Metric Spaces).

Which $L^p$ contains which? On $\mathbb{R}$, neither inclusion holds in general: $|x|^{-a}$ is in $L^p$ near $0$ when $ap < 1$ (small $p$ forgives singularities) and in $L^p$ near infinity when $ap > 1$ (large $p$ forgives slow decay). So $|x|^{-1/2}1_{(0,1)}$ is in $L^1$ but not $L^2$, and $|x|^{-1}1_{(1,\infty)}$ is in $L^2$ but not $L^1$. On a space of finite measure, large $p$ is more demanding: $L^p \subseteq L^q$ for $q \leq p$ (Exercise 7.9). On $\ell^p$ it is the other way round: $\ell^q \subseteq \ell^p$ for $q \leq p$, since small terms raised to higher powers get smaller.

7.3Young, Hölder and Minkowski

Call $p, q \in [1, \infty]$ conjugate exponents if $\frac1p + \frac1q = 1$: $(2, 2)$, $(1, \infty)$, $(3, \frac32)$.

Normalising to reduce to the case of norm $1$, then using a pointwise inequality, is a standard move: the inequality is homogeneous, so the normalised case is the general case. For $p = q = 2$ Hölder is the Cauchy–Schwarz inequality $|\langle f, g\rangle| \leq \|f\|_2\|g\|_2$.

Two consequences of Hölder are used constantly:

• Finite measure spaces. If $\mu(X) < \infty$ and $q \leq p$, then $\|f\|_q \leq \mu(X)^{\frac1q - \frac1p}\|f\|_p$ (Exercise 7.9). On a probability space, $\|f\|_p$ is increasing in $p$.
• Interpolation. If $p < r < q$ and $\frac1r = \frac\theta p + \frac{1 - \theta}q$, then $\|f\|_r \leq \|f\|_p^\theta\,\|f\|_q^{1-\theta}$ (Exercise 7.10). Control at two exponents gives control at every exponent in between. This log-convexity is the simplest case of interpolation, a theme that returns in the Sobolev inequalities (4A.10 Sobolev Embeddings and Critical Exponents) and in Gagliardo–Nirenberg-type estimates for PDE.

7.4Completeness and density

Two ideas from earlier chapters made this short: a fast subsequence converges almost everywhere, and Fatou passes the Cauchy bound to the limit. Completeness is the property the Riemann integral lacked (2B.2 Completeness and Contraction, 3A.3 The Lebesgue Integral); now every $L^p$ has it. A complete normed vector space is a Banach space, the subject of Book 4A.

Density. For $1 \leq p < \infty$, the following are dense in $L^p(\mathbb{R}^d)$: simple functions with finite-measure support; finite combinations of indicators of boxes; continuous functions with compact support; and (after 3A.8 Convolution and Mollifiers) smooth functions with compact support. So any statement that is continuous in the $L^p$ norm can be proved for nice functions and extended by approximation. This fails for $p = \infty$: the uniform limit of continuous functions is continuous, so $1_{[0,\infty)}$ is at distance at least $\tfrac12$ from every continuous function in $L^\infty$.

7.4.1The Hilbert space $L^2$

$L^2$ alone among the $L^p$ spaces has an inner product, $\langle f, g\rangle = \int f\bar g\,d\mu$, with $\|f\|_2^2 = \langle f, f\rangle$. A complete inner product space is a Hilbert space (4A.4 Hilbert Spaces and Lax–Milgram), and $L^2$ is the model one. Fourier series find their natural home here: the characters $e_k(x) = e^{2\pi ikx}$ form an orthonormal basis of $L^2(\mathbb{R}/\mathbb{Z})$, and the map $f \mapsto (\hat f(k))_{k\in\mathbb{Z}}$ is a bijection between $L^2(\mathbb{R}/\mathbb{Z})$ and $\ell^2(\mathbb{Z})$ preserving inner products. In 2B.7 Fourier Series and the First Heat Equation only continuous $f$ were allowed and the map was not onto $\ell^2$; with Lebesgue's integral, every square-summable sequence of coefficients is the Fourier series of some function. This was the content of the Riesz–Fischer theorem as first proved in 1907.

Duality is stated now and proved in 4A.3 Hahn–Banach and Duality: for $1 \leq p < \infty$ and conjugate $q$, every continuous linear functional on $L^p$ is $f \mapsto \int fg$ for a unique $g \in L^q$, and Hölder's inequality is sharp: $\|f\|_p = \sup\big\{\big|\int fg\big| : \|g\|_q \leq 1\big\}$.

7.5Jensen's inequality

The pattern "function of average versus average of function" is everywhere.

• AM–GM. With $\phi = \exp$ and $f$ taking values $\log a_1, \ldots, \log a_n$ with equal probability: $(a_1\cdots a_n)^{1/n} \leq \frac{a_1 + \cdots + a_n}{n}$.
• Power means. With $\phi(t) = t^{q/p}$ for $q \geq p$: $\|f\|_p \leq \|f\|_q$ on a probability space.
• Variance. With $\phi(t) = t^2$: $\big(\int f\big)^2 \leq \int f^2$, that is, variance is non-negative.

7.5.1Entropy is never negative

Let $p$ and $q$ be probability densities on $\mathbb{R}^d$ (or any measure space), with $q > 0$ where $p > 0$. Their relative entropy (Kullback–Leibler divergence) is

Relative entropy measures how far one probability distribution is from another, in units of information. It is not a metric (it is not symmetric), but it is non-negative and vanishes only for equal distributions, which is all that is needed to use it as a measure of distance from equilibrium.

7.6History

Leonard James Rogers (1888) and Otto Hölder (1889) proved the inequality now named after Hölder; Hermann Minkowski's appeared in his Geometrie der Zahlen (1896). W. H. Young's inequality for products dates from 1912. Frigyes Riesz introduced the $L^p$ spaces in 1910. Riesz and Ernst Fischer independently proved the completeness of $L^2$ in 1907. Johan Jensen published his inequality for convex functions in 1906. J. Willard Gibbs stated the inequality now named after him in his work on statistical mechanics (1902), and Solomon Kullback and Richard Leibler introduced relative entropy as a measure of information in 1951.

7.7Exercises

Solution

(a) $p > 1$, including $\infty$. (b) $p < 3$. (c) Near $0$: $|x|^{-p/2}|\log|x||^{-p}$ is integrable iff $p < 2$, or $p = 2$ (the log helps: $\int\frac{dx}{x\log^2x} < \infty$). Near $\infty$: integrable iff $p > 2$, or $p = 2$. So exactly $p = 2$. (d) All $p$.

Solution

(a) $\frac1T\int_0^T\sin^2 = \frac12$. (b) Square: $V^2 = V_0^2$ always. Triangle: by symmetry, the mean of $(V_0 s)^2$ for $s$ uniform on $[-1, 1]$, which is $V_0^2/3$. (c) For the square wave, the $k$-th odd harmonic has amplitude $\frac{4V_0}{\pi k}$ and mean square $\frac{8V_0^2}{\pi^2k^2}$; summing over odd $k$ gives $\frac{8V_0^2}{\pi^2}\cdot\frac{\pi^2}{8} = V_0^2$.

Solution

$\log\frac pq = -\log\sigma - \frac{x^2}{2\sigma^2} + \frac{x^2}{2}$. Integrating against $p$: $-\log\sigma - \frac12 + \frac{\sigma^2}2$. The function $s - 1 - \log s$ (with $s = \sigma^2$) is convex with minimum $0$ at $s = 1$.