Proof of Basic Properties of the Entropy on the Wasserstein Space: Comparison with the Gaussian Relative Entropy, Lower Bound, Translation Invariance, Absolute Continuity, Closed Sublevel Sets and Lower Semicontinuity

Each result cited below is universally quantified over the data in its own statement. Throughout, $\phi$ is the function $s\mapsto s\log s$ of The Function $s\log s$: Continuity, Young's Inequality and Lower Bounds, with the Elementary Bounds for the Exponential and the Logarithm, densities are those of The Radon-Nikodym Theorem for a Finite Measure and a Sigma-Finite Measure, and Uniqueness of Densities, and for $z\in\mathbb{R}^{d}$ we put

Claim 1 (The Gaussian weight). For every $z\in\mathbb{R}^{d}$ one has $0<g_{1}(z)$ and $\log g_{1}(z)=\ell(z)$; the functions $\ell$ and $1/g_{1}$ are Borel.

Since $c_{1}$ is positive and $\exp$ takes positive values by claim 2 of Basic Properties of the Exponential Function, $g_{1}(z)=c_{1}\exp(-\tfrac12\lVert z\rVert^{2})$ is positive, and by the same claim $1/g_{1}(z)=c_{1}^{-1}\exp(\tfrac12\lVert z\rVert^{2})$. By the product rule and the inverse relation recorded in The Natural Logarithm, $\log g_{1}(z)=\log c_{1}+\log\exp(-\tfrac12\lVert z\rVert^{2})=\ell(z)$. The map $z\mapsto\lVert z\rVert^{2}$ is Borel by Pairs of Euclidean Points: Coordinate Projections, Pairings, the Product Measure on a Euclidean Space, Borel Norm Functions and Finite Sets §functions, so $\ell$ and $z\mapsto\tfrac12\lVert z\rVert^{2}$ are Borel by claims 1 and 2 of Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions. The function $\exp$ is smooth on $\mathbb{R}$ by claim 3 of Basic Properties of the Exponential Function, hence continuous by claim 3 of Euclidean Space is Open in Itself, and $C^k$ Maps are Continuous, hence Borel, and the composite $z\mapsto\exp(\tfrac12\lVert z\rVert^{2})$ is Borel, both by Probability Measures on Euclidean Space and Random Vectors: Standing Notation §borel-maps; so $1/g_{1}$ is Borel by claim 2 of Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions.

Claim 2 (Densities with respect to $\lambda_{d}$ and to $\gamma_{d}$). Let $\mu\in\mathcal{P}(\mathbb{R}^{d})$ and let $\rho,f:\mathbb{R}^{d}\to\mathbb{R}$ be Borel and nonnegative with $\rho=f\,g_{1}$. Then $\rho$ is a density of $\mu$ with respect to $\lambda_{d}$ if and only if $f$ is a density of $\mu$ with respect to $\gamma_{d}$.

For $B\in\mathcal{B}(\mathbb{R}^{d})$ the function $\mathbf{1}_{B}f$ is Borel and nonnegative by claims 1 and 3 of Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions. Since $\gamma_{d}$ is the measure with density $g_{1}$ with respect to $\lambda_{d}$, claim 3 of Image Measures, Measures with Densities, and Change of Variables gives

Hence $\mu(B)=\int\mathbf{1}_{B}\rho\,d\lambda_{d}$ for every Borel $B$ if and only if $\mu(B)=\int\mathbf{1}_{B}f\,d\gamma_{d}$ for every Borel $B$, and these are the two density conditions of The Radon-Nikodym Theorem for a Finite Measure and a Sigma-Finite Measure, and Uniqueness of Densities.

Claim 3 (A pointwise identity). Let $\rho,f$ be as in Claim 2. Then $g_{1}(z)\,\phi(f(z))=\phi(\rho(z))-\rho(z)\,\ell(z)$ for every $z\in\mathbb{R}^{d}$.

If $\rho(z)=0$, then $f(z)=0$ because $0<g_{1}(z)$ by Claim 1, and both sides are $0$ since $\phi(0)=0$. If $0<\rho(z)$, then $f(z)=\rho(z)\,g_{1}(z)^{-1}$ is positive, and by the product rule of The Natural Logarithm (with $\log1=\log\exp(0)=0$, so that $\log(g_{1}(z)^{-1})=-\log g_{1}(z)$) and Claim 1, $\log f(z)=\log\rho(z)-\ell(z)$. Multiplying by $g_{1}(z)f(z)=\rho(z)$ gives $g_{1}(z)\phi(f(z))=\rho(z)\log\rho(z)-\rho(z)\ell(z)=\phi(\rho(z))-\rho(z)\ell(z)$.

Claim 4 (The Gaussian correction term). Let $\mu\in\mathcal{P}_{2}(\mathbb{R}^{d})$ have a density $\rho$ with respect to $\lambda_{d}$. Then $\rho\,\ell$ is integrable with respect to $\lambda_{d}$ and $\int_{\mathbb{R}^{d}}\rho\,\ell\,d\lambda_{d}=\log c_{1}-\tfrac12M_{2}(\mu)$.

Since $\mu(A)=\int\mathbf{1}_{A}\rho\,d\lambda_{d}$ for every Borel $A$, $\mu$ is the measure with density $\rho$ with respect to $\lambda_{d}$ of claim 3 of Image Measures, Measures with Densities, and Change of Variables. By claims 4 and 5 of Properties of the Absolute Value in an Ordered Field, $|\ell(z)|\le|\log c_{1}|+\tfrac12\lVert z\rVert^{2}$, and by claim 1 of Linearity and Monotonicity of the Lebesgue Integral the integral of the right-hand side with respect to $\mu$ is $|\log c_{1}|+\tfrac12M_{2}(\mu)$, finite because $\mu\in\mathcal{P}_{2}(\mathbb{R}^{d})$ (The Second Moment of a Probability Measure on Euclidean Space and the Probability Measures with Finite Second Moment §moment, The Second Moment of a Probability Measure on Euclidean Space and the Probability Measures with Finite Second Moment §space) and $\mu(\mathbb{R}^{d})=1$. Hence the Borel function $\ell$ (Claim 1) is integrable with respect to $\mu$ by Integrable Function and the Lebesgue Integral, the constant function and $z\mapsto\lVert z\rVert^{2}$ being integrable likewise, and $\int\ell\,d\mu=\log c_{1}-\tfrac12M_{2}(\mu)$ by claim 2 of Linearity and Monotonicity of the Lebesgue Integral. By claim 3 of Image Measures, Measures with Densities, and Change of Variables, $\ell\rho$ is integrable with respect to $\lambda_{d}$ with the same integral.

Claim 5 (Comparison with the Gaussian; claim 1). Let $\mu\in\mathcal{P}_{2}(\mathbb{R}^{d})$.

Suppose first that $\mu$ has finite entropy, with a density $\rho$ with respect to $\lambda_{d}$ for which $\phi\circ\rho$ is integrable with respect to $\lambda_{d}$ (The Entropy of a Probability Measure on Euclidean Space §entropy). Put $f=\rho\cdot(1/g_{1})$, which is Borel and nonnegative by Claim 1 and claim 3 of Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions, and satisfies $\rho=f\,g_{1}$. By Claim 2, $f$ is a density of $\mu$ with respect to $\gamma_{d}$, and $\phi\circ f$ is Borel by The Function $s\log s$: Continuity, Young's Inequality and Lower Bounds, with the Elementary Bounds for the Exponential and the Logarithm §continuous. By Claim 3, $(\phi\circ f)\,g_{1}=\phi\circ\rho-\rho\,\ell$, which by Claim 4 and claim 2 of Linearity and Monotonicity of the Lebesgue Integral is integrable with respect to $\lambda_{d}$ with integral $\mathrm{Ent}(\mu)-\log c_{1}+\tfrac12M_{2}(\mu)$. By claim 3 of Image Measures, Measures with Densities, and Change of Variables, applied to the measure $\gamma_{d}$ with density $g_{1}$, $\phi\circ f$ is integrable with respect to $\gamma_{d}$ with the same integral. So $\mu$ has finite relative entropy with respect to $\gamma_{d}$ and, by Relative Entropy of Probability Measures §relative-entropy, $H(\mu\,|\,\gamma_{d})=\mathrm{Ent}(\mu)-\log c_{1}+\tfrac12M_{2}(\mu)$.

Suppose conversely that $\mu$ has finite relative entropy with respect to $\gamma_{d}$, with a density $f$ with respect to $\gamma_{d}$ for which $\phi\circ f$ is integrable with respect to $\gamma_{d}$. Put $\rho=f\,g_{1}$, Borel and nonnegative by claim 3 of Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions. By Claim 2, $\rho$ is a density of $\mu$ with respect to $\lambda_{d}$. By claim 3 of Image Measures, Measures with Densities, and Change of Variables, $(\phi\circ f)\,g_{1}$ is integrable with respect to $\lambda_{d}$ with integral $H(\mu\,|\,\gamma_{d})$; by Claims 3 and 4 and claim 2 of Linearity and Monotonicity of the Lebesgue Integral, $\phi\circ\rho=(\phi\circ f)\,g_{1}+\rho\,\ell$ is integrable with respect to $\lambda_{d}$ with integral $H(\mu\,|\,\gamma_{d})+\log c_{1}-\tfrac12M_{2}(\mu)$. So $\mu$ has finite entropy, and by The Entropy of a Probability Measure on Euclidean Space §entropy $\mathrm{Ent}(\mu)=H(\mu\,|\,\gamma_{d})+\log c_{1}-\tfrac12M_{2}(\mu)$.

In either case the displayed identity of the claim holds, the two quantities being independent of the chosen densities by The Entropy of a Probability Measure on Euclidean Space §entropy and Relative Entropy of Probability Measures §relative-entropy.

Claim 6 (Lower bound; claim 2). Let $\mu\in\mathcal{P}_{2}^{\mathrm{Ent}}(\mathbb{R}^{d})$. By Claim 5, $\mu$ has finite relative entropy with respect to $\gamma_{d}\in\mathcal{P}(\mathbb{R}^{d})$ and $\mathrm{Ent}(\mu)=H(\mu\,|\,\gamma_{d})+\log c_{1}-\tfrac12M_{2}(\mu)$. By Relative Entropy on Euclidean Space: the Gibbs Inequality, a Variational Criterion for Finite Relative Entropy, and Closed Sublevel Sets under Weak Convergence §gibbs, with $m=d$ and $\gamma=\gamma_{d}$, $0\le H(\mu\,|\,\gamma_{d})$, so $\log c_{1}-\tfrac12M_{2}(\mu)\le\mathrm{Ent}(\mu)$.

Claim 7 (Translation invariance; claim 3). Let $\mu\in\mathcal{P}_{2}^{\mathrm{Ent}}(\mathbb{R}^{d})$ with a density $\rho$ as in The Entropy of a Probability Measure on Euclidean Space §entropy, and let $a\in\mathbb{R}^{d}$, with $\tau_{a}(x)=x+a$ as in The Wasserstein Space and Its Lift to Square-Integrable Random Vectors: Standing Notation §constants. By Gradients of Functions with Bounded Derivatives Belong to the Tangent Space; Constants Are Tangent; the Score Identity; the Score Has Mean Zero; Translation of Tangent Fields §translation, $(\tau_{a})_{\#}\mu\in\mathcal{P}_{2}(\mathbb{R}^{d})$. Put $\tilde{\rho}(y)=\rho(y-a)$; it is Borel by claim 2 of Translation and Reflection Invariance of Lebesgue Measure on $\mathbb{R}^n$ (with $-a$ in place of $a$) and nonnegative. For $B\in\mathcal{B}(\mathbb{R}^{d})$, apply claim 2 of Translation and Reflection Invariance of Lebesgue Measure on $\mathbb{R}^n$ to the Borel function $F_{B}(y)=\mathbf{1}_{B}(y)\,\rho(y-a)$, for which $F_{B}(x+a)=\mathbf{1}_{B}(x+a)\,\rho(x)=\mathbf{1}_{\tau_{a}^{-1}(B)}(x)\,\rho(x)$; with Probability Measures on Euclidean Space and Random Vectors: Standing Notation §pushforward this gives

So $\tilde{\rho}$ is a density of $(\tau_{a})_{\#}\mu$ with respect to $\lambda_{d}$. Since $\phi\circ\tilde{\rho}$ is the function $y\mapsto(\phi\circ\rho)(y-a)$ and $\phi\circ\rho$ is Borel and integrable with respect to $\lambda_{d}$, claim 3 of Translation and Reflection Invariance of Lebesgue Measure on $\mathbb{R}^n$ (with $-a$ in place of $a$) shows that $\phi\circ\tilde{\rho}$ is integrable with the same integral. Hence $(\tau_{a})_{\#}\mu\in\mathcal{P}_{2}^{\mathrm{Ent}}(\mathbb{R}^{d})$ and $\mathrm{Ent}((\tau_{a})_{\#}\mu)=\mathrm{Ent}(\mu)$.

Claim 8 (Absolute continuity; claim 4). Let $\mu\in\mathcal{P}(\mathbb{R}^{d})$ have finite entropy, with a density $\rho$ with respect to $\lambda_{d}$, and let $B\in\mathcal{B}(\mathbb{R}^{d})$ satisfy $\lambda_{d}(B)=0$. The nonnegative Borel function $\mathbf{1}_{B}\rho$ vanishes off the null set $B$, so $\mu(B)=\int\mathbf{1}_{B}\rho\,d\lambda_{d}=0$ by The Lebesgue Integral and Null Sets: Almost-Everywhere Comparison, Markov's Inequality, and Dominated Convergence Almost Everywhere §null-integral. Thus $\mu$ is absolutely continuous in the sense of Absolutely Continuous Probability Measure on Euclidean Space §ac.

Claim 9 (Convergence of moments and of integrals of bounded Lipschitz functions). Let $\mu\in\mathcal{P}_{2}(\mathbb{R}^{d})$ and let $(\mu_{n})_{n\in\mathbb{N}}$ be a sequence in $\mathcal{P}_{2}(\mathbb{R}^{d})$ with $\lim_{n\to\infty}W_{2}(\mu_{n},\mu)=0$. Then $\lim_{n\to\infty}M_{2}(\mu_{n})=M_{2}(\mu)$, and $\lim_{n\to\infty}\int h\,d\mu_{n}=\int h\,d\mu$ for every bounded Lipschitz $h:\mathbb{R}^{d}\to\mathbb{R}$ in the sense of Relative Entropy on Euclidean Space: the Gibbs Inequality, a Variational Criterion for Finite Relative Entropy, and Closed Sublevel Sets under Weak Convergence.

By The Coordinate Fields of a Coupling, and the Second Moment as a Lipschitz Function of the Wasserstein Distance §lipschitz (with $m=d$), $\bigl|\sqrt{M_{2}(\mu_{n})}-\sqrt{M_{2}(\mu)}\bigr|\le W_{2}(\mu_{n},\mu)$ for every $n$, so $\sqrt{M_{2}(\mu_{n})}\to\sqrt{M_{2}(\mu)}$ by claim 3 of Order Properties of Limits of Real Sequences, and then $M_{2}(\mu_{n})=\sqrt{M_{2}(\mu_{n})}\sqrt{M_{2}(\mu_{n})}\to M_{2}(\mu)$ by claim 2 of Arithmetic of Limits of Real Sequences. Let $h$ be bounded and Lipschitz with a constant $L\ge0$; this is the notion of Lipschitz with constant $L$ in Couplings on Euclidean Space: Product Coupling, Swap, Finiteness of the Cost, Push-Forward Couplings, Modifying One Marginal, Quantisation, Gluing over a Finitely Supported Measure, and the Lipschitz Bound. For each $n$, Existence of an Optimal Coupling of Two Probability Measures with Finite Second Moment (with $m=d$) gives $\pi_{n}\in\Pi(\mu_{n},\mu)$ with $I(\pi_{n})=W_{2}(\mu_{n},\mu)^{2}<\infty$, and Couplings on Euclidean Space: Product Coupling, Swap, Finiteness of the Cost, Push-Forward Couplings, Modifying One Marginal, Quantisation, Gluing over a Finitely Supported Measure, and the Lipschitz Bound §lipschitz shows that $h$ is Borel and integrable with respect to $\mu_{n}$ and $\mu$, with

$W_{2}(\mu_{n},\mu)$ being nonnegative, as a value of a metric, and hence the nonnegative square root of $I(\pi_{n})$. Since $L\,W_{2}(\mu_{n},\mu)\to0$ by claim 3 of Arithmetic of Limits of Real Sequences, claim 3 of Order Properties of Limits of Real Sequences gives $\int h\,d\mu_{n}\to\int h\,d\mu$.

Claim 10 (Closed sublevel sets; claim 5). Let $c$, $\mu$ and $(\mu_{n})$ be as in that claim, and put $b_{n}=c-\log c_{1}+\tfrac12M_{2}(\mu_{n})$ and $b=c-\log c_{1}+\tfrac12M_{2}(\mu)$; by Claim 9 and claims 1 and 3 of Arithmetic of Limits of Real Sequences, $b_{n}\to b$. Let $h:\mathbb{R}^{d}\to\mathbb{R}$ be bounded Lipschitz; it is continuous by A Lipschitz Map is Uniformly Continuous, hence Borel by Probability Measures on Euclidean Space and Random Vectors: Standing Notation §borel-maps, and $\kappa_{h}=\log\int\exp\circ h\,d\gamma_{d}$ is a real number by Relative Entropy on Euclidean Space: the Gibbs Inequality, a Variational Criterion for Finite Relative Entropy, and Closed Sublevel Sets under Weak Convergence §functional (with $m=d$ and $\gamma=\gamma_{d}$), so that $\Lambda_{h}(\nu)=\int h\,d\nu-\kappa_{h}$. For each $n$, since $\mu_{n}\in\mathcal{P}_{2}^{\mathrm{Ent}}(\mathbb{R}^{d})$, Claim 5 shows that $\mu_{n}$ has finite relative entropy with respect to $\gamma_{d}$ with $H(\mu_{n}\,|\,\gamma_{d})=\mathrm{Ent}(\mu_{n})-\log c_{1}+\tfrac12M_{2}(\mu_{n})\le b_{n}$, and Relative Entropy on Euclidean Space: the Gibbs Inequality, a Variational Criterion for Finite Relative Entropy, and Closed Sublevel Sets under Weak Convergence §gibbs gives $\Lambda_{h}(\mu_{n})\le H(\mu_{n}\,|\,\gamma_{d})\le b_{n}$. By Claim 9 and claim 3 of Arithmetic of Limits of Real Sequences, $\Lambda_{h}(\mu_{n})\to\Lambda_{h}(\mu)$, so claim 1 of Order Properties of Limits of Real Sequences gives $\Lambda_{h}(\mu)\le b$. As $h$ was an arbitrary bounded Lipschitz function, Relative Entropy on Euclidean Space: the Gibbs Inequality, a Variational Criterion for Finite Relative Entropy, and Closed Sublevel Sets under Weak Convergence §criterion (with $\nu=\mu$ and the constant $b$) shows that $\mu$ has finite relative entropy with respect to $\gamma_{d}$ and $H(\mu\,|\,\gamma_{d})\le b$. Since $\mu\in\mathcal{P}_{2}(\mathbb{R}^{d})$, Claim 5 shows that $\mu$ has finite entropy, so $\mu\in\mathcal{P}_{2}^{\mathrm{Ent}}(\mathbb{R}^{d})$, and $\mathrm{Ent}(\mu)=H(\mu\,|\,\gamma_{d})+\log c_{1}-\tfrac12M_{2}(\mu)\le b+\log c_{1}-\tfrac12M_{2}(\mu)=c$.

Claim 11 (Lower semicontinuity; claim 6). Suppose, for a contradiction, that $\mathrm{Ent}$ is not lower semicontinuous at some $\mu\in\mathcal{P}_{2}^{\mathrm{Ent}}(\mathbb{R}^{d})$ relative to $\mathcal{P}_{2}^{\mathrm{Ent}}(\mathbb{R}^{d})$. Negating Lower Semicontinuous Function on a Subset of a Metric Space, and using that the order of $\mathbb{R}$ is total, there is $\varepsilon\in\mathbb{R}$ with $0<\varepsilon$ such that for every positive $\delta$ some $\nu\in\mathcal{P}_{2}^{\mathrm{Ent}}(\mathbb{R}^{d})$ satisfies $W_{2}(\mu,\nu)<\delta$ and $\mathrm{Ent}(\nu)\le\mathrm{Ent}(\mu)-\varepsilon$. For $n\in\mathbb{N}$ the number $(n+1)^{-1}$ exists and is positive by claims 6 and 7 of Elementary Order Arithmetic in an Ordered Field; choose such a $\nu=\mu_{n}$ for $\delta=(n+1)^{-1}$. Then $\lim_{n\to\infty}W_{2}(\mu_{n},\mu)=0$: given a positive $\eta$, claim 3 of The Archimedean Property of the Real Numbers gives $N\in\mathbb{N}$ with $0<N^{-1}<\eta$, and for $n\ge N$ one has $0<N<n+1$, hence $(n+1)^{-1}<N^{-1}$ by claim 10 of Elementary Order Arithmetic in an Ordered Field (multiplying by the positive number $N^{-1}(n+1)^{-1}$), so that $0\le W_{2}(\mu_{n},\mu)=W_{2}(\mu,\mu_{n})<(n+1)^{-1}<\eta$, $W_{2}$ being a metric; this is convergence to $0$ in the sense of Limit of a Sequence of Real Numbers. Claim 10, applied with the constant $\mathrm{Ent}(\mu)-\varepsilon$, gives $\mathrm{Ent}(\mu)\le\mathrm{Ent}(\mu)-\varepsilon$, that is $\varepsilon\le0$, contradicting $0<\varepsilon$. Hence $\mathrm{Ent}$ is lower semicontinuous at every point of $\mathcal{P}_{2}^{\mathrm{Ent}}(\mathbb{R}^{d})$, that is, on $\mathcal{P}_{2}^{\mathrm{Ent}}(\mathbb{R}^{d})$.