Proof of The Entropy of Measures on the Torus: Nonnegativity, Absolute Continuity and Closed Sublevel Sets

Each result cited below is universally quantified over the data in its own statement and is applied to the data named where it is used. Throughout, $\lambda_{d}$ is Lebesgue measure on $\mathcal{B}(\mathbb{R}^{d})$, $\phi$ is the function of The Function $s\log s$: Continuity, Young's Inequality and Lower Bounds, with the Elementary Bounds for the Exponential and the Logarithm (so $\phi(0)=0$ and $\phi(t)=t\log t$ for $t>0$), which is the function used in The Entropy of a Probability Measure on Euclidean Space §entropy, and densities are those of The Radon-Nikodym Theorem for a Finite Measure and a Sigma-Finite Measure, and Uniqueness of Densities, as in that definition: a density of $\kappa\in\mathcal{P}(\mathbb{R}^{d})$ is a Borel $h:\mathbb{R}^{d}\to\mathbb{R}$ with $0\le h(x)$ for every $x$ and $\kappa(A)=\int\mathbf{1}_{A}h\,d\lambda_{d}$ for every $A\in\mathcal{B}(\mathbb{R}^{d})$. The results Wasserstein Convergence from Weak Convergence with Uniformly Integrable Second Moments, and Wasserstein Compactness under a Superquadratic Moment Bound and Basic Properties of the Entropy on the Wasserstein Space: Comparison with the Gaussian Relative Entropy, Lower Bound, Translation Invariance, Absolute Continuity, Closed Sublevel Sets and Lower Semicontinuity are stated in the setting of Wasserstein Spaces, Random Vectors, Vector Fields and Symmetric Matrices in Every Dimension: Standing Notation; by Wasserstein Spaces, Random Vectors, Vector Fields and Symmetric Matrices in Every Dimension: Standing Notation §dimensions the sets $\mathcal{P}(\mathbb{R}^{m})$ and $\mathcal{P}_{2}(\mathbb{R}^{m})$, the second moment $M_{2}$ and the distance $W_{2}$ named there are the objects of Probability Measures on Euclidean Space and Random Vectors: Standing Notation §measures, The Second Moment of a Probability Measure on Euclidean Space and the Probability Measures with Finite Second Moment §moment, The Second Moment of a Probability Measure on Euclidean Space and the Probability Measures with Finite Second Moment §space and The Quadratic Wasserstein Distance on Euclidean Space §distance read in dimension $m$, which are the ones used in The Torus Wasserstein Distance: Comparison with the Euclidean Distance, Wrapping, and Integrals of Periodic Functions. The dimension $d$ fixed by The Wasserstein Space and Its Lift to Square-Integrable Random Vectors: Standing Notation §data, in which Basic Properties of the Entropy on the Wasserstein Space: Comparison with the Gaussian Relative Entropy, Lower Bound, Translation Invariance, Absolute Continuity, Closed Sublevel Sets and Lower Semicontinuity is stated, is an arbitrary natural number with $1\le d$, the results stated in that setting holding for each such choice; we take it to be the dimension $d$ of the torus, and we use Wasserstein Convergence from Weak Convergence with Uniformly Integrable Second Moments, and Wasserstein Compactness under a Superquadratic Moment Bound with $m=d$.

Preliminaries.

(i) Sets of full measure. Let $\kappa\in\mathcal{P}(\mathbb{T}^{d})$. Then $\kappa(Q)=1$ by Probability Measures on the Flat Torus, the Torus Cost of a Coupling, the Torus Wasserstein Distance and Optimal Couplings §measures, where $Q$ is Borel by The Half-Open Unit Cell Tiles Euclidean Space §cell, so $\kappa(\mathbb{R}^{d}\setminus Q)=1-1=0$ by claim 3 of Basic Properties of a Measure. For $A\in\mathcal{B}(\mathbb{R}^{d})$, finite additivity (claim 1 there) applied to the disjoint Borel sets $A\cap Q$ and $A\setminus Q$, together with $0\le\kappa(A\setminus Q)\le\kappa(\mathbb{R}^{d}\setminus Q)=0$ (claim 2 there), gives $\kappa(A)=\kappa(A\cap Q)$. Moreover $\kappa\in\mathcal{P}_{2}(\mathbb{R}^{d})$ with $M_{2}(\kappa)\le d$ by The Torus Wasserstein Distance: Comparison with the Euclidean Distance, Wrapping, and Integrals of Periodic Functions §inclusion.

(ii) A density vanishing off the cell. Let $\kappa\in\mathcal{P}^{\mathrm{Ent}}(\mathbb{T}^{d})$. By The Entropy of a Probability Measure on Euclidean Space §entropy we may choose a density $\rho$ of $\kappa$ such that $\phi\circ\rho$ is integrable with respect to $\lambda_{d}$. Put $\rho_{Q}=\mathbf{1}_{Q}\rho$, a nonnegative real-valued Borel function. For $A\in\mathcal{B}(\mathbb{R}^{d})$ we have $\mathbf{1}_{A\cap Q}\,\rho=\mathbf{1}_{A}\,\rho_{Q}$ pointwise, so by (i)

$$\kappa(A)=\kappa(A\cap Q)=\int\mathbf{1}_{A\cap Q}\,\rho\,d\lambda_{d}=\int\mathbf{1}_{A}\,\rho_{Q}\,d\lambda_{d};$$

thus $\rho_{Q}$ is a density of $\kappa$. Since $\phi(0)=0$, one has $\phi\circ\rho_{Q}=\mathbf{1}_{Q}\,(\phi\circ\rho)$ pointwise; this function is Borel by The Function $s\log s$: Continuity, Young's Inequality and Lower Bounds, with the Elementary Bounds for the Exponential and the Logarithm §continuous, and $|\phi\circ\rho_{Q}|\le|\phi\circ\rho|$, so $\int|\phi\circ\rho_{Q}|\,d\lambda_{d}\le\int|\phi\circ\rho|\,d\lambda_{d}<\infty$ by claim 1 of Linearity and Monotonicity of the Lebesgue Integral and $\phi\circ\rho_{Q}$ is integrable. As the entropy does not depend on the choice of the density (The Entropy of a Probability Measure on Euclidean Space §entropy),

$$\mathrm{Ent}(\kappa)=\int\phi\circ\rho_{Q}\,d\lambda_{d}.$$

Claim 1. Let $\mu\in\mathcal{P}^{\mathrm{Ent}}(\mathbb{T}^{d})$ and let $\rho_{Q}$ be the density of $\mu$ constructed in (ii).

Absolute continuity. Let $B\in\mathcal{B}(\mathbb{R}^{d})$ with $\lambda_{d}(B)=0$. The nonnegative Borel function $\mathbf{1}_{B}\rho_{Q}$ vanishes at every point outside the null set $B$, so The Lebesgue Integral and Null Sets: Almost-Everywhere Comparison, Markov's Inequality, and Dominated Convergence Almost Everywhere §null-integral gives $\mu(B)=\int\mathbf{1}_{B}\rho_{Q}\,d\lambda_{d}=0$. Hence $\mu$ is absolutely continuous in the sense of Absolutely Continuous Probability Measure on Euclidean Space §ac, which is the notion named in Optimal Transport on the Flat Torus: Standing Notation §maps.

Nonnegativity. We claim that $\phi(\rho_{Q}(x))\ge\rho_{Q}(x)-\mathbf{1}_{Q}(x)$ for every $x\in\mathbb{R}^{d}$. For $x\in Q$ this is $\phi(\rho_{Q}(x))\ge\rho_{Q}(x)-1$, which is The Function $s\log s$: Continuity, Young's Inequality and Lower Bounds, with the Elementary Bounds for the Exponential and the Logarithm §lower applied to the nonnegative number $\rho_{Q}(x)$; for $x\notin Q$ both sides equal $0$, since $\rho_{Q}(x)=0$ and $\phi(0)=0$. The function $\rho_{Q}$ is integrable with $\int\rho_{Q}\,d\lambda_{d}=\mu(\mathbb{R}^{d})=1$ (take $A=\mathbb{R}^{d}$ in (ii)), and $\mathbf{1}_{Q}$ is integrable with $\int\mathbf{1}_{Q}\,d\lambda_{d}=\lambda_{d}(Q)=1$ by The Half-Open Unit Cell Tiles Euclidean Space §cell. By (ii) and claim 2 of Linearity and Monotonicity of the Lebesgue Integral (monotonicity and linearity for integrable functions),

$$\mathrm{Ent}(\mu)=\int\phi\circ\rho_{Q}\,d\lambda_{d}\ge\int\rho_{Q}\,d\lambda_{d}-\int\mathbf{1}_{Q}\,d\lambda_{d}=1-1=0 .$$

Claim 2. Let $C$, $\mu$ and $(\mu_{n})_{n\in\mathbb{N}}$ be as in the statement.

A bound on the cell. For $x\in Q$ one has $0\le x_{i}<1$, hence $x_{i}^{2}\le1$, for every $i\in[d]$ (The Half-Open Unit Cell Tiles Euclidean Space), so $\lVert x\rVert^{2}=\sum_{i=1}^{d}x_{i}^{2}\le d\le d^{2}$ by claim 1 of Elementary Properties of the Euclidean Norm on $\mathbb{R}^n$, and $\lVert x\rVert\le d$ by claim 2 of Monotonicity of Squaring on the Nonnegative Elements of an Ordered Field. Put $K=d$, a positive real number.

Extraction of a weakly convergent subsequence. By (i) every $\mu_{n}$ belongs to $\mathcal{P}_{2}(\mathbb{R}^{d})$ with $M_{2}(\mu_{n})\le d$, so the set $\{\mu_{n}:n\in\mathbb{N}\}$ is tight in $(\mathbb{R}^{d},d_{E})$ by Tightness from Bounded Second Moments, and Tightness of the Couplings of Two Measures with Finite Second Moment §moment, with $m=d$ and $R=d$; that is, the sequence $(\mu_{n})_{n\in\mathbb{N}}$ is tight in the sense of Tight Family of Borel Measures on a Metric Space §sequence. By Prokhorov's Theorem on Euclidean Space: a Tight Sequence of Probability Measures Has a Weakly Convergent Subsequence, with $m=d$, there are a strictly increasing sequence $(n_{k})_{k\in\mathbb{N}}$ in $\mathbb{N}$ and $\bar\mu\in\mathcal{P}(\mathbb{R}^{d})$ such that $(\mu_{n_{k}})_{k\in\mathbb{N}}$ converges weakly to $\bar\mu$ on $(\mathbb{R}^{d},d_{E})$.

Wasserstein convergence. We apply Wasserstein Convergence from Weak Convergence with Uniformly Integrable Second Moments, and Wasserstein Compactness under a Superquadratic Moment Bound §convergence to $(\mu_{n_{k}})_{k\in\mathbb{N}}$, whose terms lie in $\mathcal{P}_{2}(\mathbb{R}^{d})$, and to $\bar\mu$. Let $\varepsilon$ be a positive real; we take the positive number $K$ above. The set $A_{K}=\{x\in\mathbb{R}^{d}:K<\lVert x\rVert\}$ is Borel, $x\mapsto\lVert x\rVert$ being Borel by Pairs of Euclidean Points: Coordinate Projections, Pairings, the Product Measure on a Euclidean Space, Borel Norm Functions and Finite Sets §functions, and the integral in the hypothesis of that clause is the integral against $\mu_{n_{k}}$ of the nonnegative Borel function $\mathbf{1}_{A_{K}}\lVert\cdot\rVert^{2}$. By the bound on the cell this function vanishes at every point of $Q$, hence outside $\mathbb{R}^{d}\setminus Q$, a $\mu_{n_{k}}$-null set by (i); so its integral is $0<\varepsilon$ by The Lebesgue Integral and Null Sets: Almost-Everywhere Comparison, Markov's Inequality, and Dominated Convergence Almost Everywhere §null-integral, for every $k$. That clause therefore gives $\bar\mu\in\mathcal{P}_{2}(\mathbb{R}^{d})$ and that the real sequence $(W_{2}(\mu_{n_{k}},\bar\mu))_{k\in\mathbb{N}}$ has limit $0$.

Entropy of the limit. Every $\mu_{n_{k}}$ has finite entropy and belongs to $\mathcal{P}_{2}(\mathbb{R}^{d})$, so it belongs to $\mathcal{P}_{2}^{\mathrm{Ent}}(\mathbb{R}^{d})$ of The Entropy of a Probability Measure on Euclidean Space §entropy, and $\mathrm{Ent}(\mu_{n_{k}})\le C$. By Basic Properties of the Entropy on the Wasserstein Space: Comparison with the Gaussian Relative Entropy, Lower Bound, Translation Invariance, Absolute Continuity, Closed Sublevel Sets and Lower Semicontinuity §closed, applied with $c=C$ to the sequence $(\mu_{n_{k}})_{k\in\mathbb{N}}$ and to $\bar\mu\in\mathcal{P}_{2}(\mathbb{R}^{d})$, we get $\bar\mu\in\mathcal{P}_{2}^{\mathrm{Ent}}(\mathbb{R}^{d})$ and $\mathrm{Ent}(\bar\mu)\le C$. In particular $\bar\mu$ is absolutely continuous by Basic Properties of the Entropy on the Wasserstein Space: Comparison with the Gaussian Relative Entropy, Lower Bound, Translation Invariance, Absolute Continuity, Closed Sublevel Sets and Lower Semicontinuity §absolutely-continuous.

The limit is carried by the cell. By The Half-Open Unit Cell Tiles Euclidean Space §cell, $\overline{Q}$ is closed and Borel and $Q\subseteq\overline{Q}$. By Probability Measures on Euclidean Space and Random Vectors: Standing Notation §spaces, $\mathcal{B}(\mathbb{R}^{d})$ is the Borel $\sigma$-algebra of the nonempty metric space $(\mathbb{R}^{d},d_{E})$, so claim 4 of Portmanteau Theorem on a Metric Space applies to the weakly convergent sequence $(\mu_{n_{k}})_{k\in\mathbb{N}}$ and the closed set $\overline{Q}$, giving $\limsup_{k}\mu_{n_{k}}(\overline{Q})\le\bar\mu(\overline{Q})$. For every $k$, $1=\mu_{n_{k}}(Q)\le\mu_{n_{k}}(\overline{Q})\le1$ by monotonicity (claim 2 of Basic Properties of a Measure), so the sequence $(\mu_{n_{k}}(\overline{Q}))_{k\in\mathbb{N}}$ is constant with value $1$ and its limit superior is $1$. Hence $1\le\bar\mu(\overline{Q})\le1$, and $\bar\mu(\mathbb{R}^{d}\setminus\overline{Q})=0$ by claim 3 of Basic Properties of a Measure. Next, $\lambda_{d}(\overline{Q}\setminus Q)=\lambda_{d}(\overline{Q})-\lambda_{d}(Q)=1-1=0$ by claim 3 of Basic Properties of a Measure and The Half-Open Unit Cell Tiles Euclidean Space §cell, so $\bar\mu(\overline{Q}\setminus Q)=0$ by Absolutely Continuous Probability Measure on Euclidean Space §ac. Finite additivity (claim 1 of Basic Properties of a Measure), applied to the disjoint Borel sets $\mathbb{R}^{d}\setminus\overline{Q}$ and $\overline{Q}\setminus Q$, whose union is $\mathbb{R}^{d}\setminus Q$, gives $\bar\mu(\mathbb{R}^{d}\setminus Q)=0$, hence $\bar\mu(Q)=1$ by claim 3 there. Thus $\bar\mu\in\mathcal{P}(\mathbb{T}^{d})$ by Probability Measures on the Flat Torus, the Torus Cost of a Coupling, the Torus Wasserstein Distance and Optimal Couplings §measures.

Identification of the limit. For every $k$, The Torus Wasserstein Distance: Comparison with the Euclidean Distance, Wrapping, and Integrals of Periodic Functions §euclidean gives $0\le W_{\mathbb{T}}(\mu_{n_{k}},\bar\mu)\le W_{2}(\mu_{n_{k}},\bar\mu)$. Let $\varepsilon$ be a positive real. By Limit of a Sequence of Real Numbers there is $L\in\mathbb{N}$ with $|W_{2}(\mu_{n_{k}},\bar\mu)-0|<\varepsilon$ for every $k\ge L$, and for such $k$ we get $W_{\mathbb{T}}(\mu_{n_{k}},\bar\mu)\le W_{2}(\mu_{n_{k}},\bar\mu)<\varepsilon$. By Optimal Transport on the Flat Torus: Standing Notation §measures, $(\mathcal{P}(\mathbb{T}^{d}),W_{\mathbb{T}})$ is a metric space, and we have shown that $(\mu_{n_{k}})_{k\in\mathbb{N}}$ converges to $\bar\mu$ in it. Since $(\mu_{n})_{n\in\mathbb{N}}$ converges to $\mu$ in the same metric space, its subsequence $(\mu_{n_{k}})_{k\in\mathbb{N}}$ also converges to $\mu$, by A Subsequence of a Convergent Sequence Has the Same Limit. By Uniqueness of Limits in a Metric Space, $\bar\mu=\mu$. Consequently $\mu$ has finite entropy, so $\mu\in\mathcal{P}^{\mathrm{Ent}}(\mathbb{T}^{d})$, and $\mathrm{Ent}(\mu)=\mathrm{Ent}(\bar\mu)\le C$. $\blacksquare$