Proof of Heat Regularisation of the Entropy: a Laplacian Test Function whose Laplacian is Minus the Fisher Information

Each result cited below is universally quantified over the data in its own statement.

Fix $s$ with $0<s\le\tfrac12$, write $\Theta=\Theta_{s}$ and $p_{\mu}=p^{s}_{\mu}$ for $\mu\in\mathcal{P}(\mathbb{T}^{d})$, and let $c_{s},C_{s}$ be the constants of The Torus Heat Kernel: Regularity and Bounds, the Density of a Heat-Smoothed Measure, and Its Lipschitz Dependence on the Measure §regularity and $L_{s}$ that of The Torus Heat Kernel: Regularity and Bounds, the Density of a Heat-Smoothed Measure, and Its Lipschitz Dependence on the Measure §lipschitz. As in Heat Averages of Vector Fields on the Torus: Regularity, Contraction, Tangency, the Divergence Identity and Synchronous Couplings §average, $\int_{Q}f\,d\lambda_{d}$ stands for $\int\mathbf{1}_{Q}f\,d\lambda_{d}$; since $\lambda_{d}(Q)=1$ by The Half-Open Unit Cell Tiles Euclidean Space §cell, $|\int_{Q}f\,d\lambda_{d}|\le M$ whenever $f$ is Borel with $|f|\le M$ on $Q$. For continuous periodic $f$ this integral equals the torus integral $\int_{\mathbb{T}^{d}}f\,dx$ of The Flat Torus: Standing Notation §measure, by Zero Extension of a Real-Valued Function, and the Unit-Cell Integral of a Continuous Function §extension.

Step 0 (Kernel and density estimates). Let $\Psi$ be one of $\Theta,\partial_{1}\Theta,\dots,\partial_{d}\Theta$. By The Torus Heat Kernel: Regularity and Bounds, the Density of a Heat-Smoothed Measure, and Its Lipschitz Dependence on the Measure §regularity and Elementary Properties of Lattice-Periodic Functions §derivative, $\Psi$ is $\mathbb{Z}^{d}$-periodic and of class $C^{1}$ on $\mathbb{R}^{d}$, $|\Psi|\le C_{s}$, and all first partial derivatives of $\Psi$ are bounded by $C_{s}$ in absolute value; so the Lipschitz bound (i) of Multivariate Taylor Expansion with Uniform Second-Order Remainder gives

$$|\Psi(u)-\Psi(u')|\le\sqrt{d}\,C_{s}\,\lVert u-u'\rVert\qquad(u,u'\in\mathbb{R}^{d}).\tag{0.1}$$

For $\mu\in\mathcal{P}(\mathbb{T}^{d})$, The Torus Heat Kernel: Regularity and Bounds, the Density of a Heat-Smoothed Measure, and Its Lipschitz Dependence on the Measure §density gives $p_{\mu}\in C^{2}_{\mathrm{per}}$, $c_{s}\le p_{\mu}\le C_{s}$ and $|\partial_{k}p_{\mu}(y)|=|\int\partial_{k}\Theta(y-x)\,\mu(dx)|\le C_{s}$; it also shows that $S_{s}\mu\in\mathcal{P}^{\mathcal{I}}(\mathbb{T}^{d})$ and that $\xi_{S_{s}\mu}$ is the class of $\zeta_{\mu}=p_{\mu}^{-1}\nabla p_{\mu}$. The components of $\zeta_{\mu}$ are continuous and periodic with $|\zeta_{\mu,k}|\le C_{s}/c_{s}$, so $\zeta_{\mu}$ is Borel with $\int\lVert\zeta_{\mu}\rVert^{2}\,d(S_{s}\mu)<\infty$.

Step 1 (The candidate field; condition (a) and tangency). For $\mu\in\mathcal{P}(\mathbb{T}^{d})$ and $x\in\mathbb{R}^{d}$ put $g(\mu,x)=A^{\mu}_{s}\xi_{S_{s}\mu}(x)=\int_{Q}\zeta_{\mu}(y)\,\Theta(y-x)\,\lambda_{d}(dy)$; by Heat Averages of Vector Fields on the Torus: Regularity, Contraction, Tangency, the Divergence Identity and Synchronous Couplings §average (with $F=\zeta_{\mu}$) this does not depend on the representative, so $g$ is the map $(\mu,x)\mapsto A^{\mu}_{s}\xi_{S_{s}\mu}(x)$ of the statement. By the same clause each $g_{i}(\mu,\cdot)$ belongs to $C^{1}_{\mathrm{per}}$, which is condition (a) of a gradient field, and for all $x$ and $i,j\in[d]$

$$g_{i}(\mu,x)=\int_{Q}\zeta_{\mu,i}(y)\,\Theta(y-x)\,\lambda_{d}(dy),\qquad\partial_{j}g_{i}(\mu,x)=-\int_{Q}\zeta_{\mu,i}(y)\,\partial_{j}\Theta(y-x)\,\lambda_{d}(dy).\tag{1.1}$$

The score $\xi_{S_{s}\mu}$ lies in $T_{S_{s}\mu}$ by Finite Torus Fisher Information, the Torus Score and the Torus Fisher Information of a Probability Measure on the Torus §score, so Heat Averages of Vector Fields on the Torus: Regularity, Contraction, Tangency, the Divergence Identity and Synchronous Couplings §tangent shows that the class of $g(\mu,\cdot)$ in $L^{2}(\mu;\mathbb{R}^{d})$ lies in $T_{\mu}$.

Step 2 (The logarithm of the density). Fix $\mu$ and write $p=p_{\mu}$, $\zeta=\zeta_{\mu}$. The function $\log p$ is $\mathbb{Z}^{d}$-periodic because $p$ is. For $y\in\mathbb{R}^{d}$ and $k\in[d]$, the map $t\mapsto p(y+te_{k})$ is differentiable at $0$ with derivative $\partial_{k}p(y)$ and takes values in $(0,\infty)$, where $\log$ is differentiable with $\log'(t)=1/t$ by The Natural Logarithm; so the chain rule Chain Rule for One-Dimensional Derivatives gives $\partial_{k}(\log p)(y)=\partial_{k}p(y)/p(y)=\zeta_{k}(y)$. Since $\partial_{k}p$ and $p$ are of class $C^{1}$ (Elementary Properties of Lattice-Periodic Functions §derivative) and $p$ does not vanish, $\zeta_{k}$ is of class $C^{1}$, in particular continuous, by claim (ii) of Products and Quotients of C^k Real-Valued Maps on Euclidean Open Sets Are C^k. Hence $\log p\in C^{1}_{\mathrm{per}}$, and $\log p$ is bounded by Elementary Properties of Lattice-Periodic Functions §bounded; fix $B\ge0$ with $|\log p|\le B$. For $x\in\mathbb{R}^{d}$ the map $y\mapsto\Theta(y-x)$ lies in $C^{1}_{\mathrm{per}}$ with partial derivatives $y\mapsto\partial_{k}\Theta(y-x)$ (partial derivatives commute with translations, directly from their definition). The integration by parts formula Vanishing Mean of a Derivative and Integration by Parts on the Torus §parts, applied to these two functions, and (1.1) give, for all $x\in\mathbb{R}^{d}$ and $k\in[d]$,

$$\int_{Q}\log p(y)\,\partial_{k}\Theta(y-x)\,\lambda_{d}(dy)=-\int_{Q}\zeta_{k}(y)\,\Theta(y-x)\,\lambda_{d}(dy)=-g_{k}(\mu,x).\tag{2.1}$$

Step 3 (Condition (b): differentiability along couplings). Keep $\mu$, $p$, $\zeta$ and $B$ from Step 2 and put $K=L_{s}^{2}/c_{s}+\tfrac12 B\,d\,C_{s}$. We first show that for every $\nu\in\mathcal{P}(\mathbb{T}^{d})$ and every $\gamma\in\Pi(\mu,\nu)$

$$\bigl|\mathrm{Ent}(S_{s}\nu)-\mathrm{Ent}(S_{s}\mu)-\mathcal{J}_{\mathbb{T}}(g(\mu,\cdot),\gamma)\bigr|\le K\,I_{\mathbb{T}}(\gamma).\tag{3.1}$$

Let $q=p_{\nu}$.

(i) Entropy as a cell integral. By The Torus Heat Kernel: Regularity and Bounds, the Density of a Heat-Smoothed Measure, and Its Lipschitz Dependence on the Measure §density, $\mathbf{1}_{Q}p$ is a density of $S_{s}\mu$; since the function $\phi$ of The Function $s\log s$: Continuity, Young's Inequality and Lower Bounds, with the Elementary Bounds for the Exponential and the Logarithm vanishes at $0$, $\phi\circ(\mathbf{1}_{Q}p)=\mathbf{1}_{Q}\,p\log p$, which is Borel and bounded, hence integrable. By The Entropy of a Probability Measure on Euclidean Space §entropy (the entropy may be computed with any density), $\mathrm{Ent}(S_{s}\mu)=\int_{Q}p\log p\,d\lambda_{d}$, and likewise $\mathrm{Ent}(S_{s}\nu)=\int_{Q}q\log q\,d\lambda_{d}$. Moreover $\int_{Q}p\,d\lambda_{d}=S_{s}\mu(\mathbb{R}^{d})=1=\int_{Q}q\,d\lambda_{d}$, by the density property.

(ii) Expansion of $t\log t$. For $y\in\mathbb{R}^{d}$ put $R(y)=q\log q-p\log p-(1+\log p)(q-p)$, all functions evaluated at $y$. Using $\log q=\log(q/p)+\log p$ (The Natural Logarithm), $R=q\log(q/p)-q+p$. The bounds $1-t^{-1}\le\log t\le t-1$ of The Function $s\log s$: Continuity, Young's Inequality and Lower Bounds, with the Elementary Bounds for the Exponential and the Logarithm §log with $t=q/p$ give $q-p\le q\log(q/p)\le q(q-p)/p$, hence $0\le R\le(q-p)^{2}/p\le(q-p)^{2}/c_{s}$. By The Torus Heat Kernel: Regularity and Bounds, the Density of a Heat-Smoothed Measure, and Its Lipschitz Dependence on the Measure §lipschitz and Probability Measures on the Flat Torus, the Torus Cost of a Coupling, the Torus Wasserstein Distance and Optimal Couplings §distance, $|q(y)-p(y)|\le L_{s}W_{\mathbb{T}}(\mu,\nu)\le L_{s}\sqrt{I_{\mathbb{T}}(\gamma)}$, so $0\le\int_{Q}R\,d\lambda_{d}\le L_{s}^{2}I_{\mathbb{T}}(\gamma)/c_{s}$. By (i), $\int_{Q}(q-p)\,d\lambda_{d}=0$, so

$$\mathrm{Ent}(S_{s}\nu)-\mathrm{Ent}(S_{s}\mu)=\int_{Q}\log p\,(q-p)\,d\lambda_{d}+\int_{Q}R\,d\lambda_{d}.\tag{3.2}$$

(iii) Expansion of the kernel. Let $y\in\mathbb{R}^{d}$ and $w\in\mathbb{R}^{d+d}$, and put $x=\mathrm{pr}_{1}(w)$, $x'=\mathrm{pr}_{2}(w)$, $v=\varpi(x'-x)$. By The Flat Torus Distance: Minimality of the Wrapped Displacement, Periodicity, the Metric on the Unit Cell and the Lipschitz Bound §range, $x'-x-v\in\mathbb{Z}^{d}$, so $\Theta(y-x')=\Theta(y-x-v)$ by periodicity; and $\lVert v\rVert=d_{\mathbb{T}}(x,x')$ by The Wrapped Displacement and the Flat Torus Distance §distance. The first-order remainder bound (ii) of Multivariate Taylor Expansion with Uniform Second-Order Remainder, applied to $\Theta$ on $\mathbb{R}^{d}$ between $y-x$ and $y-x-v$ with $M_{2}=C_{s}$ (The Torus Heat Kernel: Regularity and Bounds, the Density of a Heat-Smoothed Measure, and Its Lipschitz Dependence on the Measure §regularity), gives

$$E(y,w):=\Theta(y-x')-\Theta(y-x)+\nabla\Theta(y-x)\cdot v,\qquad|E(y,w)|\le\tfrac12\,d\,C_{s}\,d_{\mathbb{T}}(x,x')^{2}.$$

Since $(\mathrm{pr}_{1})_{\#}\gamma=\mu$ and $(\mathrm{pr}_{2})_{\#}\gamma=\nu$ (Couplings of Two Probability Measures on Euclidean Space and Their Quadratic Cost §coupling), the change-of-variables formula gives $p(y)=\int\Theta(y-\mathrm{pr}_{1}(w))\,\gamma(dw)$ and $q(y)=\int\Theta(y-\mathrm{pr}_{2}(w))\,\gamma(dw)$, the integrands being bounded and Borel ($\varpi$ is Borel by The Flat Torus Distance: Minimality of the Wrapped Displacement, Periodicity, the Metric on the Unit Cell and the Lipschitz Bound §lipschitz). Hence

$$q(y)-p(y)=-\sum_{k=1}^{d}\int\partial_{k}\Theta(y-\mathrm{pr}_{1}(w))\,\varpi_{k}(\mathrm{pr}_{2}(w)-\mathrm{pr}_{1}(w))\,\gamma(dw)+\rho(y),\qquad\rho(y)=\int E(y,w)\,\gamma(dw),$$

with $|\rho(y)|\le\tfrac12\,d\,C_{s}\,I_{\mathbb{T}}(\gamma)$ by Probability Measures on the Flat Torus, the Torus Cost of a Coupling, the Torus Wasserstein Distance and Optimal Couplings §cost. In particular $|\int_{Q}\log p\,\rho\,d\lambda_{d}|\le\tfrac12 B\,d\,C_{s}\,I_{\mathbb{T}}(\gamma)$.

(iv) The first-order term. Let $U$ be the measure with density $\mathbf{1}_{Q}$ with respect to $\lambda_{d}$ (claim 3 of Image Measures, Measures with Densities, and Change of Variables); it is a probability measure since $\lambda_{d}(Q)=1$, and $\int f\,dU=\int_{Q}f\,d\lambda_{d}$ for bounded Borel $f$ by the same claim. For $k\in[d]$ let $\Phi_{k}(y,w)=\log p(y)\,\partial_{k}\Theta(y-\mathrm{pr}_{1}(w))\,\varpi_{k}(\mathrm{pr}_{2}(w)-\mathrm{pr}_{1}(w))$ on $\mathbb{R}^{d}\times\mathbb{R}^{d+d}$. It is bounded (by $B\,C_{s}\sqrt{d}$, using The Flat Torus Distance: Minimality of the Wrapped Displacement, Periodicity, the Metric on the Unit Cell and the Lipschitz Bound §range) and is the composite of a Borel map on $\mathbb{R}^{d+(d+d)}$ with the concatenation map, hence measurable for the product $\sigma$-algebra by Pairs of Euclidean Points: Coordinate Projections, Pairings, the Product Measure on a Euclidean Space, Borel Norm Functions and Finite Sets §product (applied to its positive and negative parts); so it is integrable with respect to the probability measure $U\otimes\gamma$. Fubini's theorem (Tonelli and Fubini Theorems) and then (2.1) give

$$\int_{Q}\log p(y)\Bigl(\int\partial_{k}\Theta(y-\mathrm{pr}_{1}(w))\,\varpi_{k}(\mathrm{pr}_{2}(w)-\mathrm{pr}_{1}(w))\,\gamma(dw)\Bigr)\lambda_{d}(dy)=\int\Bigl(\int_{Q}\log p(y)\,\partial_{k}\Theta(y-\mathrm{pr}_{1}(w))\,\lambda_{d}(dy)\Bigr)\varpi_{k}(\mathrm{pr}_{2}(w)-\mathrm{pr}_{1}(w))\,\gamma(dw)=-\int g_{k}(\mu,\mathrm{pr}_{1}(w))\,\varpi_{k}(\mathrm{pr}_{2}(w)-\mathrm{pr}_{1}(w))\,\gamma(dw),$$

where the left side is $\int\bigl(\int\Phi_{k}(y,w)\,\gamma(dw)\bigr)U(dy)$ and the middle is $\int\bigl(\int\Phi_{k}(y,w)\,U(dy)\bigr)\gamma(dw)$; every section of the bounded function $\Phi_{k}$ is integrable, so no exceptional null set arises. Multiplying the expansion of (iii) by $\log p(y)$, integrating over $Q$ and summing over $k$, by The Torus Displacement Pairing of a Vector Field Along a Coupling §pairing,

$$\int_{Q}\log p\,(q-p)\,d\lambda_{d}=\sum_{k=1}^{d}\int g_{k}(\mu,\mathrm{pr}_{1}(w))\,\varpi_{k}(\mathrm{pr}_{2}(w)-\mathrm{pr}_{1}(w))\,\gamma(dw)+\int_{Q}\log p\,\rho\,d\lambda_{d}=\mathcal{J}_{\mathbb{T}}(g(\mu,\cdot),\gamma)+\int_{Q}\log p\,\rho\,d\lambda_{d}.$$

(v) Conclusion of (3.1). By (3.2), (iv), (ii) and (iii), the left side of (3.1) is at most $\int_{Q}R\,d\lambda_{d}+|\int_{Q}\log p\,\rho\,d\lambda_{d}|\le L_{s}^{2}I_{\mathbb{T}}(\gamma)/c_{s}+\tfrac12 B\,d\,C_{s}\,I_{\mathbb{T}}(\gamma)=K\,I_{\mathbb{T}}(\gamma)$.

Now let $\varepsilon>0$ and put $\theta=\varepsilon/(K+1)>0$. If $\nu\in\mathcal{P}(\mathbb{T}^{d})$ and $\gamma\in\Pi(\mu,\nu)$ satisfy $I_{\mathbb{T}}(\gamma)<\theta^{2}$, then by (3.1) the defect is at most $K\sqrt{I_{\mathbb{T}}(\gamma)}\,\sqrt{I_{\mathbb{T}}(\gamma)}\le K\theta\sqrt{I_{\mathbb{T}}(\gamma)}\le\varepsilon\sqrt{I_{\mathbb{T}}(\gamma)}$. By Differentiability Along Couplings of a Function on the Torus Wasserstein Space, and Its Gradient §differentiable, $\mathrm{Ent}\circ S_{s}$ (real-valued by Heat Smoothing, Entropy, Fisher Information and Laplacian Test Functions on the Torus Wasserstein Space: Standing Notation §calculus) is differentiable along couplings at $\mu$ with gradient the class of $g(\mu,\cdot)$, and by the uniqueness in Differentiability Along Couplings of a Function on the Torus Wasserstein Space, and Its Gradient §gradient, $\nabla(\mathrm{Ent}\circ S_{s})(\mu)$ is the class of $g(\mu,\cdot)$, that is $A^{\mu}_{s}\xi_{S_{s}\mu}$. This is condition (b) of a gradient field.

Step 4 (Condition (c): joint continuity). Let $(\mu_{n})$ converge to $\mu$ in $(\mathcal{P}(\mathbb{T}^{d}),W_{\mathbb{T}})$ and $(x_{n})$ converge to $x$ in $\mathbb{R}^{d}$; write $p_{n}=p_{\mu_{n}}$, $p=p_{\mu}$, $\zeta^{n}=\zeta_{\mu_{n}}$, $\zeta=\zeta_{\mu}$. For $i\in[d]$ and $y\in\mathbb{R}^{d}$,

$$\zeta^{n}_{i}(y)-\zeta_{i}(y)=\frac{\partial_{i}p_{n}(y)-\partial_{i}p(y)}{p_{n}(y)}+\partial_{i}p(y)\,\frac{p(y)-p_{n}(y)}{p_{n}(y)\,p(y)},$$

so by Step 0 and both inequalities of The Torus Heat Kernel: Regularity and Bounds, the Density of a Heat-Smoothed Measure, and Its Lipschitz Dependence on the Measure §lipschitz, $\sup_{y}|\zeta^{n}_{i}(y)-\zeta_{i}(y)|\le\varepsilon_{n}:=(L_{s}/c_{s}+C_{s}L_{s}/c_{s}^{2})\,W_{\mathbb{T}}(\mu_{n},\mu)$. Let $\Psi\in\{\Theta,\partial_{1}\Theta,\dots,\partial_{d}\Theta\}$. Splitting $\zeta^{n}_{i}(y)\Psi(y-x_{n})-\zeta_{i}(y)\Psi(y-x)=(\zeta^{n}_{i}(y)-\zeta_{i}(y))\Psi(y-x_{n})+\zeta_{i}(y)(\Psi(y-x_{n})-\Psi(y-x))$ and using $|\Psi|\le C_{s}$, $|\zeta_{i}|\le C_{s}/c_{s}$ and (0.1),

$$\Bigl|\int_{Q}\zeta^{n}_{i}(y)\Psi(y-x_{n})\,\lambda_{d}(dy)-\int_{Q}\zeta_{i}(y)\Psi(y-x)\,\lambda_{d}(dy)\Bigr|\le C_{s}\,\varepsilon_{n}+\frac{\sqrt{d}\,C_{s}^{2}}{c_{s}}\,\lVert x_{n}-x\rVert\longrightarrow0,$$

since $W_{\mathbb{T}}(\mu_{n},\mu)\to0$ and $\lVert x_{n}-x\rVert\to0$. With $\Psi=\Theta$ and with $\Psi=\partial_{j}\Theta$, (1.1) turns this into $g_{i}(\mu_{n},x_{n})\to g_{i}(\mu,x)$ and $\partial_{j}g_{i}(\mu_{n},x_{n})\to\partial_{j}g_{i}(\mu,x)$, which is condition (c) of a gradient field.

Step 5 (Clause 1). By Steps 1, 3 and 4, $g$ is a gradient field of $\mathrm{Ent}\circ S_{s}$, and by Step 1 the class of $g(\mu,\cdot)$ lies in $T_{\mu}$ for every $\mu$. By Laplacian Test Functions on the Torus: Uniqueness of the Gradient Field, a Criterion, and Linearity of the Test Classes §criterion, $\mathrm{Ent}\circ S_{s}$ is a Laplacian test function; by Laplacian Test Functions on the Torus: Uniqueness of the Gradient Field, a Criterion, and Linearity of the Test Classes §unique its gradient field is $g$; and Step 3 gives $\nabla(\mathrm{Ent}\circ S_{s})(\mu)=A^{\mu}_{s}\xi_{S_{s}\mu}$.

Step 6 (Clause 2). Let $\mu\in\mathcal{P}(\mathbb{T}^{d})$. By The Laplacian of a Laplacian Test Function on the Torus Wasserstein Space §laplacian and Step 5, $\mathcal{L}(\mathrm{Ent}\circ S_{s})(\mu)=\int\sum_{i=1}^{d}\partial_{i}g_{i}(\mu,x)\,\mu(dx)$. Since $g(\mu,\cdot)$ is the heat average of $F=\zeta_{\mu}$, Heat Averages of Vector Fields on the Torus: Regularity, Contraction, Tangency, the Divergence Identity and Synchronous Couplings §divergence gives $\int\sum_{i}\partial_{i}g_{i}(\mu,\cdot)\,d\mu=-\langle\xi_{S_{s}\mu},\xi_{S_{s}\mu}\rangle_{S_{s}\mu}=-\lVert\xi_{S_{s}\mu}\rVert_{S_{s}\mu}^{2}$, which is $-\mathcal{I}(S_{s}\mu)$ by Finite Torus Fisher Information, the Torus Score and the Torus Fisher Information of a Probability Measure on the Torus §information. $\blacksquare$