Proof of Gaussian Smoothing of a Measure with Finite Second Moment: Positive Density, Finite Entropy, a Tangent Score, Non-Increasing Fisher Information and Exponential Integrability

Each result cited is universally quantified over the data in its own statement. Let $c_{s}$ be the constant with $g_{s}(z)=c_{s}\exp(-\lVert z\rVert^{2}/(2s))$ of The Gaussian Kernels on Euclidean Space: Scaling, Derivatives up to Order Three, the Convolution Identity, Moments and Tails, and let $\gamma_{s}$ be the measure with density $g_{s}$ with respect to $\lambda_{d}$ (Image Measures, Measures with Densities, and Change of Variables), a probability measure since $\int g_{s}\,d\lambda_{d}=1$. Integrals against a measure with a density are computed by claim 3 of Image Measures, Measures with Densities, and Change of Variables; integrals over product measures by Tonelli and Fubini Theorems, the integrands below being Borel as compositions of continuous or Borel maps (Probability Measures on Euclidean Space and Random Vectors: Standing Notation §borel-maps).

Step 0 (representation). Let $h:\mathbb{R}^{d}\to[0,\infty]$ be Borel. Then

The first equality is claim 3 of Image Measures, Measures with Densities, and Change of Variables together with $\rho_{s}(y)=\int g_{s}(y-x)\mu(dx)$ and Tonelli; the second is translation invariance of $\lambda_{d}$ (claim 2 of Translation and Reflection Invariance of Lebesgue Measure on $\mathbb{R}^n$). By linearity (R) extends to Borel $h$ for which either side is finite with $|h|$ in place of $h$. Moreover, for bounded Borel $\phi$, Gaussian Smoothing of a Probability Measure on Euclidean Space: Regularity, Mass, Duality, Approximation of a Bounded Function with a Modulus of Continuity, and the Pairing Identity §duality gives $\int\phi\,d\mu_{s}=\int(g_{s}*\phi)\,d\mu$, and by translation and reflection invariance (claim 3 of Translation and Reflection Invariance of Lebesgue Measure on $\mathbb{R}^n$) and the evenness of $g_{s}$ (The Gaussian Kernels on Euclidean Space: Scaling, Derivatives up to Order Three, the Convolution Identity, Moments and Tails §derivatives), $(g_{s}*\phi)(x)=\int\phi(x-z)\gamma_{s}(dz)=(\phi*\gamma_{s})(x)$, in the notation of The Convolution of a Bounded Continuous Function with a Probability Measure on Euclidean Space: Continuity, Boundedness and Differentiation Under the Integral §continuous.

Claim 1. By Gaussian Smoothing of a Probability Measure on Euclidean Space: Regularity, Mass, Duality, Approximation of a Bounded Function with a Modulus of Continuity, and the Pairing Identity §regularity, $\rho_{s}$ is of class $C^{3}$, hence of class $C^{2}$ (C^k Maps on a Euclidean Open Set), with

by The Gaussian Kernels on Euclidean Space: Scaling, Derivatives up to Order Three, the Convolution Identity, Moments and Tails §derivatives. Positivity. Let $R_{0}=\sqrt{2M_{2}(\mu)}+1$. Markov's inequality (The Lebesgue Integral and Null Sets: Almost-Everywhere Comparison, Markov's Inequality, and Dominated Convergence Almost Everywhere §markov, applied to $\lVert x\rVert^{2}$ at the level $R_{0}^{2}$) gives $\mu(\{R_{0}^{2}\le\lVert x\rVert^{2}\})\le M_{2}(\mu)R_{0}^{-2}\le\tfrac12$, so the ball $\bar{B}_{0}=\bar{B}(0_{\mathbb{R}^{d}},R_{0})$ has $\mu(\bar{B}_{0})\ge\tfrac12$. For $x\in\bar{B}_{0}$, $\lVert y-x\rVert^{2}\le(\lVert y\rVert+R_{0})^{2}\le2\lVert y\rVert^{2}+2R_{0}^{2}$, and $\exp$ is increasing (claim 4 of Basic Properties of the Exponential Function), so

Moments and distance. By (R) with $h=\lVert\cdot\rVert^{2}$, $\lVert x+z\rVert^{2}\le2\lVert x\rVert^{2}+2\lVert z\rVert^{2}$ (Pairs of Euclidean Points: Coordinate Projections, Pairings, the Product Measure on a Euclidean Space, Borel Norm Functions and Finite Sets §functions) and $\int\lVert z\rVert^{2}\gamma_{s}(dz)=ds$ (The Gaussian Kernels on Euclidean Space: Scaling, Derivatives up to Order Three, the Convolution Identity, Moments and Tails §moments), $M_{2}(\mu_{s})\le2M_{2}(\mu)+2ds<\infty$, so $\mu_{s}\in\mathcal{P}_{2}(\mathbb{R}^{d})$. Let $F:\mathbb{R}^{d+d}\to\mathbb{R}^{d+d}$ be $F(x,z)=(x+z,x)$, which is continuous, hence Borel, and let $\pi=F_{\#}(\mu\boxtimes\gamma_{s})$, with the product of Probability Measures on Euclidean Space and Random Vectors: Standing Notation §pairs. By the change of variables for push-forwards (Probability Measures on Euclidean Space and Random Vectors: Standing Notation §pushforward), Tonelli and (R), the first marginal of $\pi$ is $\mu_{s}$ and the second is $\mu$. So $\pi$ is a coupling of $\mu_{s}$ and $\mu$, with cost $\int\lVert(x+z)-x\rVert^{2}\,d(\mu\boxtimes\gamma_{s})=ds$. Hence $W_{2}(\mu_{s},\mu)^{2}\le ds$ by the definition of $W_{2}$ as an infimum over couplings (The Wasserstein Space and Its Lift to Square-Integrable Random Vectors: Standing Notation §wasserstein).

Claim 2. By (3) and the monotonicity and product rule of $\log$ (The Natural Logarithm, with claim 4 of Basic Properties of the Exponential Function), $\log\rho_{s}(y)\ge\log(\tfrac12c_{s})-s^{-1}(\lVert y\rVert^{2}+R_{0}^{2})$. By The Function $s\log s$: Continuity, Young's Inequality and Lower Bounds, with the Elementary Bounds for the Exponential and the Logarithm §log and the bound $\rho_{s}\le A_{0}s^{-d/2}$ of Gaussian Smoothing of a Probability Measure on Euclidean Space: Regularity, Mass, Duality, Approximation of a Bounded Function with a Modulus of Continuity, and the Pairing Identity §regularity, $\log\rho_{s}(y)\le\rho_{s}(y)-1\le A_{0}s^{-d/2}$. So $|\log\rho_{s}(y)|\le K_{0}+s^{-1}\lVert y\rVert^{2}$ with $K_{0}=A_{0}s^{-d/2}+|\log(\tfrac12c_{s})|+s^{-1}R_{0}^{2}$. With $\phi(u)=u\log u$ as in The Entropy of a Probability Measure on Euclidean Space, the function $\phi\circ\rho_{s}$ is Borel and satisfies $|\phi\circ\rho_{s}(y)|\le\rho_{s}(y)(K_{0}+s^{-1}\lVert y\rVert^{2})$, whose $\lambda_{d}$-integral is $K_{0}+s^{-1}M_{2}(\mu_{s})<\infty$. Since $\rho_{s}$ is a density of $\mu_{s}$ with respect to $\lambda_{d}$, The Entropy of a Probability Measure on Euclidean Space §entropy gives $\mu_{s}\in\mathcal{P}_{2}^{\mathrm{Ent}}(\mathbb{R}^{d})$.

Claim 3. Measurability and bound. The components $\rho_{s}^{-1}\partial_{i}\rho_{s}$ of $r_{s}$ are continuous by claim 2 of Continuity of the Reciprocal of a Nonvanishing Real-Valued Function on a Metric Space and claim 5 of Continuity of Sums and Products of Real-Valued Functions on a Metric Space, so $r_{s}$ is Borel. Fix $y$ and let $w_{y}(x)=g_{s}(y-x)\rho_{s}(y)^{-1}$, so that $w_{y}\ge0$ and $\int w_{y}\,d\mu=1$. By (2), $r_{s}(y)=-s^{-1}\int(y-x)w_{y}(x)\mu(dx)$ componentwise. Hoelder's Inequality, for Two and for Finitely Many Factors §holder with both exponents equal to $2$ (applied to $|y_{i}-x_{i}|\sqrt{w_{y}}$ and $\sqrt{w_{y}}$) gives $(\int(y_{i}-x_{i})w_{y}\,d\mu)^{2}\le\int(y_{i}-x_{i})^{2}w_{y}\,d\mu$. Summing over $i$,

Integrating in $y$ with Tonelli and translation invariance, $\int\lVert r_{s}\rVert^{2}d\mu_{s}\le s^{-2}\int\!\!\int\lVert z\rVert^{2}g_{s}(z)\,dz\,\mu(dx)=s^{-2}ds=ds^{-1}$.

Convexity. The logarithm is smooth on the open set $(0,\infty)$ with $\log'(u)=u^{-1}$ (The Natural Logarithm), and $\rho_{s}$ is $C^{2}$ with values in $(0,\infty)$ by (3). So $\Phi(y)=\tfrac12\lVert y\rVert^{2}+s\log\rho_{s}(y)$ is of class $C^{2}$ by A Composition of $C^k$ Maps Between Euclidean Open Sets is of Class $C^k$ and Constants, Coordinate Functions, Sums and Products of $C^k$ Functions on a Euclidean Open Set, with $\partial_{i}\Phi=y_{i}+s\rho_{s}^{-1}\partial_{i}\rho_{s}$ and

by claim 1 of A Composition of $C^k$ Maps Between Euclidean Open Sets is of Class $C^k$, Reciprocal Rule for One-Dimensional Derivatives and the product rule. So $\nabla\Phi=\mathrm{id}+s\,r_{s}$. Write $u(x)=y-x$ and $m_{i}=\int u_{i}w_{y}\,d\mu$ at a fixed $y$. By (2), $\rho_{s}^{-1}\partial_{i}\rho_{s}=-s^{-1}m_{i}$ and $\rho_{s}^{-1}\partial_{j}\partial_{i}\rho_{s}=s^{-2}\int u_{i}u_{j}w_{y}\,d\mu-s^{-1}\delta_{ij}$, so

The integrands are bounded (The Gaussian Kernels on Euclidean Space: Scaling, Derivatives up to Order Three, the Convolution Identity, Moments and Tails §bounds with the formulas of The Gaussian Kernels on Euclidean Space: Scaling, Derivatives up to Order Three, the Convolution Identity, Moments and Tails §derivatives). For $v\in\mathbb{R}^{d}$, bilinearity gives $v^{\mathsf T}D^{2}\Phi(y)v=s^{-1}\bigl(\int(v\cdot u)^{2}w_{y}\,d\mu-(\int(v\cdot u)w_{y}\,d\mu)^{2}\bigr)\ge0$, by Hölder as above with $v\cdot u$ in place of $u_{i}$. So $0_{d}\preceq D^{2}\Phi(y)$ for every $y$ (The Positive Semidefinite Ordering on Symmetric Matrices). Since $\mathbb{R}^{d}$ is open and convex, and $\Phi$ has continuous partial derivatives of orders one and two (C^k Maps on a Euclidean Open Set), which is the $C^{2}$ hypothesis of A Function of Class $C^2$ with Positive Semidefinite Hessian is Convex, that theorem shows that $\Phi$ is convex.

Tangency and the score. Since $\lVert\nabla\Phi\rVert^{2}\le2\lVert\mathrm{id}\rVert^{2}+2s^{2}\lVert r_{s}\rVert^{2}$ (Pairs of Euclidean Points: Coordinate Projections, Pairings, the Product Measure on a Euclidean Space, Borel Norm Functions and Finite Sets §functions) is $\mu_{s}$-integrable, The Gradient of a Convex Continuously Differentiable Function Belongs to the Tangent Space When It Is Square-Integrable §tangent gives $\nabla\Phi\in T_{\mu_{s}}$. Also $\mathrm{id}\in T_{\mu_{s}}$ by Basic Properties of the Tangent Space: Closed Subspace, the Identity Map Belongs to It, Second-Moment Limits, and Representation of Bounded Functionals on Gradients §identity, and $T_{\mu_{s}}$ is a linear subspace (Basic Properties of the Tangent Space: Closed Subspace, the Identity Map Belongs to It, Second-Moment Limits, and Representation of Bounded Functionals on Gradients §closed), so $r_{s}=s^{-1}(\nabla\Phi-\mathrm{id})\in T_{\mu_{s}}$. Now The Score of a Measure with a Positive Continuously Differentiable Density applies with the density $\rho_{s}$ (of class $C^{1}$ and positive), the measure $\mu_{s}\in\mathcal{P}_{2}(\mathbb{R}^{d})$ and the field $r_{s}$ (square-integrable). The Score of a Measure with a Positive Continuously Differentiable Density §projection gives $\mu_{s}\in\mathcal{P}_{2}^{\mathcal{I}}(\mathbb{R}^{d})$, and The Score of a Measure with a Positive Continuously Differentiable Density §equality gives $\xi_{\mu_{s}}=r_{s}$.

Claim 4. Let $\mu\in\mathcal{P}_{2}^{\mathcal{I}}(\mathbb{R}^{d})$ and $\psi\in C_{c}^{\infty}(\mathbb{R}^{d})$. Each $\partial_{i}\psi$ is again a test function: it is smooth (Smooth Map on a Euclidean Open Set) and compactly supported (claim 1 of The Gradient of a Test Function is Bounded and Square-Integrable, and Its Laplacian Bounded and Integrable, Against Every Probability Measure). So $\psi$, its first partials and its second partials are bounded (same claim). By The Convolution of a Bounded Continuous Function with a Probability Measure on Euclidean Space: Continuity, Boundedness and Differentiation Under the Integral §derivative, applied to $\psi$ and then to each $\partial_{i}\psi$ with the probability measure $\gamma_{s}$, the function $\Psi=\psi*\gamma_{s}$ is of class $C^{2}$ with $\partial_{i}\Psi=(\partial_{i}\psi)*\gamma_{s}$ and $\partial_{j}\partial_{i}\Psi=(\partial_{j}\partial_{i}\psi)*\gamma_{s}$, all bounded (The Convolution of a Bounded Continuous Function with a Probability Measure on Euclidean Space: Continuity, Boundedness and Differentiation Under the Integral §continuous); in particular $\Delta\Psi=(\Delta\psi)*\gamma_{s}$ by linearity of the integral. By Finite Fisher Information, the Score and the Fisher Information of a Probability Measure §score, Step 0 with $\phi=\Delta\psi$, and Gradients of Functions with Bounded Derivatives Belong to the Tangent Space; Constants Are Tangent; the Score Identity; the Score Has Mean Zero; Translation of Tangent Fields §score-identity applied to $\mu$ and $f=\Psi$,

By Hölder with respect to $\gamma_{s}$ as in Claim 3, $\lVert\nabla\Psi(x)\rVert^{2}=\sum_{i}((\partial_{i}\psi)*\gamma_{s})(x)^{2}\le(\lVert\nabla\psi\rVert^{2}*\gamma_{s})(x)$, so Step 0 gives $\lVert\nabla\Psi\rVert_{\mu}^{2}\le\int\lVert\nabla\psi\rVert^{2}*\gamma_{s}\,d\mu=\lVert\nabla\psi\rVert_{\mu_{s}}^{2}$. By The Cauchy-Schwarz Inequality in a Real Inner Product Space in $L^{2}(\mu;\mathbb{R}^{d})$, $|\langle\xi_{\mu_{s}},\nabla\psi\rangle_{\mu_{s}}|\le\lVert\xi_{\mu}\rVert_{\mu}\lVert\nabla\psi\rVert_{\mu_{s}}$. Now apply Basic Properties of the Tangent Space: Closed Subspace, the Identity Map Belongs to It, Second-Moment Limits, and Representation of Bounded Functionals on Gradients §representation at $\mu_{s}$ with the linear functional $\ell(\psi)=\langle\xi_{\mu_{s}},\nabla\psi\rangle_{\mu_{s}}$ and $C=\lVert\xi_{\mu}\rVert_{\mu}$. The unique element of $T_{\mu_{s}}$ representing $\ell$ is $\xi_{\mu_{s}}$, and its norm is at most $C$.

Claim 5. By (R) applied to $|h|$, and since $\mu(\bar{B}(0_{\mathbb{R}^{d}},R))=1$ while $\exp$ is increasing and multiplicative (claims 1 and 4 of Basic Properties of the Exponential Function),

Since $0\le(B\sqrt{s}-\tfrac12s^{-1/2}\lVert z\rVert)^{2}$, one has $B\lVert z\rVert\le B^{2}s+\tfrac14s^{-1}\lVert z\rVert^{2}$. Hence $\exp(B\lVert z\rVert)g_{s}(z)\le\exp(B^{2}s)\,c_{s}\exp(-\lVert z\rVert^{2}/(4s))=\exp(B^{2}s)\,c_{s}c_{2s}^{-1}g_{2s}(z)$, where $g_{2s}$ is defined because $2s\le1$. The weight $g_{2s}$ has integral $1$, and $c_{s}c_{2s}^{-1}=(\sqrt2)^{d}$ by The Gaussian Kernels on Euclidean Space: Scaling, Derivatives up to Order Three, the Convolution Identity, Moments and Tails §scaling. So $\int|h|\,d\mu_{s}\le A\exp(BR+B^{2}s)(\sqrt2)^{d}<\infty$, and $h$ is $\mu_{s}$-integrable. $\blacksquare$