Proof of The Gaussian Entropy Pair is a Noise Penalty Pair, with Nonnegative Penalty and Dense Score Domain

Each result cited is universally quantified over the data in its own statement.

Throughout, $\mathcal{D}$, $\mathcal{D}_{\Sigma}$, $\mathcal{E}$ and $\Sigma$ are as in The Gaussian Entropy Pair on the Noise Wasserstein Space: Relative Entropy and the Noise Score Field §pair, and the Euclidean items cited for measures on $\mathbb{R}^{n}$ are read with $n$ in place of the dimension $d$, as fixed in A Diagonal Gaussian Reference Measure on the Noise Wasserstein Space, Rescaled Heads and Gaussian Tails: Standing Notation §euclidean. For $n\in\mathbb{N}$ let $(\mathcal{D}^{(n)},\mathcal{D}^{(n)}_{\Sigma},\mathcal{E}^{(n)},\Sigma^{(n)})$ be the Gaussian free-energy pair on $\mathcal{P}_{2}(\mathbb{R}^{n})$ with variances $\tilde{c}^{(n)}$ and temperature $1$; this is admissible because $\tilde{c}^{(n)}$ is a variance vector by A Diagonal Gaussian Reference Measure on the Noise Wasserstein Space, Rescaled Heads and Gaussian Tails: Standing Notation §heads, and the Gaussian measure with variances $\tilde{c}^{(n)}$ in that definition is $\tilde{\gamma}_{n}$. By that definition, $\mathcal{D}^{(n)}$ is the set of the $\sigma\in\mathcal{P}_{2}(\mathbb{R}^{n})$ of finite relative entropy with respect to $\tilde{\gamma}_{n}$, and $\mathcal{D}^{(n)}_{\Sigma}$ the set of the $\sigma\in\mathcal{D}^{(n)}$ of finite Fisher information relative to $\tilde{\gamma}_{n}$. By The Gaussian Free-Energy Pair is a Penalty Pair: Translations, the First Variation, Lower Semicontinuity, the Moment Bound and the Map Property §pair this quadruple is a penalty pair on $\mathcal{P}_{2}(\mathbb{R}^{n})$.

Step 0 (Gaussian-tail extensions of the Euclidean score domain lie in $\mathcal{D}_{\Sigma}$). Let $n\in\mathbb{N}$ and $\sigma\in\mathcal{D}^{(n)}_{\Sigma}$. Since $\sigma\in\mathcal{P}_{2}(\mathbb{R}^{n})$, its Gaussian-tail extension $E_{n}(\sigma)$ lies in $\mathcal{P}_{2}(X)$ by Gaussian-Tail Extensions: the Gaussian Reference Measure, Marginals, Densities, Relative Entropy, Relative Score and the Noise Wasserstein Distance §second-moment, in particular in $\mathcal{P}(X)$. Since $\sigma$ has finite relative entropy with respect to $\tilde{\gamma}_{n}$, $E_{n}(\sigma)$ has finite relative entropy with respect to $\gamma_{c}$ by Gaussian-Tail Extensions: the Gaussian Reference Measure, Marginals, Densities, Relative Entropy, Relative Score and the Noise Wasserstein Distance §entropy, so $E_{n}(\sigma)\in\mathcal{D}$ by The Gaussian Entropy Pair on the Noise Wasserstein Space: Relative Entropy and the Noise Score Field §penalty-domain. Since $\sigma$ has finite Fisher information relative to $\tilde{\gamma}_{n}$, $E_{n}(\sigma)$ has a relative score with respect to $\gamma_{c}$ by Gaussian-Tail Extensions: the Gaussian Reference Measure, Marginals, Densities, Relative Entropy, Relative Score and the Noise Wasserstein Distance §score and finite Fisher information relative to $\gamma_{c}$ with weights $a$ by Gaussian-Tail Extensions: the Gaussian Reference Measure, Marginals, Densities, Relative Entropy, Relative Score and the Noise Wasserstein Distance §fisher. Hence $E_{n}(\sigma)\in\mathcal{D}_{\Sigma}$ by The Gaussian Entropy Pair on the Noise Wasserstein Space: Relative Entropy and the Noise Score Field §score-domain.

Step 1 (clause 1). Let $\mu\in\mathcal{D}$. Then $\mu$ is a probability measure on $(X,\mathcal{B}(X))$ of finite relative entropy with respect to the probability measure $\gamma_{c}$, so $0\le H(\mu\,|\,\gamma_{c})$ by the Gibbs inequality Relative Entropy on a Measurable Space: the Gibbs Inequality, the Variational Criterion and Formula, the Entropy Inequality, Small Sets and Data Processing §gibbs. Since $\beta>0$, $0\le\beta\,H(\mu\,|\,\gamma_{c})=\mathcal{E}(\mu)$ by The Gaussian Entropy Pair on the Noise Wasserstein Space: Relative Entropy and the Noise Score Field §pair.

Step 2 (clause 2). Let $\mu\in\mathcal{D}$ and let $\varepsilon\in\mathbb{R}$ be positive. By the hypothesis $c_{k}\le\kappa a_{k}$ and Tail Replacement in the Noise Wasserstein Distance: a Measure of Finite Relative Entropy is Approximated by the Gaussian-Tail Extensions of Its Rescaled Heads §convergence, $W_{a}(\mu,E_{n}(\tilde{\mu}_{n}))\to0$ as $n\to\infty$; fix $n\in\mathbb{N}$ with $W_{a}(\mu,E_{n}(\tilde{\mu}_{n}))<\varepsilon/2$, where $\mu,E_{n}(\tilde{\mu}_{n})\in\mathcal{P}^{a}_{\rho}$ by Tail Replacement in the Noise Wasserstein Distance: a Measure of Finite Relative Entropy is Approximated by the Gaussian-Tail Extensions of Its Rescaled Heads §membership. Since $\mu\in\mathcal{P}_{2}(X)$ by The Gaussian Entropy Pair on the Noise Wasserstein Space: Relative Entropy and the Noise Score Field §penalty-domain, the rescaled head $\tilde{\mu}_{n}$ lies in $\mathcal{P}_{2}(\mathbb{R}^{n})$ by Lifting Euclidean Tangent Fields of the Rescaled Head to Noise Tangent Fields on a Hilbert Space §head, and it has finite relative entropy with respect to $\tilde{\gamma}_{n}$ by Gaussian-Tail Extensions: the Gaussian Reference Measure, Marginals, Densities, Relative Entropy, Relative Score and the Noise Wasserstein Distance §heads; thus $\tilde{\mu}_{n}\in\mathcal{D}^{(n)}$. By the density condition Penalty Pairs on the Wasserstein Space: the Penalty, Its Score, and Their Domains §dense of the penalty pair $(\mathcal{D}^{(n)},\mathcal{D}^{(n)}_{\Sigma},\mathcal{E}^{(n)},\Sigma^{(n)})$, applied to $\tilde{\mu}_{n}$ and $\varepsilon/2$, there is $\sigma\in\mathcal{D}^{(n)}_{\Sigma}$ with $W_{2}(\sigma,\tilde{\mu}_{n})<\varepsilon/2$. Put $\nu=E_{n}(\sigma)$; then $\nu\in\mathcal{D}_{\Sigma}$ by Step 0. Since $\sigma,\tilde{\mu}_{n}\in\mathcal{P}_{2}(\mathbb{R}^{n})$, Gaussian-Tail Extensions: the Gaussian Reference Measure, Marginals, Densities, Relative Entropy, Relative Score and the Noise Wasserstein Distance §distance gives $\nu,E_{n}(\tilde{\mu}_{n})\in\mathcal{P}^{a}_{\rho}$ and $W_{a}(\nu,E_{n}(\tilde{\mu}_{n}))\le W_{2}(\sigma,\tilde{\mu}_{n})<\varepsilon/2$. By the triangle inequality The Noise Wasserstein Distance is a Metric on the Measures Noise-Connected to the Reference Measure: Existence of Noise-Optimal Couplings, Comparison with the Quadratic Wasserstein Distance and Lower Semicontinuity §triangle and the symmetry The Noise Wasserstein Distance is a Metric on the Measures Noise-Connected to the Reference Measure: Existence of Noise-Optimal Couplings, Comparison with the Quadratic Wasserstein Distance and Lower Semicontinuity §symmetry for the three members $\nu$, $E_{n}(\tilde{\mu}_{n})$, $\mu$ of $\mathcal{P}^{a}_{\rho}$,

$$W_{a}(\nu,\mu)\le W_{a}(\nu,E_{n}(\tilde{\mu}_{n}))+W_{a}(\mu,E_{n}(\tilde{\mu}_{n}))<\varepsilon/2+\varepsilon/2=\varepsilon .$$

Step 3 (clause 3). By The Gaussian Entropy Pair on the Noise Wasserstein Space: Relative Entropy and the Noise Score Field §pair, $\mathcal{D}_{\Sigma}\subseteq\mathcal{D}\subseteq\mathcal{P}^{a}_{\rho}$, $\mathcal{E}$ is a real-valued function on $\mathcal{D}$, and $\Sigma(\mu)=\beta Z^{a}_{\mu}\in T^{a}_{\mu}$ for every $\mu\in\mathcal{D}_{\Sigma}$ by The Gaussian Entropy Pair on the Noise Wasserstein Space: Relative Entropy and the Noise Score Field §score-domain; the setting A Diagonal Gaussian Reference Measure on the Noise Wasserstein Space, Rescaled Heads and Gaussian Tails: Standing Notation §background carries that of Probability Measures on a Hilbert Space Transported in the Noise Norm: Standing Notation, and A Diagonal Gaussian Reference Measure on the Noise Wasserstein Space, Rescaled Heads and Gaussian Tails: Standing Notation §gaussian fixes the reference measure $\rho=\gamma_{c}$. We verify the four conditions of Noise Penalty Pairs on the Noise Wasserstein Space: the Penalty, Its Score, and Their Domains §pair.

Nonempty score domain (Noise Penalty Pairs on the Noise Wasserstein Space: the Penalty, Its Score, and Their Domains §nonempty). The score domain $\mathcal{D}^{(1)}_{\Sigma}$ is nonempty by Penalty Pairs on the Wasserstein Space: the Penalty, Its Score, and Their Domains §nonempty, the quadruple $(\mathcal{D}^{(1)},\mathcal{D}^{(1)}_{\Sigma},\mathcal{E}^{(1)},\Sigma^{(1)})$ being a penalty pair. For any $\sigma\in\mathcal{D}^{(1)}_{\Sigma}$, Step 0 gives $E_{1}(\sigma)\in\mathcal{D}_{\Sigma}$.

Lower bound (Noise Penalty Pairs on the Noise Wasserstein Space: the Penalty, Its Score, and Their Domains §bound). Take $C=0$. For $\mu\in\mathcal{D}$, $-0\cdot(1+W_{a}(\mu,\rho)^{2})=0\le\mathcal{E}(\mu)$ by Step 1.

First variation (Noise Penalty Pairs on the Noise Wasserstein Space: the Penalty, Its Score, and Their Domains §variation). Let $\mu\in\mathcal{D}_{\Sigma}$ and $\psi\in\mathcal{F}C^{2}_{b}(X)$. By Bounded C^2 Cylindrical Functions on a Hilbert Space with an Orthonormal Basis §cylindrical there are $n\in\mathbb{N}$ and $g\in C^{2}_{b}(\mathbb{R}^{n})$ with $\psi=g\circ p_{n}$. Since $\mu\in\mathcal{D}$ has finite relative entropy with respect to $\gamma_{c}$, Relative Entropy along a Noise-Gradient Perturbation of the Identity: the Formula, the Second-Order Expansion and the First Variation §variation, applied to $\mu$, $n$ and $g$, gives a positive $t_{0}\in\mathbb{R}$ such that for every $t\in(-t_{0},t_{0})$ the probability measure $\mu_{t}=(\mathrm{id}+t\nabla_{a}\psi)_{\#}\mu$ has finite relative entropy with respect to $\gamma_{c}$, hence lies in $\mathcal{D}$ by The Gaussian Entropy Pair on the Noise Wasserstein Space: Relative Entropy and the Noise Score Field §penalty-domain, and such that $f:(-t_{0},t_{0})\to\mathbb{R}$, $f(t)=H(\mu_{t}\,|\,\gamma_{c})$, is differentiable at $0$ with derivative $L^{a}_{\mu}(g)$. Here $\mathrm{id}+t\nabla_{a}\psi$ is in both places the map $x\mapsto x+t\,\nabla_{a}\psi(x)$, by the preambles of that lemma and of Noise Penalty Pairs on the Noise Wasserstein Space: the Penalty, Its Score, and Their Domains. The function $t\mapsto\mathcal{E}(\mu_{t})$ on $(-t_{0},t_{0})$ is $\beta f$, and $0$ is an interior point of the interval $(-t_{0},t_{0})$, as recorded in Noise Penalty Pairs on the Noise Wasserstein Space: the Penalty, Its Score, and Their Domains §variation; so by the constant-multiple rule, claim 2 of Sum, Constant Multiple, and Product Rules for One-Dimensional Derivatives, it is differentiable at $0$ with derivative $\beta L^{a}_{\mu}(g)$. Since $\mu\in\mathcal{P}_{2}(X)$ has a relative score with respect to $\gamma_{c}$ and finite Fisher information relative to $\gamma_{c}$ with weights $a$, The Noise Score Field of a Measure of Finite Weighted Fisher Information: Existence, Norm, Pairing with Noise Gradients, Head Approximation and Tangency §pairing-functional gives $L^{a}_{\mu}(g)=\langle Z^{a}_{\mu},\nabla_{a}\psi\rangle_{\mu}$. By linearity of the inner product of the real Hilbert space $L^{2}(\mu;X^{a})$ of Probability Measures on a Hilbert Space Transported in the Noise Norm: Standing Notation §fields,

$$\beta L^{a}_{\mu}(g)=\beta\langle Z^{a}_{\mu},\nabla_{a}\psi\rangle_{\mu}=\langle\beta Z^{a}_{\mu},\nabla_{a}\psi\rangle_{\mu}=\langle\Sigma(\mu),\nabla_{a}\psi\rangle_{\mu},$$

which is the required derivative.

Density (Noise Penalty Pairs on the Noise Wasserstein Space: the Penalty, Its Score, and Their Domains §dense). This is Step 2.

Hence $(\mathcal{D},\mathcal{D}_{\Sigma},\mathcal{E},\Sigma)$ is a noise penalty pair on $\mathcal{P}^{a}_{\rho}$.