Proof of The Gibbs Entropy Pair is Noise-Closed, and Its Penalty Bounds the Squared Noise Wasserstein Distance to the Reference Measure

Each result cited is universally quantified over the data in its own statement.

Real order and arithmetic are those of The Real Numbers: Standing Notation and Background §background. Throughout, $\rho=\gamma_{c}$ as in A Diagonal Gaussian Reference Measure on the Noise Wasserstein Space, Rescaled Heads and Gaussian Tails: Standing Notation §gaussian. By The Gibbs Entropy Pair on the Noise Wasserstein Space: Relative Entropy to the Gibbs Measure and the Gibbs Score Field §penalty-domain and The Gibbs Entropy Pair on the Noise Wasserstein Space: Relative Entropy to the Gibbs Measure and the Gibbs Score Field §pair, $\mathcal{D}$ is the set of the $\mu\in\mathcal{P}(X)$ that have finite relative entropy with respect to $\gamma^{V}_{\beta}$, $\mathcal{D}\subseteq\mathcal{P}^{a}_{\rho}$, and $\mathcal{E}(\mu)=\beta\,H(\mu\,|\,\gamma^{V}_{\beta})$ for $\mu\in\mathcal{D}$. $Z_{V,\beta}$ is a positive real number by The Gibbs Measure of an Admissible Cylindrical Potential Relative to a Diagonal Gaussian Measure §normaliser, so $\log Z_{V,\beta}$ is a real number. The letter $\ell$ denotes the real number quantified as $c$ in Noise-Closed Noise Penalty Pairs §noise-closed.

Step 1 (the basic estimate). Let $\mu\in\mathcal{D}$. By Entropy and Score Relative to the Gibbs Measure Split into Their Gaussian Parts and the Potential, with a Fisher Information Bound §entropy, $\mu$ has finite relative entropy with respect to $\gamma_{c}$, $V$ is integrable with respect to $\mu$, and

$$H(\mu\,|\,\gamma^{V}_{\beta})=H(\mu\,|\,\gamma_{c})+\frac{1}{\beta}\int_{X}V\,d\mu+\log Z_{V,\beta}.$$

By Basic Properties of an Admissible Cylindrical Potential: Continuity, Growth under Noise Translations, Integrability, the Tangent Inequality and Tangency of the Noise Gradient §continuity, $-b\le V(x)$ for every $x\in X$; the constant function $-b$ is integrable with respect to the probability measure $\mu$ with integral $-b$, so the monotonicity of Linearity and Monotonicity of the Lebesgue Integral §integrable gives $-b\le\int_{X}V\,d\mu$. Since $\beta>0$, $-\frac{1}{\beta}\int_{X}V\,d\mu\le\frac{b}{\beta}$; and $H(\mu\,|\,\gamma^{V}_{\beta})=\mathcal{E}(\mu)/\beta$, since $\mathcal{E}=\beta H(\cdot\,|\,\gamma^{V}_{\beta})$ on $\mathcal{D}$ by The Gibbs Entropy Pair on the Noise Wasserstein Space: Relative Entropy to the Gibbs Measure and the Gibbs Score Field §pair. Hence

$$H(\mu\,|\,\gamma_{c})=H(\mu\,|\,\gamma^{V}_{\beta})-\frac{1}{\beta}\int_{X}V\,d\mu-\log Z_{V,\beta}\le\frac{1}{\beta}\Bigl(\mathcal{E}(\mu)+b-\beta\log Z_{V,\beta}\Bigr).\tag{1}$$

Since $\mu\in\mathcal{P}(X)$ has finite relative entropy with respect to $\gamma_{c}$, $\rho=\gamma_{c}$ and $c_{k}\le\kappa\,a_{k}$ for every $k\in\mathbb{N}$, Talagrand's inequality in the noise norm (in the setting Probability Measures on a Hilbert Space Transported in the Noise Norm: Standing Notation, which A Diagonal Gaussian Reference Measure on the Noise Wasserstein Space, Rescaled Heads and Gaussian Tails: Standing Notation §background carries) gives $W_{a}(\mu,\rho)^{2}\le2\kappa\,H(\mu\,|\,\gamma_{c})$. Multiplying (1) by the positive number $2\kappa$,

$$W_{a}(\mu,\rho)^{2}\le\frac{2\kappa}{\beta}\Bigl(\mathcal{E}(\mu)+b-\beta\log Z_{V,\beta}\Bigr)\qquad(\mu\in\mathcal{D}).\tag{2}$$

This is claim 2.

Step 2 (closed sublevel sets). Let $\ell\in\mathbb{R}$, and let $(\mu_{j})_{j\in\mathbb{N}}$ be a sequence in $\mathcal{D}$ with $\mathcal{E}(\mu_{j})\le\ell$ for every $j$ that converges in $(\mathcal{P}^{a}_{\rho},W_{a})$ to some $\mu\in\mathcal{P}^{a}_{\rho}$; by Convergent Sequence in a Metric Space, for every real $\varepsilon>0$ there is $N\in\mathbb{N}$ with $W_{a}(\mu_{j},\mu)<\varepsilon$ for all $j\ge N$. All $\mu_{j}$ and $\mu$ lie in $\mathcal{P}^{a}_{\rho}\subseteq\mathcal{P}_{2}(X)$ by The Noise Wasserstein Distance is a Metric on the Measures Noise-Connected to the Reference Measure: Existence of Noise-Optimal Couplings, Comparison with the Quadratic Wasserstein Distance and Lower Semicontinuity §moments, and $W_{2}(\mu_{j},\mu)\le\sqrt{\bar{a}}\,W_{a}(\mu_{j},\mu)$ for every $j$ by The Noise Wasserstein Distance is a Metric on the Measures Noise-Connected to the Reference Measure: Existence of Noise-Optimal Couplings, Comparison with the Quadratic Wasserstein Distance and Lower Semicontinuity §comparison, where $\bar{a}>0$ by Probability Measures on a Hilbert Space Transported in the Noise Norm: Standing Notation §weights, so that its square root $\sqrt{\bar{a}}$ of Existence and Uniqueness of the Nonnegative Square Root is positive. Given a real $\varepsilon>0$, choose $N$ for the positive number $\varepsilon/\sqrt{\bar{a}}$; then $W_{2}(\mu_{j},\mu)<\varepsilon$ for all $j\ge N$, so $\lim_{j}W_{2}(\mu_{j},\mu)=0$ in the sense of Limit of a Sequence of Real Numbers, the distances being nonnegative. By Wasserstein Convergence on a Hilbert Space: Weak Convergence, Integrals of Continuous Functions of Quadratic Growth, Convergence from Weak Convergence with Uniformly Integrable Second Moments, and Compactness §weak, $\mu_{j}\Rightarrow\mu$, weak convergence being that of Weak Convergence of Finite Borel Measures on a Metric Space by Borel Probability Measures on a Real Hilbert Space with an Orthonormal Basis: Standing Notation §weak.

We apply Relative Entropy on a Metric Space: the Variational Criterion over Bounded Lipschitz Functions and Sequentially Closed Sublevel Sets under Weak Convergence §closed with the metric space $(S,d)=(X,d)$, which is nonempty as it contains $0_{X}$, with $\gamma=\gamma^{V}_{\beta}$, a Borel measure on $(X,d)$ with $\gamma^{V}_{\beta}(X)=1$ by The Gibbs Measure of an Admissible Cylindrical Potential Relative to a Diagonal Gaussian Measure §gibbs, with $\nu_{j}=\mu_{j}$ and $\nu=\mu$, Borel measures on $(X,d)$ of total mass $1$ as members of $\mathcal{P}(X)$ (Borel Probability Measures on a Real Hilbert Space with an Orthonormal Basis: Standing Notation §measures), and with $C=\ell/\beta$: each $\mu_{j}$ has finite relative entropy with respect to $\gamma^{V}_{\beta}$, and $\beta\,H(\mu_{j}\,|\,\gamma^{V}_{\beta})=\mathcal{E}(\mu_{j})\le\ell$ gives $H(\mu_{j}\,|\,\gamma^{V}_{\beta})\le\ell/\beta$ because $\beta>0$. That claim yields that $\mu$ has finite relative entropy with respect to $\gamma^{V}_{\beta}$ and $H(\mu\,|\,\gamma^{V}_{\beta})\le\ell/\beta$. Since $\mu\in\mathcal{P}(X)$, $\mu\in\mathcal{D}$ by The Gibbs Entropy Pair on the Noise Wasserstein Space: Relative Entropy to the Gibbs Measure and the Gibbs Score Field §penalty-domain, and $\mathcal{E}(\mu)=\beta\,H(\mu\,|\,\gamma^{V}_{\beta})\le\ell$. This is Noise-Closed Noise Penalty Pairs §closed for $\ell$.

Step 3 (bounded sublevel sets). Let $\ell\in\mathbb{R}$, let $m$ be the larger of the real numbers $\frac{2\kappa}{\beta}\bigl(\ell+b-\beta\log Z_{V,\beta}\bigr)$ and $0$, and by Existence and Uniqueness of the Nonnegative Square Root let $B\ge0$ be the real number with $B^{2}=m$. Let $\mu\in\mathcal{D}$ with $\mathcal{E}(\mu)\le\ell$. By (2) and $\frac{2\kappa}{\beta}>0$,

$$W_{a}(\mu,\rho)^{2}\le\frac{2\kappa}{\beta}\Bigl(\mathcal{E}(\mu)+b-\beta\log Z_{V,\beta}\Bigr)\le\frac{2\kappa}{\beta}\Bigl(\ell+b-\beta\log Z_{V,\beta}\Bigr)\le m=B^{2}.$$

Here $W_{a}(\mu,\rho)\ge0$, since $W_{a}$ is a metric on $\mathcal{P}^{a}_{\rho}$ (The Noise Wasserstein Distance is a Metric on the Measures Noise-Connected to the Reference Measure: Existence of Noise-Optimal Couplings, Comparison with the Quadratic Wasserstein Distance and Lower Semicontinuity §metric) and $\mu,\rho\in\mathcal{P}^{a}_{\rho}$ (The Noise Wasserstein Distance is a Metric on the Measures Noise-Connected to the Reference Measure: Existence of Noise-Optimal Couplings, Comparison with the Quadratic Wasserstein Distance and Lower Semicontinuity §reference); with $B\ge0$, the weak form, claim 2, of Monotonicity of Squaring on the Nonnegative Elements of an Ordered Field gives $W_{a}(\mu,\rho)\le B$. This is Noise-Closed Noise Penalty Pairs §bounded for $\ell$.

Step 4 (conclusion). The quadruple is a noise penalty pair on $\mathcal{P}^{a}_{\rho}$ by The Gibbs Entropy Pair is a Noise Penalty Pair, with Nonnegative Penalty and Dense Score Domain §pair, and Steps 2 and 3 verify both conditions of Noise-Closed Noise Penalty Pairs §noise-closed for every $\ell\in\mathbb{R}$; so the pair is noise-closed, which is claim 1. Claim 2 is (2) of Step 1.