Proof of The Gaussian Entropy Pair is Noise-Closed, and Its Penalty Bounds the Squared Noise Wasserstein Distance to the Reference Measure

Each result cited is universally quantified over the data in its own statement.

Throughout, $\rho=\gamma_{c}$ as in A Diagonal Gaussian Reference Measure on the Noise Wasserstein Space, Rescaled Heads and Gaussian Tails: Standing Notation §gaussian, and by The Gaussian Entropy Pair on the Noise Wasserstein Space: Relative Entropy and the Noise Score Field §penalty-domain and The Gaussian Entropy Pair on the Noise Wasserstein Space: Relative Entropy and the Noise Score Field §pair the domain $\mathcal{D}$ is the set of the $\mu\in\mathcal{P}(X)$ with finite relative entropy with respect to $\gamma_{c}$, it satisfies $\mathcal{D}\subseteq\mathcal{P}^{a}_{\rho}$, and $\mathcal{E}(\mu)=\beta\,H(\mu\,|\,\gamma_{c})$ for $\mu\in\mathcal{D}$. For every $\mu\in\mathcal{D}$, Talagrand's inequality in the noise norm applies with the given $\kappa$, since $\rho=\gamma_{c}$, $c_{k}\le\kappa\,a_{k}$ for every $k\in\mathbb{N}$, and $\mu$ has finite relative entropy with respect to $\gamma_{c}$; hence

$$W_{a}(\mu,\rho)^{2}\le2\kappa\,H(\mu\,|\,\gamma_{c})=\frac{2\kappa}{\beta}\,\mathcal{E}(\mu)\qquad(\mu\in\mathcal{D}),\tag{$*$}$$

the equality because $\beta>0$.

Step 1 (closed sublevel sets). Let $b\in\mathbb{R}$ (the bound quantified as $c$ in Noise-Closed Noise Penalty Pairs §noise-closed). Let $(\mu_{j})_{j\in\mathbb{N}}$ be a sequence in $\mathcal{D}$ with $\mathcal{E}(\mu_{j})\le b$ for every $j$, converging in $(\mathcal{P}^{a}_{\rho},W_{a})$ to some $\mu\in\mathcal{P}^{a}_{\rho}$; by Convergent Sequence in a Metric Space, for every real $\varepsilon>0$ there is $N\in\mathbb{N}$ with $W_{a}(\mu_{j},\mu)<\varepsilon$ for all $j\ge N$. All $\mu_{j}$ and $\mu$ lie in $\mathcal{P}^{a}_{\rho}\subseteq\mathcal{P}_{2}(X)$ by The Noise Wasserstein Distance is a Metric on the Measures Noise-Connected to the Reference Measure: Existence of Noise-Optimal Couplings, Comparison with the Quadratic Wasserstein Distance and Lower Semicontinuity §moments, and $W_{2}(\mu_{j},\mu)\le\sqrt{\bar{a}}\,W_{a}(\mu_{j},\mu)$ for every $j$ by The Noise Wasserstein Distance is a Metric on the Measures Noise-Connected to the Reference Measure: Existence of Noise-Optimal Couplings, Comparison with the Quadratic Wasserstein Distance and Lower Semicontinuity §comparison, where $\bar{a}>0$, so that $\sqrt{\bar{a}}$, the square root of Existence and Uniqueness of the Nonnegative Square Root, is a positive real number. Given $\varepsilon>0$, first choose $N$ for the positive number $\varepsilon/\sqrt{\bar{a}}$; then $W_{2}(\mu_{j},\mu)<\varepsilon$ for all $j\ge N$, so $W_{2}(\mu_{j},\mu)\to0$. Since $\beta>0$, the bound $\beta\,H(\mu_{j}\,|\,\gamma_{c})\le b$ gives $H(\mu_{j}\,|\,\gamma_{c})\le b/\beta$, so each $\mu_{j}$ lies in the sublevel set $\mathcal{K}_{b/\beta}$ of Relative Entropy with Respect to a Diagonal Gaussian Measure on a Hilbert Space: the Moment Bound, the Cutoff Projections, and Bounded, Tight, Weakly Closed, Wasserstein-Closed and Weakly Sequentially Compact Sublevel Sets (the measures with finite relative entropy with respect to $\gamma_{c}$ and $H(\cdot\,|\,\gamma_{c})\le b/\beta$). By Relative Entropy with Respect to a Diagonal Gaussian Measure on a Hilbert Space: the Moment Bound, the Cutoff Projections, and Bounded, Tight, Weakly Closed, Wasserstein-Closed and Weakly Sequentially Compact Sublevel Sets §closed with $C=b/\beta$, also $\mu\in\mathcal{K}_{b/\beta}$: $\mu$ has finite relative entropy with respect to $\gamma_{c}$, so $\mu\in\mathcal{D}$ by The Gaussian Entropy Pair on the Noise Wasserstein Space: Relative Entropy and the Noise Score Field §penalty-domain, and $\mathcal{E}(\mu)=\beta\,H(\mu\,|\,\gamma_{c})\le b$. This is Noise-Closed Noise Penalty Pairs §closed for $b$.

Step 2 (bounded sublevel sets). Let $b\in\mathbb{R}$, and let $m$ be the larger of $2\kappa b/\beta$ and $0$. By Existence and Uniqueness of the Nonnegative Square Root there is a real $B\ge0$ with $B^{2}=m$. Let $\mu\in\mathcal{D}$ with $\mathcal{E}(\mu)\le b$. By $(*)$ and $2\kappa/\beta>0$, $W_{a}(\mu,\rho)^{2}\le(2\kappa/\beta)\,\mathcal{E}(\mu)\le2\kappa b/\beta\le m=B^{2}$. Since $W_{a}(\mu,\rho)\ge0$ (The Noise Wasserstein Distance is a Metric on the Measures Noise-Connected to the Reference Measure: Existence of Noise-Optimal Couplings, Comparison with the Quadratic Wasserstein Distance and Lower Semicontinuity §metric) and $B\ge0$, the weak form (claim 2) of Monotonicity of Squaring on the Nonnegative Elements of an Ordered Field gives $W_{a}(\mu,\rho)\le B$. This is Noise-Closed Noise Penalty Pairs §bounded for $b$. As $b$ was arbitrary, Steps 1 and 2 show that the pair is noise-closed, which is claim 1.

Step 3 (growth bound). Let $\mu\in\mathcal{D}$. The first inequality of claim 2 is $(*)$. By Properties of the Absolute Value in an Ordered Field §bounds, $\mathcal{E}(\mu)\le|\mathcal{E}(\mu)|\le1+|\mathcal{E}(\mu)|$, and multiplying by the positive number $2\kappa/\beta$ gives the second inequality.