Relative Entropy with Respect to a Diagonal Gaussian Measure on a Hilbert Space: the Moment Bound, the Cutoff Projections, and Bounded, Tight, Weakly Closed, Wasserstein-Closed and Weakly Sequentially Compact Sublevel Sets

In the setting of Borel Probability Measures on a Real Hilbert Space with an Orthonormal Basis: Standing Notation, with the coordinate maps $p_{n}$, push-forwards, weak convergence $\Rightarrow$ and tightness as in Borel Probability Measures on a Real Hilbert Space with an Orthonormal Basis: Standing Notation §coordinates, Borel Probability Measures on a Real Hilbert Space with an Orthonormal Basis: Standing Notation §pushforward and Borel Probability Measures on a Real Hilbert Space with an Orthonormal Basis: Standing Notation §weak, let $c$ be a variance sequence with truncations $c^{(n)}$ and diagonal Gaussian measures $\gamma_{c^{(n)}}$ on $\mathbb{R}^{n}$, and let $\gamma_{c}$ be the diagonal Gaussian measure on $X$ with variances $c$, so that $(p_{n})_{\#}\gamma_{c}=\gamma_{c^{(n)}}$ for every $n\in\mathbb{N}$ by that definition. Let $\bar{c}$ be the sum of the series $\sum_{k=1}^{\infty}c_{k}$, which converges by Variance Sequences and Their Truncations §variances; $\bar{c}$ is a positive real number, since $0<c_{1}\le\bar{c}$ by Series of Nonnegative Real Numbers, Comparison, and the Geometric Series §dominates. Finite relative entropy and $H(\cdot\,|\,\cdot)$ are those of that definition on the measurable spaces $(X,\mathcal{B}(X))$ and $(\mathbb{R}^{n},\mathcal{B}(\mathbb{R}^{n}))$; $\mathcal{P}_{2}(X)$ and $M_{2}$ are the set and the second moment of The Second Moment of a Borel Probability Measure on a Hilbert Space and the Probability Measures with Finite Second Moment §space and The Second Moment of a Borel Probability Measure on a Hilbert Space and the Probability Measures with Finite Second Moment §moment, and $W_{2}$ is the Wasserstein distance on $\mathcal{P}_{2}(X)$. For $C\in\mathbb{R}$ let $\mathcal{K}_{C}$ be the set of the $\mu\in\mathcal{P}(X)$ that have finite relative entropy with respect to $\gamma_{c}$ and satisfy $H(\mu\,|\,\gamma_{c})\le C$.

1. (Moment bound)¶ Let $\mu\in\mathcal{P}(X)$ have finite relative entropy with respect to $\gamma_{c}$. Then $\mu\in\mathcal{P}_{2}(X)$ and

$$M_{2}(\mu)\le4\bar{c}\,H(\mu\,|\,\gamma_{c})+2\bar{c}.$$

2. (Bounded sublevel sets)¶ Let $C\in\mathbb{R}$. Then $\mathcal{K}_{C}\subseteq\mathcal{P}_{2}(X)$, and $M_{2}(\mu)\le4\bar{c}\,C+2\bar{c}$ for every $\mu\in\mathcal{K}_{C}$.

3. (Cutoffs)¶ Let $\mu\in\mathcal{P}(X)$ and $\mu_{n}=(p_{n})_{\#}\mu$ for $n\in\mathbb{N}$. Then $\mu$ has finite relative entropy with respect to $\gamma_{c}$ if and only if there is $C\in\mathbb{R}$ such that for every $n\in\mathbb{N}$ the measure $\mu_{n}$ has finite relative entropy with respect to $\gamma_{c^{(n)}}$ and $H(\mu_{n}\,|\,\gamma_{c^{(n)}})\le C$; in that case the sequence $\bigl(H(\mu_{n}\,|\,\gamma_{c^{(n)}})\bigr)_{n\in\mathbb{N}}$ is nondecreasing and converges to $H(\mu\,|\,\gamma_{c})$.

4. (Tight sublevel sets)¶ For every $C\in\mathbb{R}$ the set $\mathcal{K}_{C}$ is tight.

5. (Weakly closed sublevel sets)¶ Let $C\in\mathbb{R}$, let $(\mu_{j})_{j\in\mathbb{N}}$ be a sequence in $\mathcal{K}_{C}$, and let $\mu\in\mathcal{P}(X)$ satisfy $\mu_{j}\Rightarrow\mu$. Then $\mu\in\mathcal{K}_{C}$.

6. (Wasserstein-closed sublevel sets)¶ Let $C\in\mathbb{R}$, let $(\mu_{j})_{j\in\mathbb{N}}$ be a sequence in $\mathcal{K}_{C}$, a subset of $\mathcal{P}_{2}(X)$ by claim 2, and let $\mu\in\mathcal{P}_{2}(X)$ satisfy $W_{2}(\mu_{j},\mu)\to0$ as $j\to\infty$. Then $\mu\in\mathcal{K}_{C}$.

7. (Weak sequential compactness)¶ Let $C\in\mathbb{R}$. Every sequence in $\mathcal{K}_{C}$ has a subsequence that converges weakly to an element of $\mathcal{K}_{C}$.