Relative Entropy of the Finite-Dimensional Projections of Borel Probability Measures on a Hilbert Space: Monotonicity and Approximation

In the setting of Borel Probability Measures on a Real Hilbert Space with an Orthonormal Basis: Standing Notation, with the coordinate maps $p_{n}$ of Borel Probability Measures on a Real Hilbert Space with an Orthonormal Basis: Standing Notation §coordinates, which are Borel by Borel Sets of a Hilbert Space with an Orthonormal Basis: Coordinates, Determination by Finite-Dimensional Projections, and Pairs §continuity, and push-forwards as in Borel Probability Measures on a Real Hilbert Space with an Orthonormal Basis: Standing Notation §pushforward, let $\mu,\gamma\in\mathcal{P}(X)$, and for $n\in\mathbb{N}$ let $\mu_{n}=(p_{n})_{\#}\mu$ and $\gamma_{n}=(p_{n})_{\#}\gamma$, Borel probability measures on $\mathbb{R}^{n}$. Finite relative entropy and $H(\cdot\,|\,\cdot)$ are those of that definition on the measurable spaces $(X,\mathcal{B}(X))$ and $(\mathbb{R}^{n},\mathcal{B}(\mathbb{R}^{n}))$.

1. (Monotonicity in the dimension)¶ Let $m,n\in\mathbb{N}$ with $m\le n$. If $\mu_{n}$ has finite relative entropy with respect to $\gamma_{n}$, then $\mu_{m}$ has finite relative entropy with respect to $\gamma_{m}$, and $H(\mu_{m}\,|\,\gamma_{m})\le H(\mu_{n}\,|\,\gamma_{n})$.

2. (Projections)¶ If $\mu$ has finite relative entropy with respect to $\gamma$, then for every $n\in\mathbb{N}$ the measure $\mu_{n}$ has finite relative entropy with respect to $\gamma_{n}$, and $H(\mu_{n}\,|\,\gamma_{n})\le H(\mu\,|\,\gamma)$.

3. (Bounded projections)¶ Let $C\in\mathbb{R}$ be such that for every $n\in\mathbb{N}$ the measure $\mu_{n}$ has finite relative entropy with respect to $\gamma_{n}$ and $H(\mu_{n}\,|\,\gamma_{n})\le C$. Then $\mu$ has finite relative entropy with respect to $\gamma$, and $H(\mu\,|\,\gamma)\le C$.

4. (Limit)¶ If $\mu$ has finite relative entropy with respect to $\gamma$, then the sequence $\bigl(H(\mu_{n}\,|\,\gamma_{n})\bigr)_{n\in\mathbb{N}}$ is nondecreasing and converges to $H(\mu\,|\,\gamma)$.