Relative Entropy on Euclidean Space: the Gibbs Inequality, a Variational Criterion for Finite Relative Entropy, and Closed Sublevel Sets under Weak Convergence

In the setting of Probability Measures on Euclidean Space and Random Vectors: Standing Notation, let $m$ be a natural number with $1\le m$ and let $\gamma\in\mathcal{P}(\mathbb{R}^{m})$. Finite relative entropy with respect to $\gamma$ and $H(\cdot\,|\,\gamma)$ are those of that definition, for the measurable space $(\mathbb{R}^{m},\mathcal{B}(\mathbb{R}^{m}))$; weak convergence $\nu_{n}\Rightarrow\nu$ is taken in the metric space $(\mathbb{R}^{m},d_{E})$; $\exp$ is the exponential function and $\log$ the natural logarithm. A function $h:\mathbb{R}^{m}\to\mathbb{R}$ is called bounded Lipschitz if it is bounded and Lipschitz with a constant for $d_{E}$ and the absolute-value metric. For a bounded Borel $h:\mathbb{R}^{m}\to\mathbb{R}$ and $\nu\in\mathcal{P}(\mathbb{R}^{m})$, $h$ is integrable with respect to $\nu$ by Probability Measures on Euclidean Space and Random Vectors: Standing Notation §measures, and, $\int_{\mathbb{R}^{m}}\exp\circ h\,d\gamma$ being a positive real number by claim 1 below, we write

1. (The exponential moment)¶ For every bounded Borel $h:\mathbb{R}^{m}\to\mathbb{R}$ the function $\exp\circ h$ is Borel and bounded, and $\int_{\mathbb{R}^{m}}\exp\circ h\,d\gamma$ is a positive real number.

2. (Gibbs inequality)¶ Let $\nu\in\mathcal{P}(\mathbb{R}^{m})$ have finite relative entropy with respect to $\gamma$. Then $\Lambda_{h}(\nu)\le H(\nu\,|\,\gamma)$ for every bounded Borel $h:\mathbb{R}^{m}\to\mathbb{R}$; in particular $0\le H(\nu\,|\,\gamma)$.

3. (Variational criterion)¶ Let $\nu\in\mathcal{P}(\mathbb{R}^{m})$ and $c\in\mathbb{R}$ be such that $\Lambda_{h}(\nu)\le c$ for every bounded Lipschitz $h:\mathbb{R}^{m}\to\mathbb{R}$. Then $\nu$ has finite relative entropy with respect to $\gamma$, and $H(\nu\,|\,\gamma)\le c$.

4. (Closed sublevel sets)¶ Let $c\in\mathbb{R}$, let $\nu\in\mathcal{P}(\mathbb{R}^{m})$, and let $(\nu_{n})_{n\in\mathbb{N}}$ be a sequence in $\mathcal{P}(\mathbb{R}^{m})$ such that each $\nu_{n}$ has finite relative entropy with respect to $\gamma$ with $H(\nu_{n}\,|\,\gamma)\le c$, and $\nu_{n}\Rightarrow\nu$. Then $\nu$ has finite relative entropy with respect to $\gamma$, and $H(\nu\,|\,\gamma)\le c$.