Gibbs Maximisers of the Relative Free Energy: the Variational Principle and the Relative Score

In the setting of Square-Integrable Vector Fields Against a Probability Measure on Euclidean Space, and Test Functions: Standing Notation, let $\lambda_{d}$ be Lebesgue measure on $\mathcal{B}(\mathbb{R}^{d})$, with its null sets; a function on $\mathbb{R}^{d}$ is integrable with respect to $\lambda_{d}$ in the sense of Integrable Function and the Lebesgue Integral, and integrable with respect to a probability measure on $\mathbb{R}^{d}$ is as in Probability Measures on Euclidean Space and Random Vectors: Standing Notation §measures. $\exp$ is the exponential function and $\log$ the natural logarithm. For a Borel $\rho:\mathbb{R}^{d}\to[0,\infty)$ with $\int_{\mathbb{R}^{d}}\rho\,d\lambda_{d}=1$, the measure with density $\rho$ is the measure with density $\rho$ with respect to $\lambda_{d}$ of claim 3 of that lemma.

Let $D\subseteq\mathbb{R}^{d}$ be open, convex and nonempty, and let $U:D\to\mathbb{R}$ be a penalty on $D$; thus $U$ is of class $C^{2}$ on $D$ by the regularity clause of that definition, and we write $DU(x)$ for its gradient at $x\in D$. Assume that $U$ has a monotone gradient:

Let $p_{0}=\inf_{x\in D}U(x)$, a real number by the lower bound for a penalty, and let $a\in\mathbb{R}$ be positive. The maps $\bar{U}$ and $\nabla U$, the set $\mathcal{D}_{U,a}$ and the relative free energy $\mathcal{E}_{U,a}$, and the set $\mathcal{D}^{\Sigma}_{U,a}$ and the relative score $\Sigma_{U,a}$ are those of The Relative Free Energy and the Relative Score of a Probability Measure for a Potential on an Open Set for these $D$, $U$ and $a$. Assume the Gibbs integrability condition: the function $G:\mathbb{R}^{d}\to[0,\infty)$ given by

which is Borel (Step 1 of the proof), is integrable with respect to $\lambda_{d}$.

Let $f:D\to\mathbb{R}$ be semiconvex on $D$ with a constant $K\ge0$ and bounded above; $f$ is continuous on $D$ by the continuity of semiconvex functions. At a point $x\in D$ at which $f$ is differentiable, the partial derivatives $\partial_{i}f(x)$ exist and form the derivative matrix of $f$ at $x$ by A Derivative Matrix is the Jacobian Matrix, and is Unique, and $Df(x)\in\mathbb{R}^{d}$ is the gradient of $f$ at $x$. Assume that there are real numbers $A,B\ge0$ with

Let $\bar{f}:\mathbb{R}^{d}\to\mathbb{R}$ be the map equal to $f$ on $D$ and to $0$ off $D$, and let $\gamma:\mathbb{R}^{d}\to[0,\infty)$ be given by $\gamma(x)=\exp\bigl((f(x)-U(x))/a\bigr)$ for $x\in D$ and $\gamma(x)=0$ for $x\notin D$; both are Borel (Step 1 of the proof), and $Z=\int_{\mathbb{R}^{d}}\gamma\,d\lambda_{d}\in[0,\infty]$. When $0<Z<\infty$, which is the case by clause 1, $\pi_{f}$ denotes the measure with density $Z^{-1}\gamma$. Gradient maps of $f$ are those of Gradient Maps of a Function on an Open Subset of Euclidean Space for this $D$ and $f$.

1. (The Gibbs measure)¶ $0<Z<\infty$; $\pi_{f}$ is a probability measure on $\mathbb{R}^{d}$ with $\pi_{f}(D)=1$ and $\pi_{f}\in\mathcal{D}_{U,a}$; $\bar{f}$ is integrable with respect to $\pi_{f}$; and

2. (The variational principle)¶ For every $\mu\in\mathcal{D}_{U,a}$ such that $\bar{f}$ is integrable with respect to $\mu$,

Consequently, by clause 1, $\pi_{f}$ maximises $\mu\mapsto\int_{\mathbb{R}^{d}}\bar{f}\,d\mu-\mathcal{E}_{U,a}(\mu)$ over these $\mu$.

3. (The relative score)¶ The function $f$ has a gradient map, and $\pi_{f}\in\mathcal{D}^{\Sigma}_{U,a}$. For every gradient map $\nabla f$ of $f$ one has $\int_{\mathbb{R}^{d}}\lVert\nabla f\rVert^{2}\,d\pi_{f}<\infty$, so that the class of $\nabla f$ lies in $L^{2}(\pi_{f};\mathbb{R}^{d})$ and is again written $\nabla f$; the score of $\pi_{f}$ is $\xi_{\pi_{f}}=a^{-1}(\nabla f-\nabla U)$; and