Relative Entropy and Relative Score with Respect to a Diagonal Gaussian Measure: Comparison with the Entropy, the Score and the Relative Free Energy of a Quadratic Potential

In the settings of The Real Numbers: Standing Notation and Background and The Intrinsic Calculus on the Wasserstein Space: Standing Notation, let $c$ be a variance vector, with scaling map $S_{c}$ and weighted square $|x|_{c}^{2}$ ($x\in\mathbb{R}^{d}$) and normalizing constant $Z_{c}$, let $\gamma_{c}$ be the diagonal Gaussian measure with variances $c$, and for $\mu\in\mathcal{P}_{2}(\mathbb{R}^{d})$ let the class of $S_{c}$ in $L^{2}(\mu;\mathbb{R}^{d})$ be as in Finite Fisher Information Relative to a Diagonal Gaussian Measure: the Ornstein-Uhlenbeck Functional, the Relative Score and the Relative Fisher Information; $T_{\mu}$ is the tangent space. Finite relative entropy with respect to $\gamma_{c}$ and $H(\cdot\,|\,\gamma_{c})$ are those of that definition on $(\mathbb{R}^{d},\mathcal{B}(\mathbb{R}^{d}))$; finite entropy, $\mathrm{Ent}$ and $\mathcal{P}_{2}^{\mathrm{Ent}}(\mathbb{R}^{d})$ are those of that definition; finite Fisher information, $\mathcal{P}_{2}^{\mathcal{I}}(\mathbb{R}^{d})$ and the score $\xi_{\mu}$ are those of that definition; finite Fisher information relative to $\gamma_{c}$ and the relative score $\zeta^{c}_{\mu}$ are those of that definition; integrable is as in Probability Measures on Euclidean Space and Random Vectors: Standing Notation §measures. Let $\mu\in\mathcal{P}_{2}(\mathbb{R}^{d})$.

1. (Integrability)¶ The function $x\mapsto|x|_{c}^{2}$ is integrable with respect to $\mu$.

2. (Relative entropy and entropy)¶ The measure $\mu$ has finite relative entropy with respect to $\gamma_{c}$ if and only if it has finite entropy, and in that case

$$H(\mu\,|\,\gamma_{c})=\mathrm{Ent}(\mu)+\tfrac12\int_{\mathbb{R}^{d}}|x|_{c}^{2}\,\mu(dx)+Z_{c}.$$

3. (The scaling map is tangent)¶ The class $S_{c}$ belongs to $T_{\mu}$.

4. (Relative score and score)¶ The measure $\mu$ has finite Fisher information relative to $\gamma_{c}$ if and only if $\mu\in\mathcal{P}_{2}^{\mathcal{I}}(\mathbb{R}^{d})$, and in that case $\zeta^{c}_{\mu}=\xi_{\mu}+S_{c}$ in $T_{\mu}$.

For clauses 5 and 6, let $a\in\mathbb{R}$ be positive and let $V_{c}:\mathbb{R}^{d}\to\mathbb{R}$, $V_{c}(x)=\tfrac{a}{2}|x|_{c}^{2}$, a finite sum of the quadratic monomials $x\mapsto\tfrac{a}{2c_{i}}x_{i}^{2}$, hence of class $C^{2}$ on $\mathbb{R}^{d}$ with gradient $DV_{c}(x)=a\,S_{c}(x)$ by Quadratic and Affine Functions of Class $C^2$, Translation, and Quadratic Perturbation of Semiconvexity §quadratic, applied to the symmetric diagonal matrix with diagonal entries $a/c_{1},\dots,a/c_{d}$, linear coefficient $0$ and constant term $0$, so that the class of its gradient map in $L^{2}(\mu;\mathbb{R}^{d})$ is $a\,S_{c}$. Let $\mathcal{D}_{V_{c},a}$, $\mathcal{E}_{V_{c},a}$, $\mathcal{D}^{\Sigma}_{V_{c},a}$ and $\Sigma_{V_{c},a}$ be the set, the relative free energy, the set and the relative score of that definition for the open set $\mathbb{R}^{d}$, the potential $V_{c}$ and the temperature $a$.

5. (The relative free energy of the quadratic potential)¶ $\mu\in\mathcal{D}_{V_{c},a}$ if and only if $\mu$ has finite relative entropy with respect to $\gamma_{c}$, and then $\mathcal{E}_{V_{c},a}(\mu)=a\,H(\mu\,|\,\gamma_{c})-a\,Z_{c}$.

6. (The relative score of the quadratic potential)¶ $\mu\in\mathcal{D}^{\Sigma}_{V_{c},a}$ if and only if $\mu$ has finite relative entropy with respect to $\gamma_{c}$ and finite Fisher information relative to $\gamma_{c}$, and then $\Sigma_{V_{c},a}(\mu)=a\,\zeta^{c}_{\mu}$.