Proof of Relative Entropy and Relative Score with Respect to a Diagonal Gaussian Measure: Comparison with the Entropy, the Score and the Relative Free Energy of a Quadratic Potential

Each result cited below is universally quantified over the data in its own statement. Throughout, $\phi$ is the function $s\mapsto s\log s$ (with $\phi(0)=0$) of The Function $s\log s$: Continuity, Young's Inequality and Lower Bounds, with the Elementary Bounds for the Exponential and the Logarithm, densities are those of The Radon-Nikodym Theorem for a Finite Measure and a Sigma-Finite Measure, and Uniqueness of Densities, $\lambda_{d}$ is Lebesgue measure on $\mathcal{B}(\mathbb{R}^{d})$, $M_{2}$ is the second moment of The Second Moment of a Probability Measure on Euclidean Space and the Probability Measures with Finite Second Moment §moment, and $c_{\min}$ is the least variance of The Diagonal Gaussian Density on Euclidean Space and Its Notation §variances. For $x\in\mathbb{R}^{d}$ we put

$$h_{c}(x)=\tfrac12|x|_{c}^{2},\qquad \Phi_{c}(x)=-\tfrac12|x|_{c}^{2}-Z_{c},$$

so that $\rho_{c}(x)=\exp(\Phi_{c}(x))$ by The Diagonal Gaussian Density on Euclidean Space and Its Notation §density.

Step 0 (The quadratic form $h_{c}$). Let $M_{c}$ be the real $d\times d$ matrix with diagonal entries $1/c_{1},\dots,1/c_{d}$ and all other entries $0$; it equals its transpose, so $M_{c}\in\mathcal{S}(d)$ (Real Matrices, Symmetric Matrices and the Semidefinite Ordering: Standing Notation §symmetric), and for $x\in\mathbb{R}^{d}$ the $i$th entry of the matrix-vector product $M_{c}x$ is $x_{i}/c_{i}$, so that $M_{c}x=S_{c}(x)$ and $x\cdot(M_{c}x)=|x|_{c}^{2}$ by The Diagonal Gaussian Density on Euclidean Space and Its Notation §scaling. By Quadratic and Affine Functions of Class $C^2$, Translation, and Quadratic Perturbation of Semiconvexity §quadratic, with $n=d$, the matrix $M_{c}$, linear coefficient $0_{\mathbb{R}^{d}}$ and constant term $0$, the function $h_{c}$ is of class $C^{2}$ on $\mathbb{R}^{d}$ with gradient $Dh_{c}(x)=S_{c}(x)$ for every $x\in\mathbb{R}^{d}$. Since $\mathbb{R}^{d}$ is open in itself by claim 1 of Euclidean Space is Open in Itself, and $C^k$ Maps are Continuous, claims 2 and 3 of that lemma show that $h_{c}$ is of class $C^{1}$ on $\mathbb{R}^{d}$ and continuous on $\mathbb{R}^{d}$ as a map into $\mathbb{R}$ with the absolute-value metric, hence Borel by Probability Measures on Euclidean Space and Random Vectors: Standing Notation §borel-maps. Consequently $x\mapsto|x|_{c}^{2}=2h_{c}(x)$ and $\Phi_{c}=-h_{c}-Z_{c}$ are Borel by claims 1 and 2 of Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions. Finally, since $0<c_{\min}\le c_{i}$ for every $i\in[d]$, one has $0\le x_{i}^{2}/c_{i}\le c_{\min}^{-1}x_{i}^{2}$, and summing over $i$,

$$0\le|x|_{c}^{2}\le c_{\min}^{-1}\lVert x\rVert^{2}\qquad\text{for every }x\in\mathbb{R}^{d}.$$

Step 1 (Clause 1). The function $x\mapsto|x|_{c}^{2}$ is Borel and nonnegative by Step 0, and by the displayed inequality and claim 1 of Linearity and Monotonicity of the Lebesgue Integral (monotonicity and positive homogeneity of the integral of nonnegative functions),

$$\int_{\mathbb{R}^{d}}|x|_{c}^{2}\,\mu(dx)\le c_{\min}^{-1}\int_{\mathbb{R}^{d}}\lVert x\rVert^{2}\,\mu(dx)=c_{\min}^{-1}M_{2}(\mu)<\infty,$$

the second moment being finite because $\mu\in\mathcal{P}_{2}(\mathbb{R}^{d})$ (The Second Moment of a Probability Measure on Euclidean Space and the Probability Measures with Finite Second Moment §space). Since the function equals its own absolute value, it is integrable with respect to $\mu$ by Integrable Function and the Lebesgue Integral. This is clause 1.

Step 2 (Clause 2). The argument parallels the comparison of the entropy with the standard Gaussian relative entropy, with $\rho_{c}$ in place of the standard Gaussian weight.

(a) The reciprocal weight. For every $x$, $\rho_{c}(x)=\exp(\Phi_{c}(x))$ is positive and $1/\rho_{c}(x)=\exp(-\Phi_{c}(x))$ by claim 2 of Basic Properties of the Exponential Function, and $\log\rho_{c}(x)=\Phi_{c}(x)$, $\log(1/\rho_{c}(x))=-\Phi_{c}(x)$ by the inverse relation of The Natural Logarithm. The function $\exp$ is smooth on $\mathbb{R}$ by claim 3 of Basic Properties of the Exponential Function; identifying $\mathbb{R}$ with $\mathbb{R}^{1}$, whose Euclidean distance is the absolute-value metric $d_{\mathbb{R}}$ by The Euclidean Distance on the Real Line is the Absolute Value Metric, and using that $\mathbb{R}^{1}$ is open in itself by claim 1 of Euclidean Space is Open in Itself, and $C^k$ Maps are Continuous, claim 3 of that lemma shows that $\exp$ is continuous from $(\mathbb{R},d_{\mathbb{R}})$ to $(\mathbb{R},d_{\mathbb{R}})$. Hence $\exp$ is measurable with respect to $\mathcal{B}(\mathbb{R})$ and $\mathcal{B}(\mathbb{R})$ by claim 3 of Borel Measurability and Bounded Integration on a Metric Space, the Borel $\sigma$-algebra of $(\mathbb{R},d_{\mathbb{R}})$ being $\mathcal{B}(\mathbb{R})$ by claim 2 there. As $-\Phi_{c}$ is Borel (Step 0 and claim 2 of Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions), the function $1/\rho_{c}=\exp\circ(-\Phi_{c})$ is Borel by claim 4 of Borel Measurability and Bounded Integration on a Metric Space. The function $\rho_{c}$ is Borel and positive by The Diagonal Gaussian Density on Euclidean Space: Regularity, Gradient, Normalization, Second Moments and Exponential Moments of Diagonal Quadratic Forms §regularity.

(b) Densities with respect to $\lambda_{d}$ and to $\gamma_{c}$. Let $\rho,f:\mathbb{R}^{d}\to\mathbb{R}$ be Borel and nonnegative with $\rho=f\rho_{c}$. Then $\rho$ is a density of $\mu$ with respect to $\lambda_{d}$ if and only if $f$ is a density of $\mu$ with respect to $\gamma_{c}$. Indeed, for $B\in\mathcal{B}(\mathbb{R}^{d})$ the function $\mathbf{1}_{B}f$ is Borel and nonnegative by claims 1 and 3 of Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions, and since $\gamma_{c}$ is the measure with density $\rho_{c}$ with respect to $\lambda_{d}$ (Diagonal Gaussian Measures on Euclidean Space §measure), claim 3 of Image Measures, Measures with Densities, and Change of Variables gives

$$\int_{\mathbb{R}^{d}}\mathbf{1}_{B}f\,d\gamma_{c}=\int_{\mathbb{R}^{d}}\mathbf{1}_{B}f\rho_{c}\,d\lambda_{d}=\int_{\mathbb{R}^{d}}\mathbf{1}_{B}\rho\,d\lambda_{d}.$$

So $\mu(B)=\int\mathbf{1}_{B}\rho\,d\lambda_{d}$ for every Borel $B$ if and only if $\mu(B)=\int\mathbf{1}_{B}f\,d\gamma_{c}$ for every Borel $B$, and these are exactly the two density conditions of The Radon-Nikodym Theorem for a Finite Measure and a Sigma-Finite Measure, and Uniqueness of Densities. (In particular either condition forces $\mu(B)=0$ whenever $\lambda_{d}(B)=0$, respectively $\gamma_{c}(B)=0$; no separate absolute-continuity hypothesis is involved, since a density in that theorem is defined by this identity alone.)

(c) A pointwise identity. With $\rho,f$ as in (b), $\rho_{c}(x)\,\phi(f(x))=\phi(\rho(x))-\rho(x)\,\Phi_{c}(x)$ for every $x\in\mathbb{R}^{d}$. If $\rho(x)=0$, then $f(x)=0$ since $\rho_{c}(x)>0$, and both sides vanish because $\phi(0)=0$. If $\rho(x)>0$, then $f(x)=\rho(x)\cdot(1/\rho_{c}(x))>0$ and, by the product rule $\log(st)=\log s+\log t$ of The Natural Logarithm and (a), $\log f(x)=\log\rho(x)-\Phi_{c}(x)$; multiplying by $\rho_{c}(x)f(x)=\rho(x)$ gives $\rho_{c}(x)\phi(f(x))=\rho(x)\log\rho(x)-\rho(x)\Phi_{c}(x)=\phi(\rho(x))-\rho(x)\Phi_{c}(x)$.

(d) The correction term. Let $\rho$ be a density of $\mu$ with respect to $\lambda_{d}$. Then $\rho\,\Phi_{c}$ is integrable with respect to $\lambda_{d}$ and

$$\int_{\mathbb{R}^{d}}\rho\,\Phi_{c}\,d\lambda_{d}=-\tfrac12\int_{\mathbb{R}^{d}}|x|_{c}^{2}\,\mu(dx)-Z_{c}.$$

Indeed, the constant function $Z_{c}$ is integrable with respect to $\mu$ with integral $Z_{c}\,\mu(\mathbb{R}^{d})=Z_{c}$ by claim 6(a) of Borel Measurability and Bounded Integration on a Metric Space, $\mu$ being a Borel measure on $(\mathbb{R}^{d},d_{E})$ with $\mu(\mathbb{R}^{d})=1$ (Probability Measures on Euclidean Space and Random Vectors: Standing Notation §measures); $x\mapsto|x|_{c}^{2}$ is integrable with respect to $\mu$ by Step 1; so the Borel function $\Phi_{c}$ is integrable with respect to $\mu$ with the displayed integral by claim 2 of Linearity and Monotonicity of the Lebesgue Integral. Since $\mu(A)=\int\mathbf{1}_{A}\rho\,d\lambda_{d}$ for every Borel $A$, $\mu$ is the measure with density $\rho$ with respect to $\lambda_{d}$ of claim 3 of Image Measures, Measures with Densities, and Change of Variables, and that claim shows that $\Phi_{c}\rho$ is integrable with respect to $\lambda_{d}$ with the same integral.

(e) Conclusion. Suppose first that $\mu$ has finite entropy, with a density $\rho$ with respect to $\lambda_{d}$ for which $\phi\circ\rho$ is integrable with respect to $\lambda_{d}$ (The Entropy of a Probability Measure on Euclidean Space §entropy). Put $f=\rho\cdot(1/\rho_{c})$, Borel and nonnegative by (a) and claim 3 of Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions, with $\rho=f\rho_{c}$. By (b), $f$ is a density of $\mu$ with respect to $\gamma_{c}$, and $\phi\circ f$ is Borel by The Function $s\log s$: Continuity, Young's Inequality and Lower Bounds, with the Elementary Bounds for the Exponential and the Logarithm §continuous. By (c), $(\phi\circ f)\rho_{c}=\phi\circ\rho-\rho\,\Phi_{c}$, which by (d) and claim 2 of Linearity and Monotonicity of the Lebesgue Integral is integrable with respect to $\lambda_{d}$ with integral $\mathrm{Ent}(\mu)+\tfrac12\int|x|_{c}^{2}\,\mu(dx)+Z_{c}$. By claim 3 of Image Measures, Measures with Densities, and Change of Variables, applied to $\gamma_{c}$ with density $\rho_{c}$, $\phi\circ f$ is integrable with respect to $\gamma_{c}$ with the same integral. As $\gamma_{c}$ is a probability measure (Diagonal Gaussian Measures on Euclidean Space §measure), $\mu$ has finite relative entropy with respect to $\gamma_{c}$ and, by Relative Entropy of Probability Measures §relative-entropy, $H(\mu\,|\,\gamma_{c})=\mathrm{Ent}(\mu)+\tfrac12\int|x|_{c}^{2}\,\mu(dx)+Z_{c}$.

Suppose conversely that $\mu$ has finite relative entropy with respect to $\gamma_{c}$, with a density $f$ with respect to $\gamma_{c}$ for which $\phi\circ f$ is integrable with respect to $\gamma_{c}$. Put $\rho=f\rho_{c}$, Borel and nonnegative by (a) and claim 3 of Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions. By (b), $\rho$ is a density of $\mu$ with respect to $\lambda_{d}$. By claim 3 of Image Measures, Measures with Densities, and Change of Variables, $(\phi\circ f)\rho_{c}$ is integrable with respect to $\lambda_{d}$ with integral $H(\mu\,|\,\gamma_{c})$; by (c), (d) and claim 2 of Linearity and Monotonicity of the Lebesgue Integral, the Borel function $\phi\circ\rho=(\phi\circ f)\rho_{c}+\rho\,\Phi_{c}$ (Borel by The Function $s\log s$: Continuity, Young's Inequality and Lower Bounds, with the Elementary Bounds for the Exponential and the Logarithm §continuous) is integrable with respect to $\lambda_{d}$ with integral $H(\mu\,|\,\gamma_{c})-\tfrac12\int|x|_{c}^{2}\,\mu(dx)-Z_{c}$. So $\mu$ has finite entropy, and by The Entropy of a Probability Measure on Euclidean Space §entropy $\mathrm{Ent}(\mu)$ equals that number.

In either case the displayed identity of clause 2 holds, both $\mathrm{Ent}(\mu)$ and $H(\mu\,|\,\gamma_{c})$ being independent of the chosen densities by The Entropy of a Probability Measure on Euclidean Space §entropy and Relative Entropy of Probability Measures §relative-entropy.

Step 3 (Clause 3). By Step 0, $h_{c}:\mathbb{R}^{d}\to\mathbb{R}$ is of class $C^{1}$ on $\mathbb{R}^{d}$ in the sense of Differential Calculus and Convexity on Euclidean Open Sets: Standing Notation §derivatives, with gradient map $\nabla h_{c}=S_{c}$. The domain $\mathbb{R}^{d}$ is a convex subset of $\mathbb{R}^{d}$, since $t\,x+(1-t)\,y\in\mathbb{R}^{d}$ for all $x,y\in\mathbb{R}^{d}$ and $0\le t\le1$, and $h_{c}$ is convex on $\mathbb{R}^{d}$ in the sense of Convex Real-Valued Function on a Convex Subset of $\mathbb{R}^n$: for real $a,b$ and $t$ with $0\le t\le1$, expanding gives $t\,a^{2}+(1-t)\,b^{2}-(t\,a+(1-t)\,b)^{2}=t(1-t)(a-b)^{2}\ge0$; applying this with $a=x_{i}$, $b=y_{i}$, multiplying by $1/(2c_{i})>0$ and summing over $i\in[d]$, the $i$th component of $t\,x+(1-t)\,y$ being $t\,x_{i}+(1-t)\,y_{i}$, yields

$$t\,h_{c}(x)+(1-t)\,h_{c}(y)-h_{c}\bigl(t\,x+(1-t)\,y\bigr)=\tfrac12\,t(1-t)\,|x-y|_{c}^{2}\ge0$$

for all $x,y\in\mathbb{R}^{d}$. Moreover, $S_{c}$ is Borel by claims 1, 2 and 5 of The Borel Sigma-Algebra of a Euclidean Space as a Product, and Measurability of Projections, Sequentially Continuous Maps, and Open and Closed Sets and claim 2 of Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions (its $i$th component is $c_{i}^{-1}$ times the $i$th coordinate projection), so $x\mapsto\lVert S_{c}(x)\rVert^{2}$ is Borel by Pairs of Euclidean Points: Coordinate Projections, Pairings, the Product Measure on a Euclidean Space, Borel Norm Functions and Finite Sets §functions; since $0<c_{\min}\le c_{i}$ for every $i\in[d]$, $\lVert S_{c}(x)\rVert^{2}=\sum_{i=1}^{d}x_{i}^{2}/c_{i}^{2}\le c_{\min}^{-2}\lVert x\rVert^{2}$ for every $x\in\mathbb{R}^{d}$, and monotonicity and positive homogeneity of the integral of nonnegative measurable functions (Linearity and Monotonicity of the Lebesgue Integral §nonnegative) give

\int_{\mathbb{R}^{d}}\lVert S_{c}\rVert^{2}\,d\mu\le c_{\min}^{-2}\int_{\mathbb{R}^{d}}\lVert x\rVert^{2}\,\mu(dx)=c_{\min}^{-2}M_{2}(\mu)<\infty. $$ Hence [The Gradient of a Convex Continuously Differentiable Function Belongs to the Tangent Space When It Is Square-Integrable §tangent](/theorems/3b2dda99-dd88-440f-bd3c-4dc63af7eba1?v=04eebb1f-ec87-488a-8271-98e21b355396#clause-tangent), applied to $f=h_{c}$, shows that the class of $\nabla h_{c}=S_{c}$ in $L^{2}(\mu;\mathbb{R}^{d})$, which is the class written $S_{c}$, belongs to $T_{\mu}$. This is [clause 3](/theorems/f754b703-fb19-478d-b57f-3908b13ff136?v=5a05a5a6-11d1-4635-a538-b5f06b15d618#clause-tangent). **Step 4 (Clause 4).** By [Square-Integrable Vector Fields Against a Probability Measure on Euclidean Space, and Test Functions: Standing Notation §l2mu](/theorems/525e9aba-ad22-4bbc-b284-967de0a91469?v=83783c6c-8ef9-4ee8-8ed3-9d70d20cd59a#clause-l2mu), $L^{2}(\mu;\mathbb{R}^{d})$ is a real Hilbert space, in particular a real inner product space ([Real Hilbert Space §hilbert](/theorems/df9d54cf-fbdf-4d11-b0b3-129b6d29c98c?v=8ad06291-e649-4b96-928c-0bd7f8f79a79#clause-hilbert)) with inner product $\langle\cdot,\cdot\rangle_{\mu}$ and norm $\lVert\cdot\rVert_{\mu}$. Put $K=\lVert S_{c}\rVert_{\mu}$, a nonnegative real number. For $\psi\in C_{c}^{\infty}(\mathbb{R}^{d})$, [The Cauchy-Schwarz Inequality in a Real Inner Product Space](/theorems/75e68dc9-c425-4d4e-afd8-53eba404a50e?v=638ad390-44c9-472a-bccb-9ba6242bb219) gives

|\langle S_{c},\nabla\psi\rangle_{\mu}|\le K,\lVert\nabla\psi\rVert_{\mu},

and by [Finite Fisher Information Relative to a Diagonal Gaussian Measure: the Ornstein-Uhlenbeck Functional, the Relative Score and the Relative Fisher Information §functional](/theorems/d6475c3c-0db3-4295-914c-f755a2adc3c7?v=f877bac6-ee09-4301-83a2-5a6eb574db25#clause-functional), $\ell^{c}_{\mu}(\psi)=\int\Delta\psi\,d\mu-\langle S_{c},\nabla\psi\rangle_{\mu}$. If $\mu\in\mathcal{P}_{2}^{\mathcal{I}}(\mathbb{R}^{d})$, with a constant $C\ge0$ as in [Finite Fisher Information, the Score and the Fisher Information of a Probability Measure §finite](/theorems/7917fccc-0aab-409d-8894-4bd1dd8226d6?v=133c3f9a-a639-4a89-9abf-8dbcddc04767#clause-finite), then by claims 2 and 5 of [Properties of the Absolute Value in an Ordered Field](/theorems/cbeef0cf-b6b0-4ce2-8b8a-d9fb43b5711a?v=2a9a804d-7459-42fd-9385-3c4d7fdc97de), $|\ell^{c}_{\mu}(\psi)|\le|\int\Delta\psi\,d\mu|+|\langle S_{c},\nabla\psi\rangle_{\mu}|\le(C+K)\lVert\nabla\psi\rVert_{\mu}$ for every $\psi$, so $\mu$ has finite Fisher information relative to $\gamma_{c}$ ([Finite Fisher Information Relative to a Diagonal Gaussian Measure: the Ornstein-Uhlenbeck Functional, the Relative Score and the Relative Fisher Information §finite](/theorems/d6475c3c-0db3-4295-914c-f755a2adc3c7?v=f877bac6-ee09-4301-83a2-5a6eb574db25#clause-finite)) with constant $C+K\ge0$. Conversely, if $\mu$ has finite Fisher information relative to $\gamma_{c}$, with a constant $C'\ge0$ as in [Finite Fisher Information Relative to a Diagonal Gaussian Measure: the Ornstein-Uhlenbeck Functional, the Relative Score and the Relative Fisher Information §finite](/theorems/d6475c3c-0db3-4295-914c-f755a2adc3c7?v=f877bac6-ee09-4301-83a2-5a6eb574db25#clause-finite), then $\int\Delta\psi\,d\mu=\ell^{c}_{\mu}(\psi)+\langle S_{c},\nabla\psi\rangle_{\mu}$, and the same claims give $|\int\Delta\psi\,d\mu|\le(C'+K)\lVert\nabla\psi\rVert_{\mu}$ for every $\psi$; as $\mu\in\mathcal{P}_{2}(\mathbb{R}^{d})$, this says $\mu\in\mathcal{P}_{2}^{\mathcal{I}}(\mathbb{R}^{d})$ by [Finite Fisher Information, the Score and the Fisher Information of a Probability Measure §finite](/theorems/7917fccc-0aab-409d-8894-4bd1dd8226d6?v=133c3f9a-a639-4a89-9abf-8dbcddc04767#clause-finite). Now let $\mu$ satisfy these equivalent conditions, and put $\eta=\xi_{\mu}+S_{c}$. Since $\xi_{\mu}\in T_{\mu}$ by [Finite Fisher Information, the Score and the Fisher Information of a Probability Measure §score](/theorems/7917fccc-0aab-409d-8894-4bd1dd8226d6?v=133c3f9a-a639-4a89-9abf-8dbcddc04767#clause-score), $S_{c}\in T_{\mu}$ by Step 3, and $T_{\mu}$ is a linear subspace of $L^{2}(\mu;\mathbb{R}^{d})$ by [Basic Properties of the Tangent Space: Closed Subspace, the Identity Map Belongs to It, Second-Moment Limits, and Representation of Bounded Functionals on Gradients §closed](/theorems/c557b279-9f31-4911-b5f7-31420742f7c8?v=a49b7507-17a5-40b4-8693-2069ff96df71#clause-closed), we have $\eta\in T_{\mu}$. For every $\psi\in C_{c}^{\infty}(\mathbb{R}^{d})$, additivity of the inner product in its first argument ([Real Inner Product Space §inner-product](/theorems/9b2285b3-45e0-4efe-8368-ceb779c4bbc7?v=b196350d-5105-4be4-87a2-8e7fdecd75b8#clause-inner-product)) and the defining identity of $\xi_{\mu}$ in [Finite Fisher Information, the Score and the Fisher Information of a Probability Measure §score](/theorems/7917fccc-0aab-409d-8894-4bd1dd8226d6?v=133c3f9a-a639-4a89-9abf-8dbcddc04767#clause-score) give

\langle\eta,\nabla\psi\rangle_{\mu}=\langle\xi_{\mu},\nabla\psi\rangle_{\mu}+\langle S_{c},\nabla\psi\rangle_{\mu}=-\int_{\mathbb{R}^{d}}\Delta\psi,d\mu+\langle S_{c},\nabla\psi\rangle_{\mu}=-\ell^{c}_{\mu}(\psi).

By the uniqueness in [Finite Fisher Information Relative to a Diagonal Gaussian Measure: the Ornstein-Uhlenbeck Functional, the Relative Score and the Relative Fisher Information §score](/theorems/d6475c3c-0db3-4295-914c-f755a2adc3c7?v=f877bac6-ee09-4301-83a2-5a6eb574db25#clause-score), $\zeta^{c}_{\mu}=\eta=\xi_{\mu}+S_{c}$. This is [clause 4](/theorems/f754b703-fb19-478d-b57f-3908b13ff136?v=5a05a5a6-11d1-4635-a538-b5f06b15d618#clause-score). **Step 5 (Clause 5).** Apply [The Relative Free Energy and the Relative Score of a Probability Measure for a Potential on an Open Set §energy](/theorems/0eea6d8e-847f-42f9-b65e-3a9720ba7014?v=6303adc5-b7f9-48f6-a764-1df48fba42cd#clause-energy) with $D=\mathbb{R}^{d}$, which is open by claim 1 of [Euclidean Space is Open in Itself, and $C^k$ Maps are Continuous](/theorems/cbd495a3-9ee9-498d-bd11-b22f41fc1ed0?v=bab38cb2-ed48-4534-8c30-654eda99c219), $U=V_{c}$ and temperature $a$. That $V_{c}$ is of class $C^{2}$ on $\mathbb{R}^{d}$ with $DV_{c}(x)=a\,S_{c}(x)$, as stated before clause 5, also follows from [Quadratic and Affine Functions of Class $C^2$, Translation, and Quadratic Perturbation of Semiconvexity §quadratic](/theorems/53755eb4-dc50-47f1-97dd-8a7a52f18ff3?v=a69f97a3-eaa4-4395-bd9b-599c3bc69330#clause-quadratic) with the symmetric matrix $a\,M_{c}$ of Step 0, whose matrix-vector product with $x$ is $a\,S_{c}(x)$. Since $D=\mathbb{R}^{d}$, the extended maps of that definition are $\bar{V}_{c}=V_{c}$ and $\nabla V_{c}=a\,S_{c}$, and $\mu(D)=\mu(\mathbb{R}^{d})=1$ holds for every $\mu\in\mathcal{P}(\mathbb{R}^{d})$. By Step 1 and claim 2 of [Linearity and Monotonicity of the Lebesgue Integral](/theorems/56ed866b-2761-445c-a3de-5a72e9a7f46c?v=09e8eada-637a-41e6-81e8-77291e1bdea6), $V_{c}=\tfrac{a}{2}|\cdot|_{c}^{2}$ is integrable with respect to $\mu$, with $\int V_{c}\,d\mu=\tfrac{a}{2}\int|x|_{c}^{2}\,\mu(dx)$. Hence, $\mu$ lying in $\mathcal{P}_{2}(\mathbb{R}^{d})$, $\mu\in\mathcal{D}_{V_{c},a}$ if and only if $\mu$ has finite entropy, that is ([The Entropy of a Probability Measure on Euclidean Space §entropy](/theorems/587ba69f-eb94-4439-b9a2-de24ff354c6d?v=8c6ade6c-1ae7-4388-816f-d6ce087c6efb#clause-entropy)) $\mu\in\mathcal{P}_{2}^{\mathrm{Ent}}(\mathbb{R}^{d})$; by Step 2 this holds if and only if $\mu$ has finite relative entropy with respect to $\gamma_{c}$. In that case Step 2 gives $\mathrm{Ent}(\mu)=H(\mu\,|\,\gamma_{c})-\tfrac12\int|x|_{c}^{2}\,\mu(dx)-Z_{c}$, so

\mathcal{E}{V{c},a}(\mu)=a,\mathrm{Ent}(\mu)+\tfrac{a}{2}\int_{\mathbb{R}^{d}}|x|{c}^{2},\mu(dx)=a,H(\mu,|,\gamma{c})-a,Z_{c}.

This is [clause 5](/theorems/f754b703-fb19-478d-b57f-3908b13ff136?v=5a05a5a6-11d1-4635-a538-b5f06b15d618#clause-free-energy). **Step 6 (Clause 6).** By [The Relative Free Energy and the Relative Score of a Probability Measure for a Potential on an Open Set §score](/theorems/0eea6d8e-847f-42f9-b65e-3a9720ba7014?v=6303adc5-b7f9-48f6-a764-1df48fba42cd#clause-score), $\mu\in\mathcal{D}^{\Sigma}_{V_{c},a}$ if and only if $\mu\in\mathcal{D}_{V_{c},a}$, $\mu\in\mathcal{P}_{2}^{\mathcal{I}}(\mathbb{R}^{d})$ and $\int\lVert\nabla V_{c}\rVert^{2}\,d\mu<\infty$. The last condition always holds: $\lVert a\,S_{c}(x)\rVert^{2}=a^{2}\lVert S_{c}(x)\rVert^{2}$, so by positive homogeneity of the integral ([Linearity and Monotonicity of the Lebesgue Integral §nonnegative](/theorems/56ed866b-2761-445c-a3de-5a72e9a7f46c?v=09e8eada-637a-41e6-81e8-77291e1bdea6#clause-nonnegative)) and the bound $\int_{\mathbb{R}^{d}}\lVert S_{c}\rVert^{2}\,d\mu\le c_{\min}^{-2}M_{2}(\mu)$ derived in Step 3, $\int\lVert\nabla V_{c}\rVert^{2}\,d\mu=a^{2}\int\lVert S_{c}\rVert^{2}\,d\mu\le a^{2}c_{\min}^{-2}M_{2}(\mu)<\infty$. Hence, by Step 5 and Step 4, $\mu\in\mathcal{D}^{\Sigma}_{V_{c},a}$ if and only if $\mu$ has finite relative entropy with respect to $\gamma_{c}$ and finite Fisher information relative to $\gamma_{c}$. In that case the class of $\nabla V_{c}$ in $L^{2}(\mu;\mathbb{R}^{d})$ is $a\,S_{c}$, as stated before clause 5, and Step 4 gives $\zeta^{c}_{\mu}=\xi_{\mu}+S_{c}$, so, computing in the real vector space $L^{2}(\mu;\mathbb{R}^{d})$,

\Sigma_{V_{c},a}(\mu)=a,S_{c}+a,\xi_{\mu}=a,(\xi_{\mu}+S_{c})=a,\zeta^{c}_{\mu}.

This is [clause 6](/theorems/f754b703-fb19-478d-b57f-3908b13ff136?v=5a05a5a6-11d1-4635-a538-b5f06b15d618#clause-relative-score).