Proof of Changing the Variances of a Diagonal Gaussian Reference Measure: Equal Domains, the Shift of the Relative Entropy and the Shift of the Relative Score

Each result cited is universally quantified over the data in its own statement. Let $\mu\in\mathcal{P}_{2}(\mathbb{R}^{d})$ be as in the statement. Elementary ordered-field arithmetic is used without citation (The Real Numbers: Standing Notation and Background §background). The result Relative Entropy and Relative Score with Respect to a Diagonal Gaussian Measure: Comparison with the Entropy, the Score and the Relative Free Energy of a Quadratic Potential is applied twice to this $\mu$, once with the variance vector $c$ and once with the variance vector $c'$; in its clauses the entropy $\mathrm{Ent}(\mu)$, the set $\mathcal{P}_{2}^{\mathcal{I}}(\mathbb{R}^{d})$ and the score $\xi_{\mu}$ depend on $\mu$ alone, not on the variance vector.

Clause 1. By Relative Entropy and Relative Score with Respect to a Diagonal Gaussian Measure: Comparison with the Entropy, the Score and the Relative Free Energy of a Quadratic Potential §entropy with $c$, $\mu$ has finite relative entropy with respect to $\gamma_{c}$ if and only if $\mu$ has finite entropy; by the same clause with $c'$, $\mu$ has finite entropy if and only if it has finite relative entropy with respect to $\gamma_{c'}$. Combining the two equivalences gives the first assertion. Likewise, by Relative Entropy and Relative Score with Respect to a Diagonal Gaussian Measure: Comparison with the Entropy, the Score and the Relative Free Energy of a Quadratic Potential §score with $c$ and with $c'$, $\mu$ has finite Fisher information relative to $\gamma_{c}$ if and only if $\mu\in\mathcal{P}_{2}^{\mathcal{I}}(\mathbb{R}^{d})$, if and only if $\mu$ has finite Fisher information relative to $\gamma_{c'}$. This proves clause 1.

Clause 2. By Relative Entropy and Relative Score with Respect to a Diagonal Gaussian Measure: Comparison with the Entropy, the Score and the Relative Free Energy of a Quadratic Potential §integrable with $c$ and with $c'$, the functions $x\mapsto|x|_{c}^{2}$ and $x\mapsto|x|_{c'}^{2}$ are integrable with respect to $\mu$. By Linearity and Monotonicity of the Lebesgue Integral §integrable, applied on the measure space $(\mathbb{R}^{d},\mathcal{B}(\mathbb{R}^{d}),\mu)$ (whose integrals are those of Probability Measures on Euclidean Space and Random Vectors: Standing Notation §measures) with the coefficients $1$ and $-1$, the function $x\mapsto|x|_{c'}^{2}-|x|_{c}^{2}$ is integrable and

$$\int_{\mathbb{R}^{d}}\bigl(|x|_{c'}^{2}-|x|_{c}^{2}\bigr)\,\mu(dx)=\int_{\mathbb{R}^{d}}|x|_{c'}^{2}\,\mu(dx)-\int_{\mathbb{R}^{d}}|x|_{c}^{2}\,\mu(dx).\tag{2.1}$$

This is the first assertion. Now suppose that $\mu$ has finite relative entropy with respect to $\gamma_{c}$. By clause 1 it also has finite relative entropy with respect to $\gamma_{c'}$, and by Relative Entropy and Relative Score with Respect to a Diagonal Gaussian Measure: Comparison with the Entropy, the Score and the Relative Free Energy of a Quadratic Potential §entropy it has finite entropy; thus $H(\mu\,|\,\gamma_{c})$, $H(\mu\,|\,\gamma_{c'})$ and $\mathrm{Ent}(\mu)$ are real numbers (Relative Entropy of Probability Measures §relative-entropy and The Entropy of a Probability Measure on Euclidean Space §entropy). The formula of Relative Entropy and Relative Score with Respect to a Diagonal Gaussian Measure: Comparison with the Entropy, the Score and the Relative Free Energy of a Quadratic Potential §entropy, with $c$ and with $c'$, reads

$$H(\mu\,|\,\gamma_{c})=\mathrm{Ent}(\mu)+\tfrac12\int_{\mathbb{R}^{d}}|x|_{c}^{2}\,\mu(dx)+Z_{c},\qquad H(\mu\,|\,\gamma_{c'})=\mathrm{Ent}(\mu)+\tfrac12\int_{\mathbb{R}^{d}}|x|_{c'}^{2}\,\mu(dx)+Z_{c'}.$$

Subtracting the first identity from the second and using (2.1),

$$H(\mu\,|\,\gamma_{c'})-H(\mu\,|\,\gamma_{c})=\tfrac12\Bigl(\int_{\mathbb{R}^{d}}|x|_{c'}^{2}\,\mu(dx)-\int_{\mathbb{R}^{d}}|x|_{c}^{2}\,\mu(dx)\Bigr)+Z_{c'}-Z_{c}=\tfrac12\int_{\mathbb{R}^{d}}\bigl(|x|_{c'}^{2}-|x|_{c}^{2}\bigr)\,\mu(dx)+Z_{c'}-Z_{c},$$

which, after adding $H(\mu\,|\,\gamma_{c})$ to both sides, is the asserted formula. This proves clause 2.

Clause 3. Suppose that $\mu$ has finite Fisher information relative to $\gamma_{c}$. By Relative Entropy and Relative Score with Respect to a Diagonal Gaussian Measure: Comparison with the Entropy, the Score and the Relative Free Energy of a Quadratic Potential §score with $c$, $\mu\in\mathcal{P}_{2}^{\mathcal{I}}(\mathbb{R}^{d})$, so its score $\xi_{\mu}\in T_{\mu}$ is defined, and $\zeta^{c}_{\mu}=\xi_{\mu}+S_{c}$ in $T_{\mu}$. By clause 1, $\mu$ has finite Fisher information relative to $\gamma_{c'}$, so $\zeta^{c'}_{\mu}$ is defined (Finite Fisher Information Relative to a Diagonal Gaussian Measure: the Ornstein-Uhlenbeck Functional, the Relative Score and the Relative Fisher Information §score), and by Relative Entropy and Relative Score with Respect to a Diagonal Gaussian Measure: Comparison with the Entropy, the Score and the Relative Free Energy of a Quadratic Potential §score with $c'$, $\zeta^{c'}_{\mu}=\xi_{\mu}+S_{c'}$ in $T_{\mu}$. Since $T_{\mu}\subseteq L^{2}(\mu;\mathbb{R}^{d})$ (The Tangent Space of the Wasserstein Space at a Probability Measure §tangent), both identities hold in $L^{2}(\mu;\mathbb{R}^{d})$, where $S_{c}$ and $S_{c'}$ are the classes of the preamble of Finite Fisher Information Relative to a Diagonal Gaussian Measure: the Ornstein-Uhlenbeck Functional, the Relative Score and the Relative Fisher Information. The space $L^{2}(\mu;\mathbb{R}^{d})$ is a real Hilbert space (Square-Integrable Vector Fields Against a Probability Measure on Euclidean Space, and Test Functions: Standing Notation §l2mu), hence a real inner product space (Real Hilbert Space §hilbert) and in particular a vector space over $\mathbb{R}$ (Real Inner Product Space §inner-product). Writing $u-v=u+(-v)$ as in claim 2 of Elementary Identities in a Vector Space, the associativity and commutativity of addition, the additive inverse and the zero vector of Vector Space over a Field give, in $L^{2}(\mu;\mathbb{R}^{d})$,

$$\zeta^{c}_{\mu}+S_{c'}-S_{c}=\bigl(\xi_{\mu}+S_{c}\bigr)+S_{c'}+\bigl(-S_{c}\bigr)=\xi_{\mu}+S_{c'}+\bigl(S_{c}+(-S_{c})\bigr)=\xi_{\mu}+S_{c'}=\zeta^{c'}_{\mu}.$$

This proves clause 3.