Relative Entropy along a Noise-Gradient Perturbation of the Identity: the Formula, the Second-Order Expansion and the First Variation

In the setting of A Diagonal Gaussian Reference Measure on the Noise Wasserstein Space, Rescaled Heads and Gaussian Tails: Standing Notation, let $\mu\in\mathcal{P}(X)$ have finite relative entropy with respect to $\gamma_{c}$; then $\mu\in\mathcal{P}_{2}(X)$, the set of The Second Moment of a Borel Probability Measure on a Hilbert Space and the Probability Measures with Finite Second Moment §space, by Relative Entropy with Respect to a Diagonal Gaussian Measure on a Hilbert Space: the Moment Bound, the Cutoff Projections, and Bounded, Tight, Weakly Closed, Wasserstein-Closed and Weakly Sequentially Compact Sublevel Sets §moment. Let $n\in\mathbb{N}$ and let $g\in C^{2}_{b}(\mathbb{R}^{n})$, the set of Bounded Twice Continuously Differentiable Functions with Bounded First and Second Partial Derivatives on Euclidean Space §bounded with $q=n$, with partial derivatives $\partial_{k}g$ and $\partial_{j}\partial_{i}g$ as in Differential Calculus and Convexity on Euclidean Open Sets: Standing Notation §derivatives, and let $L^{a}_{\mu}(g)$ be the noise Ornstein-Uhlenbeck functional of $\mu$ at $g$. A function of class $C^{2}$ on $\mathbb{R}^{n}$ is of class $C^{1}$ there by clause 2 of C^k Maps on a Euclidean Open Set, so $g$ belongs to the set $C^{1}_{b}(\mathbb{R}^{n})$ of Bounded Continuously Differentiable Functions with Bounded Partial Derivatives on Euclidean Space §bounded, and $\psi=g\circ p_{n}$ belongs to $\mathcal{F}C^{2}_{b}(X)$ and to $\mathcal{F}C^{1}_{b}(X)$, the sets of Bounded C^2 Cylindrical Functions on a Hilbert Space with an Orthonormal Basis §cylindrical and Bounded C^1 Cylindrical Functions on a Hilbert Space with an Orthonormal Basis §cylindrical; its noise gradient is $\nabla_{a}\psi(x)=\sum_{k=1}^{n}a_{k}\,\partial_{k}g(p_{n}(x))\,e_{k}$ by that clause and Bounded C^1 Cylindrical Functions: Linear Structure, the Gradient and the Partial Derivatives, Bounds and Integrability, and Density in the Square-Integrable Functions §partial. For $t\in\mathbb{R}$, $\mathrm{id}+t\nabla_{a}\psi$ is the map $x\mapsto x+t\,\nabla_{a}\psi(x)$. Let $B,b\in\mathbb{R}$ be nonnegative with $|\partial_{k}g(u)|\le B$ and $a_{i}^{1/2}a_{j}^{1/2}|\partial_{j}\partial_{i}g(u)|\le b$ for all $u\in\mathbb{R}^{n}$ and $i,j,k\in[n]$. For $x\in X$ let $M(x)$ be the real $n\times n$ matrix with entry $a_{i}^{1/2}a_{j}^{1/2}\,\partial_{j}\partial_{i}g(p_{n}(x))$ in row $i$ and column $j$, and let $I_{n}$ be the identity matrix; $\det$ is the determinant, $\log$ the natural logarithm, and $\theta_{n}$ and $K_{n}$ are fixed constants as provided by Determinants of Positive Definite Matrices: Positivity, the Bound $\log\det A\le\mathrm{tr}\,A-d$, Bounds under Pinching, and the Expansion of $\det(I+tB)$ §expansion with $d=n$, where they are written $c_{d}$ and $K_{d}$.

1. (The relative entropy of the push-forward)¶ Let $t\in\mathbb{R}$ satisfy $2n\,|t|\,b\le1$. Then $0<\det(I_{n}+tM(x))$ for every $x\in X$, the function $x\mapsto\log\det(I_{n}+tM(x))$ is Borel and bounded, the map $\mathrm{id}+t\nabla_{a}\psi$ is Borel, the measure $(\mathrm{id}+t\nabla_{a}\psi)_{\#}\mu$ has finite relative entropy with respect to $\gamma_{c}$, and

$$H\bigl((\mathrm{id}+t\nabla_{a}\psi)_{\#}\mu\,\big|\,\gamma_{c}\bigr)=H(\mu\,|\,\gamma_{c})+\sum_{k=1}^{n}\int_{X}\Bigl(\frac{t\,a_{k}\,x_{k}}{c_{k}}\,\partial_{k}g(p_{n}(x))+\frac{t^{2}a_{k}^{2}}{2c_{k}}\,\partial_{k}g(p_{n}(x))^{2}\Bigr)\mu(dx)-\int_{X}\log\det\bigl(I_{n}+tM(x)\bigr)\,\mu(dx).$$

2. (Second-order expansion)¶ Let $t\in\mathbb{R}$ satisfy $2n\,|t|\,b\le1$ and $|t|\,b\le\theta_{n}$. Then

$$\Bigl|H\bigl((\mathrm{id}+t\nabla_{a}\psi)_{\#}\mu\,\big|\,\gamma_{c}\bigr)-H(\mu\,|\,\gamma_{c})-t\,L^{a}_{\mu}(g)\Bigr|\le t^{2}\Bigl(K_{n}\,b^{2}+B^{2}\sum_{k=1}^{n}\frac{a_{k}^{2}}{2c_{k}}\Bigr).$$

3. (First variation)¶ There is a positive real number $t_{0}$ such that $(\mathrm{id}+t\nabla_{a}\psi)_{\#}\mu$ has finite relative entropy with respect to $\gamma_{c}$ for every $t$ in the open interval $(-t_{0},t_{0})$, and the function $t\mapsto H\bigl((\mathrm{id}+t\nabla_{a}\psi)_{\#}\mu\,|\,\gamma_{c}\bigr)$ on $(-t_{0},t_{0})$ is differentiable at $0$, in the sense of Derivative at an Interior Point, with derivative $L^{a}_{\mu}(g)$.