Proof of Relative Entropy along a Noise-Gradient Perturbation of the Identity: the Formula, the Second-Order Expansion and the First Variation

Each result cited is universally quantified over the data in its own statement. Throughout, $\mathbb{R}^{n}$ is open by claim 1 of Euclidean Space is Open in Itself, and $C^k$ Maps are Continuous, and partial derivatives, the class $C^{k}$, gradients and Hessian matrices are those of Differential Calculus and Convexity on Euclidean Open Sets: Standing Notation §derivatives. Let $\delta_{n}^{-1}:\mathbb{R}^{n}\to\mathbb{R}^{n}$ be the map $u\mapsto(a_{1}^{1/2}u_{1},\dots,a_{n}^{1/2}u_{n})$; it is the inverse of the map $\delta_{n}$ of A Diagonal Gaussian Reference Measure on the Noise Wasserstein Space, Rescaled Heads and Gaussian Tails: Standing Notation §heads, since $a_{k}^{1/2}a_{k}^{-1/2}=1$, and as $r_{n}=\delta_{n}\circ p_{n}$ there, $p_{n}=\delta_{n}^{-1}\circ r_{n}$. Let $h=g\circ\delta_{n}^{-1}$, and for $x\in X$ and $k\in[n]$ let $\beta_{k}(x)=\partial_{k}g(p_{n}(x))$.

Step 1 (Partial derivatives along $\delta_{n}^{-1}$). Let $f:\mathbb{R}^{n}\to\mathbb{R}$, $y\in\mathbb{R}^{n}$ and $i\in[n]$, and suppose that the partial derivative of $f$ with respect to the $i$th variable exists at $\delta_{n}^{-1}(y)$ with value $L$. For $s\in\mathbb{R}$, the point obtained from $y$ by adding $s$ to its $i$th coordinate is mapped by $\delta_{n}^{-1}$ to the point obtained from $\delta_{n}^{-1}(y)$ by adding $a_{i}^{1/2}s$ to its $i$th coordinate. Given $\varepsilon>0$, Partial Derivative on a Euclidean Open Set gives $\eta>0$ such that the difference quotient of $f$ at $\delta_{n}^{-1}(y)$ in the $i$th variable with increment $\sigma$ differs from $L$ by less than $\varepsilon/a_{i}^{1/2}$ whenever $0<|\sigma|<\eta$. If $0<|s|<\eta/a_{i}^{1/2}$, then $\sigma=a_{i}^{1/2}s$ satisfies $0<|\sigma|<\eta$, and the difference quotient of $f\circ\delta_{n}^{-1}$ at $y$ in the $i$th variable with increment $s$ is $a_{i}^{1/2}$ times the former one with increment $\sigma$, so it differs from $a_{i}^{1/2}L$ by less than $\varepsilon$. Hence $\partial_{i}(f\circ\delta_{n}^{-1})(y)$ exists and equals $a_{i}^{1/2}\,\partial_{i}f(\delta_{n}^{-1}(y))$. Moreover $\delta_{n}^{-1}$ is continuous at every point: with $A=\sum_{k=1}^{n}a_{k}$, each $(y_{k}-y'_{k})^{2}$ is at most $\sum_{j=1}^{n}(y_{j}-y'_{j})^{2}$, so $\sum_{k=1}^{n}a_{k}(y_{k}-y'_{k})^{2}\le A\sum_{j=1}^{n}(y_{j}-y'_{j})^{2}$, which is less than $\varepsilon^{2}$ when $\sum_{j}(y_{j}-y'_{j})^{2}<(\varepsilon/(1+A))^{2}$. So if $f$ is continuous at every point, so is $f\circ\delta_{n}^{-1}$, by Composition of Continuous Euclidean Maps. Consequently, if $f$ is of class $C^{1}$ on $\mathbb{R}^{n}$ (C^k Maps on a Euclidean Open Set, clauses 1 and 3), then so is $f\circ\delta_{n}^{-1}$, with $\partial_{i}(f\circ\delta_{n}^{-1})=a_{i}^{1/2}\,(\partial_{i}f)\circ\delta_{n}^{-1}$, a constant multiple of a function continuous at every point. Such a multiple is again continuous at every point: if $\varphi:\mathbb{R}^{n}\to\mathbb{R}$ is continuous at $y$ and $\varepsilon>0$, choose $\delta>0$ for $\varphi$ at $y$ and $\varepsilon/(1+a_{i}^{1/2})$ in place of $\varepsilon$; then $\sum_{j}(y'_{j}-y_{j})^{2}<\delta^{2}$ gives $\bigl(a_{i}^{1/2}\varphi(y')-a_{i}^{1/2}\varphi(y)\bigr)^{2}=a_{i}\bigl(\varphi(y')-\varphi(y)\bigr)^{2}<a_{i}\varepsilon^{2}/(1+a_{i}^{1/2})^{2}<\varepsilon^{2}$.

Step 2 ($h$ is of class $C^{2}$ with Hessian $M$). The function $g$ is of class $C^{2}$, hence of class $C^{1}$ by claim 2 of Euclidean Space is Open in Itself, and $C^k$ Maps are Continuous, and each $\partial_{i}g$ is of class $C^{1}$ by clause 2 of C^k Maps on a Euclidean Open Set. By Step 1 with $f=g$, $h$ is of class $C^{1}$ with $\partial_{i}h=a_{i}^{1/2}(\partial_{i}g)\circ\delta_{n}^{-1}$. By Step 1 with $f=\partial_{i}g$, the function $(\partial_{i}g)\circ\delta_{n}^{-1}$ is of class $C^{1}$ with $j$th partial derivative $a_{j}^{1/2}(\partial_{j}\partial_{i}g)\circ\delta_{n}^{-1}$; so by claims 3 and 1 of Constants, Coordinate Functions, Sums and Products of $C^k$ Functions on a Euclidean Open Set, $\partial_{i}h$ is of class $C^{1}$ and

$$\partial_{j}\partial_{i}h(u)=a_{i}^{1/2}a_{j}^{1/2}\,\partial_{j}\partial_{i}g(\delta_{n}^{-1}(u))\qquad(u\in\mathbb{R}^{n},\ i,j\in[n]).$$

Hence $h$ is of class $C^{2}$ by clause 2 of C^k Maps on a Euclidean Open Set. By Hessian Matrix of a C^2 Function, the entry of $D^{2}h(u)$ in row $i$ and column $j$ is $\partial_{i}\partial_{j}h(u)=a_{i}^{1/2}a_{j}^{1/2}\partial_{i}\partial_{j}g(\delta_{n}^{-1}(u))$, which is at most $b$ in absolute value by the hypothesis on $b$ (with the indices interchanged). For $x\in X$ and $u=r_{n}(x)$ we have $\delta_{n}^{-1}(u)=p_{n}(x)$, so the entry of $D^{2}h(r_{n}(x))$ in row $i$ and column $j$ is the entry of $M(x)$ in row $j$ and column $i$; since $D^{2}h(r_{n}(x))$ is symmetric by claim 2 of Equality of Mixed Second Partial Derivatives and Symmetry of the Hessian, it follows that $D^{2}h(r_{n}(x))=M(x)$. Also $\partial_{i}h(r_{n}(x))=a_{i}^{1/2}\beta_{i}(x)$.

Step 3 (The potential). Let $q(u)=\tfrac12\lVert u\rVert^{2}=\tfrac12\sum_{l=1}^{n}\pi_{l}(u)^{2}$ (Elementary Properties of the Euclidean Norm on $\mathbb{R}^n$ §square), with the coordinate functions $\pi_{l}(u)=u_{l}$. By claims 2 and 3 of Constants, Coordinate Functions, Sums and Products of $C^k$ Functions on a Euclidean Open Set, $q$ is smooth, hence of class $C^{2}$ (Smooth Map on a Euclidean Open Set). Directly from Partial Derivative on a Euclidean Open Set, $\partial_{i}\pi_{l}$ is the constant $1$ if $l=i$ and $0$ otherwise, the difference quotients being constant; so claim 1 of Constants, Coordinate Functions, Sums and Products of $C^k$ Functions on a Euclidean Open Set gives $\partial_{i}q=\pi_{i}$ and $\partial_{j}\partial_{i}q$ equal to $1$ if $j=i$ and $0$ otherwise, that is, $D^{2}q(u)=I_{n}$. For $t\in\mathbb{R}$ let $\Phi_{t}=q+t\,h$. By claims 3 and 1 of Constants, Coordinate Functions, Sums and Products of $C^k$ Functions on a Euclidean Open Set (applied to $q,h$ and to their first partial derivatives), $\Phi_{t}$ is of class $C^{2}$, $\partial_{i}\Phi_{t}(u)=u_{i}+t\,\partial_{i}h(u)$, and $D^{2}\Phi_{t}(u)=I_{n}+t\,D^{2}h(u)$ for every $u$.

Step 4 (Pinching). Let $t\in\mathbb{R}$ satisfy $2n|t|b\le1$, and let $u,z\in\mathbb{R}^{n}$ and $A=D^{2}h(u)$. By Step 2 and $2|z_{i}||z_{j}|\le z_{i}^{2}+z_{j}^{2}$,

$$|z\cdot(Az)|=\Bigl|\sum_{i,j=1}^{n}z_{i}A_{ij}z_{j}\Bigr|\le b\sum_{i,j=1}^{n}|z_{i}||z_{j}|\le\frac{b}{2}\sum_{i,j=1}^{n}(z_{i}^{2}+z_{j}^{2})=n\,b\,\lVert z\rVert^{2},$$

using Elementary Properties of the Euclidean Norm on $\mathbb{R}^n$ §square. Since $z\cdot(D^{2}\Phi_{t}(u)z)=z\cdot z+t\,z\cdot(Az)=\lVert z\rVert^{2}+t\,z\cdot(Az)$ and $n|t|b\le\tfrac12$, we get $\tfrac12\lVert z\rVert^{2}\le z\cdot(D^{2}\Phi_{t}(u)z)\le\tfrac32\lVert z\rVert^{2}$, that is, $z\cdot((\tfrac12I_{n})z)\le z\cdot(D^{2}\Phi_{t}(u)z)\le z\cdot((\tfrac32I_{n})z)$. The matrix $D^{2}\Phi_{t}(u)$ lies in $\mathcal{S}(n)$ by Differential Calculus and Convexity on Euclidean Open Sets: Standing Notation §derivatives, so by The Positive Semidefinite Ordering on Symmetric Matrices

$$\tfrac12I_{n}\preceq D^{2}\Phi_{t}(u)\preceq\tfrac32I_{n}\qquad(u\in\mathbb{R}^{n}).$$

Step 5 (The head map). Keep $t$ as in Step 4 and let $F_{t}=\nabla\Phi_{t}$, the gradient map $u\mapsto D\Phi_{t}(u)$, so that by Steps 2 and 3 its $k$th coordinate is $(F_{t})_{k}(u)=u_{k}+t\,a_{k}^{1/2}\partial_{k}g(\delta_{n}^{-1}(u))$. By The Gradient of a Twice Continuously Differentiable Function with Hessian Pinched between Two Positive Multiples of the Identity is a Bi-Lipschitz Bijection of Euclidean Space with Continuously Differentiable Inverse with $d=n$, $\Phi=\Phi_{t}$, $\varepsilon=\tfrac12$ and $L=\tfrac32$, which applies by Step 4: $F_{t}$ has components of class $C^{1}$ (as recorded in its preamble) and is a bijection of $\mathbb{R}^{n}$ (The Gradient of a Twice Continuously Differentiable Function with Hessian Pinched between Two Positive Multiples of the Identity is a Bi-Lipschitz Bijection of Euclidean Space with Continuously Differentiable Inverse §bijection); and by The Gradient of a Twice Continuously Differentiable Function with Hessian Pinched between Two Positive Multiples of the Identity is a Bi-Lipschitz Bijection of Euclidean Space with Continuously Differentiable Inverse §inverse, for every $u$ the matrix $DF_{t}(u)=D^{2}\Phi_{t}(u)$ is symmetric and positive definite, the inverse map $F_{t}^{-1}$ has components of class $C^{1}$, and $D(F_{t}^{-1})(F_{t}(u))$ is the inverse matrix of $DF_{t}(u)$. So $F_{t}$ satisfies the hypotheses imposed on $F$ in Moving the Rescaled Head of the Diagonal Gaussian Measure by a Diffeomorphism: the Image Has an Explicit Positive Density Depending on the Head Only, and $0<\det DF_{t}(u)$ for every $u$, as recorded there.

Step 6 (The perturbation is $T_{F_{t}}$). For $x\in X$ and $u=r_{n}(x)$, Step 5 and $\delta_{n}^{-1}(u)=p_{n}(x)$ give $(F_{t}(u)-u)_{k}=t\,a_{k}^{1/2}\beta_{k}(x)$, so by the definition of the lift in Lifting Euclidean Tangent Fields of the Rescaled Head to Noise Tangent Fields on a Hilbert Space and $a_{k}^{1/2}a_{k}^{1/2}=a_{k}$,

$$\Lambda_{n}(F_{t}-\mathrm{id})(x)=\sum_{k=1}^{n}a_{k}^{1/2}\,t\,a_{k}^{1/2}\beta_{k}(x)\,e_{k}=t\sum_{k=1}^{n}a_{k}\,\partial_{k}g(p_{n}(x))\,e_{k}=t\,\nabla_{a}\psi(x),$$

the last equality being the formula for $\nabla_{a}\psi$ recorded in the statement. Hence $T_{F_{t}}=\mathrm{id}+t\nabla_{a}\psi$. By Moving the Rescaled Head of the Diagonal Gaussian Measure by a Diffeomorphism: the Image Has an Explicit Positive Density Depending on the Head Only §bijection, this map is a bijection of $X$, it and its inverse $T_{F_{t}^{-1}}$ are Borel, and $r_{n}(T_{F_{t}}(x))=F_{t}(r_{n}(x))$ for every $x\in X$.

Step 7 (The log-determinant). For $x\in X$, Steps 3, 2 and 5 give $DF_{t}(r_{n}(x))=D^{2}\Phi_{t}(r_{n}(x))=I_{n}+t\,M(x)$, so $0<\det(I_{n}+tM(x))$ by Step 5. By Determinants of Positive Definite Matrices: Positivity, the Bound $\log\det A\le\mathrm{tr}\,A-d$, Bounds under Pinching, and the Expansion of $\det(I+tB)$ §pinching with $d=n$, $\varepsilon=\tfrac12$ and $L=\tfrac32$, which applies by Step 4, $n-2n\le\log\det(I_{n}+tM(x))\le\tfrac32n-n$; so $|\log\det(I_{n}+tM(x))|\le n$. The function $u\mapsto\det DF_{t}(u)$ is continuous on $\mathbb{R}^{n}$ by Change of Variables for the Lebesgue Integral under a Continuously Differentiable Bijection of Euclidean Space with Symmetric Positive Definite Jacobian Matrix, and the Density of a Push-Forward §regularity, and it takes values in $(0,\infty)$ by Step 5; $\log$ is smooth on the open set $(0,\infty)$ by The Natural Logarithm, hence continuous relative to $(0,\infty)$ at every point of $(0,\infty)$ by claim 3 of Euclidean Space is Open in Itself, and $C^k$ Maps are Continuous, the Euclidean distance on the real line being the absolute-value metric by The Euclidean Distance on the Real Line is the Absolute Value Metric; and $r_{n}$ is continuous by Lifting Euclidean Tangent Fields of the Rescaled Head to Noise Tangent Fields on a Hilbert Space §head. So $x\mapsto\log\det(I_{n}+tM(x))=\log\det DF_{t}(r_{n}(x))$ is continuous, as a composite of continuous maps between metric spaces, the inner ones taking values in the sets on which the outer ones are continuous (two applications of Continuous Map Between Metric Spaces), hence Borel by claim 3 of Borel Measurability and Bounded Integration on a Metric Space, and bounded.

Step 8 (Integrability). Let $k\in[n]$. The function $\partial_{k}g$ is of class $C^{1}$ by clause 2 of C^k Maps on a Euclidean Open Set and is bounded with bounded partial derivatives by Bounded Twice Continuously Differentiable Functions with Bounded First and Second Partial Derivatives on Euclidean Space §bounded, so it lies in $C^{1}_{b}(\mathbb{R}^{n})$ (Bounded Continuously Differentiable Functions with Bounded Partial Derivatives on Euclidean Space §bounded) and $\beta_{k}=(\partial_{k}g)\circ p_{n}\in\mathcal{F}C^{1}_{b}(X)$ by Bounded C^1 Cylindrical Functions on a Hilbert Space with an Orthonormal Basis §cylindrical. Since $\mu\in\mathcal{P}_{2}(X)$, as recorded in the statement, the function $x\mapsto x_{k}\beta_{k}(x)$ is integrable with respect to $\mu$ by Bounded C^1 Cylindrical Functions: Linear Structure, the Gradient and the Partial Derivatives, Bounds and Integrability, and Density in the Square-Integrable Functions §coordinate-integrable. The function $\partial_{k}g$, being of class $C^{1}$, is continuous on $\mathbb{R}^{n}$ as a map from $(\mathbb{R}^{n},d_{E})$ into $(\mathbb{R},d_{\mathbb{R}})$, where $d_{\mathbb{R}}(s,s')=|s-s'|$, by claim 3 of Euclidean Space is Open in Itself, and $C^k$ Maps are Continuous. The function $\partial_{k}\partial_{k}g$ is continuous at every point in the Euclidean sense by clause 1 of C^k Maps on a Euclidean Open Set, applied to the function $\partial_{k}g$ of class $C^{1}$ ($g$ itself is only of class $C^{2}$, so claim 3 of Euclidean Space is Open in Itself, and $C^k$ Maps are Continuous is not applied to $\partial_{k}\partial_{k}g$). Since $d_{E}(y,y')=\lVert y-y'\rVert$ is nonnegative with $d_{E}(y,y')^{2}=\sum_{j=1}^{n}(y_{j}-y'_{j})^{2}$ by Elementary Properties of the Euclidean Norm on $\mathbb{R}^n$ §distance and Elementary Properties of the Euclidean Norm on $\mathbb{R}^n$ §square, the condition $\sum_{j}(y_{j}-y'_{j})^{2}<\delta^{2}$ is equivalent to $d_{E}(y,y')<\delta$, and for real $s,s'$ the condition $(s-s')^{2}<\varepsilon^{2}$ is equivalent to $|s-s'|<\varepsilon$; so $\partial_{k}\partial_{k}g$ is continuous on $\mathbb{R}^{n}$ as a map from $(\mathbb{R}^{n},d_{E})$ into $(\mathbb{R},d_{\mathbb{R}})$. The map $p_{n}:(X,d)\to(\mathbb{R}^{n},d_{E})$ is continuous by Borel Sets of a Hilbert Space with an Orthonormal Basis: Coordinates, Determination by Finite-Dimensional Projections, and Pairs §continuity, being Lipschitz there (A Lipschitz Map is Uniformly Continuous); so $\beta_{k}$, $\beta_{k}^{2}$ (by Continuity of Sums and Products of Real-Valued Functions on a Metric Space §on-set) and $x\mapsto\partial_{k}\partial_{k}g(p_{n}(x))$ are continuous, as composites of continuous maps between metric spaces, hence Borel by claim 3 of Borel Measurability and Bounded Integration on a Metric Space, and they are bounded, by $B$, $B^{2}$ and a bound for $\partial_{k}\partial_{k}g$ respectively. By claim 6(b) of Borel Measurability and Bounded Integration on a Metric Space they, and the bounded Borel function of Step 7, are integrable with respect to the probability measure $\mu$. Hence, by Linearity and Monotonicity of the Lebesgue Integral §integrable, the function

$$\Theta_{t}(x)=\sum_{k=1}^{n}\Bigl(\frac{t\,a_{k}\,x_{k}}{c_{k}}\,\beta_{k}(x)+\frac{t^{2}a_{k}^{2}}{2c_{k}}\,\beta_{k}(x)^{2}\Bigr)-\log\det\bigl(I_{n}+tM(x)\bigr)$$

is integrable with respect to $\mu$, and so is each summand.

Step 9 (The logarithm of the density along the map). Let $G=G_{F_{t}}$ be the function of Moving the Rescaled Head of the Diagonal Gaussian Measure by a Diffeomorphism: the Image Has an Explicit Positive Density Depending on the Head Only for $F=F_{t}$. By Moving the Rescaled Head of the Diagonal Gaussian Measure by a Diffeomorphism: the Image Has an Explicit Positive Density Depending on the Head Only §density, $G$ is Borel and positive, and $G\circ r_{n}$, which is Borel by claim 4 of Borel Measurability and Bounded Integration on a Metric Space and positive, is a density of $(T_{F_{t}})_{\#}\gamma_{c}$ with respect to $\gamma_{c}$. Let $x\in X$ and $u=r_{n}(x)$, so that $a_{k}^{1/2}u_{k}=x_{k}$ and $F_{t}(u)_{k}=u_{k}+t\,a_{k}^{1/2}\beta_{k}(x)$. By Step 6 and Moving the Rescaled Head of the Diagonal Gaussian Measure by a Diffeomorphism: the Image Has an Explicit Positive Density Depending on the Head Only §log-density,

$$\log G\bigl(r_{n}(T_{F_{t}}(x))\bigr)=\log G(F_{t}(u))=\tfrac12\,|F_{t}(u)|^{2}_{\tilde{c}^{(n)}}-\tfrac12\,|u|^{2}_{\tilde{c}^{(n)}}-\log\det DF_{t}(u).$$

By The Diagonal Gaussian Density on Euclidean Space and Its Notation §scaling and $\tilde{c}^{(n)}_{k}=c_{k}/a_{k}$ (A Diagonal Gaussian Reference Measure on the Noise Wasserstein Space, Rescaled Heads and Gaussian Tails: Standing Notation §heads), $|v|^{2}_{\tilde{c}^{(n)}}=\sum_{k=1}^{n}a_{k}v_{k}^{2}/c_{k}$ for $v\in\mathbb{R}^{n}$, and for each $k$

$$\frac{a_{k}}{2c_{k}}\Bigl(\bigl(u_{k}+t\,a_{k}^{1/2}\beta_{k}(x)\bigr)^{2}-u_{k}^{2}\Bigr)=\frac{a_{k}}{c_{k}}\,t\,a_{k}^{1/2}u_{k}\,\beta_{k}(x)+\frac{t^{2}a_{k}^{2}}{2c_{k}}\,\beta_{k}(x)^{2}=\frac{t\,a_{k}\,x_{k}}{c_{k}}\,\beta_{k}(x)+\frac{t^{2}a_{k}^{2}}{2c_{k}}\,\beta_{k}(x)^{2}.$$

Together with $\det DF_{t}(u)=\det(I_{n}+tM(x))$ (Step 7), this gives $\log G\bigl(r_{n}(T_{F_{t}}(x))\bigr)=\Theta_{t}(x)$ for every $x\in X$.

Step 10 (Claim 1). Apply Relative Entropy of the Image under a Bimeasurable Bijection that Moves the Reference Measure by a Positive Density §entropy on $(S,\mathcal{S})=(X,\mathcal{B}(X))$ with $\gamma=\gamma_{c}$, $\nu=\mu$, $T=T_{F_{t}}$ and the density $G\circ r_{n}$: $T$ is a bijection with $T$ and $T^{-1}$ Borel (Step 6); $G\circ r_{n}$ is measurable, positive and a density of $T_{\#}\gamma_{c}$ with respect to $\gamma_{c}$ (Step 9); $\mu$ has finite relative entropy with respect to $\gamma_{c}$; and $s\mapsto\log\bigl((G\circ r_{n})(T(s))\bigr)=\Theta_{t}(s)$ is integrable with respect to $\mu$ (Steps 8 and 9). The image measure $T_{\#}\mu$ there is the push-forward $(\mathrm{id}+t\nabla_{a}\psi)_{\#}\mu$ of Borel Probability Measures on a Real Hilbert Space with an Orthonormal Basis: Standing Notation §pushforward, by Step 6. Hence $(\mathrm{id}+t\nabla_{a}\psi)_{\#}\mu$ has finite relative entropy with respect to $\gamma_{c}$ and

$$H\bigl((\mathrm{id}+t\nabla_{a}\psi)_{\#}\mu\,\big|\,\gamma_{c}\bigr)=H(\mu\,|\,\gamma_{c})+\int_{X}\Theta_{t}\,d\mu ,$$

and splitting $\int_{X}\Theta_{t}\,d\mu$ into the integrals of its summands by Linearity and Monotonicity of the Lebesgue Integral §integrable (Step 8) gives the displayed formula of claim 1. The remaining assertions of claim 1 are Step 7 (positivity of the determinant; the log-determinant is Borel and bounded) and Step 6 (the map $\mathrm{id}+t\nabla_{a}\psi$ is Borel).

Step 11 (Claim 2). Let $t$ satisfy $2n|t|b\le1$ and $|t|b\le\theta_{n}$. By The Noise Ornstein-Uhlenbeck Functional of a Probability Measure at a Bounded C^2 Function of Finitely Many Coordinates §functional and Linearity and Monotonicity of the Lebesgue Integral §integrable (with the integrable functions of Step 8),

$$t\,L^{a}_{\mu}(g)=\sum_{k=1}^{n}\int_{X}\frac{t\,a_{k}\,x_{k}}{c_{k}}\,\beta_{k}(x)\,\mu(dx)-\int_{X}t\,\mathrm{tr}\,M(x)\,\mu(dx),$$

since $\sum_{k=1}^{n}a_{k}\,\partial_{k}\partial_{k}g(p_{n}(x))=\sum_{k=1}^{n}M(x)_{kk}=\mathrm{tr}\,M(x)$ by Trace of a Real Square Matrix and $a_{k}^{1/2}a_{k}^{1/2}=a_{k}$. Subtracting this from the formula of claim 1, again by linearity of the integral,

$$H\bigl((\mathrm{id}+t\nabla_{a}\psi)_{\#}\mu\,\big|\,\gamma_{c}\bigr)-H(\mu\,|\,\gamma_{c})-t\,L^{a}_{\mu}(g)=\sum_{k=1}^{n}\int_{X}\frac{t^{2}a_{k}^{2}}{2c_{k}}\,\beta_{k}^{2}\,d\mu-\int_{X}\Bigl(\log\det\bigl(I_{n}+tM(x)\bigr)-t\,\mathrm{tr}\,M(x)\Bigr)\mu(dx).$$

Since $0\le\beta_{k}^{2}\le B^{2}$ and $\mu(X)=1$, monotonicity of the integral (Linearity and Monotonicity of the Lebesgue Integral §integrable) and claim 6(a) of Borel Measurability and Bounded Integration on a Metric Space give $0\le\int_{X}\frac{t^{2}a_{k}^{2}}{2c_{k}}\beta_{k}^{2}\,d\mu\le t^{2}B^{2}\frac{a_{k}^{2}}{2c_{k}}$. For each $x\in X$, the matrix $M(x)$ has entries of absolute value at most $b$ by the hypothesis on $b$, and $|t|\,b\le\theta_{n}\le1$, $\theta_{n}$ being the constant $c_{n}\le1$ of Determinants of Positive Definite Matrices: Positivity, the Bound $\log\det A\le\mathrm{tr}\,A-d$, Bounds under Pinching, and the Expansion of $\det(I+tB)$ §expansion with $d=n$, as in the statement; so Determinants of Positive Definite Matrices: Positivity, the Bound $\log\det A\le\mathrm{tr}\,A-d$, Bounds under Pinching, and the Expansion of $\det(I+tB)$ §expansion with $d=n$, the matrix $M(x)$ in place of $B$ and $m=b$ gives $\bigl|\log\det(I_{n}+tM(x))-t\,\mathrm{tr}\,M(x)\bigr|\le K_{n}t^{2}b^{2}$. The function $x\mapsto t\,\mathrm{tr}\,M(x)=\sum_{k=1}^{n}t\,a_{k}\,\partial_{k}\partial_{k}g(p_{n}(x))$ is continuous by Continuity of Sums and Products of Real-Valued Functions on a Metric Space §on-set, applied to the continuous functions $x\mapsto\partial_{k}\partial_{k}g(p_{n}(x))$ of Step 8; so the integrand of the last integral, its difference with the continuous function of Step 7, is continuous by the same theorem, hence Borel by claim 3 of Borel Measurability and Bounded Integration on a Metric Space, and bounded by $K_{n}t^{2}b^{2}$; and so claim 6(b) of Borel Measurability and Bounded Integration on a Metric Space bounds the absolute value of that integral by $K_{n}t^{2}b^{2}$. The triangle inequality now gives claim 2.

Step 12 (Claim 3). If $b>0$, let $t_{0}$ be the lesser of $1/(2nb)$ and $\theta_{n}/b$; if $b=0$, let $t_{0}=1$. Then $t_{0}>0$, $\theta_{n}$ being positive, and every $t$ in the open interval $I=(-t_{0},t_{0})$ satisfies $2n|t|b\le1$ and $|t|b\le\theta_{n}$ (if $b>0$ because $|t|<t_{0}$, and if $b=0$ trivially). The set $I$ is an interval: if $s_{1},s_{3}\in I$ and $s_{1}\le s_{2}\le s_{3}$, then $-t_{0}<s_{1}\le s_{2}\le s_{3}<t_{0}$, so $-t_{0}<s_{2}<t_{0}$ and $s_{2}\in I$. By claim 1, $(\mathrm{id}+t\nabla_{a}\psi)_{\#}\mu$ has finite relative entropy with respect to $\gamma_{c}$ for every $t\in I$; let $\mathcal{H}(t)$ be its relative entropy. Since $\mathrm{id}+0\nabla_{a}\psi$ is the identity map of $X$, whose push-forward of $\mu$ is $\mu$, $\mathcal{H}(0)=H(\mu\,|\,\gamma_{c})$. The point $0$ is an interior point of $I$, as $-t_{0}/2<0<t_{0}/2$ with both bounds in $I$. Let $C=K_{n}b^{2}+B^{2}\sum_{k=1}^{n}a_{k}^{2}/(2c_{k})$, a nonnegative number, and $L=L^{a}_{\mu}(g)$. For $s\in I$ with $s\ne0$, claim 2 divided by $|s|$ gives

$$\Bigl|\frac{\mathcal{H}(s)-\mathcal{H}(0)}{s}-L\Bigr|\le C\,|s| .$$

Given $\varepsilon>0$, let $\delta$ be the lesser of $t_{0}$ and $\varepsilon/(C+1)$. If $0<|s|<\delta$, then $s\in I$ and $C|s|\le C\varepsilon/(C+1)<\varepsilon$. Hence, by Derivative at an Interior Point, the function $\mathcal{H}$ on $I$ is differentiable at $0$ with derivative $L^{a}_{\mu}(g)$, which is claim 3.