Proof of Relative Scores of the Finite-Dimensional Projections on a Hilbert Space: Orthogonal Projections of the Score, and the Weighted Fisher Information as the Limit of Its Cutoffs

Each result cited is universally quantified over the data in its own statement.

Conventions. Each $p_{n}$ is Borel by Borel Sets of a Hilbert Space with an Orthonormal Basis: Coordinates, Determination by Finite-Dimensional Projections, and Pairs §continuity. For a measure space, $L^{2}$ is the Lebesgue space of The Lebesgue Space of Power-Integrable Functions §space, whose elements are classes $[f]$ of $2$-integrable functions under almost-everywhere equality (The Lebesgue Space of Power-Integrable Functions §equivalence), with inner product $\langle[f],[g]\rangle=\int fg$ and norm $\lVert[f]\rVert^{2}=\int f^{2}$ by The Lebesgue Space of Square-Integrable Functions is a Real Hilbert Space §inner-product; it is a real Hilbert space by The Lebesgue Space of Square-Integrable Functions is a Real Hilbert Space §hilbert. For $1\le n\le m$ let $\pi_{m,n}:\mathbb{R}^{m}\to\mathbb{R}^{n}$, $\pi_{m,n}(y)=(y_{1},\dots,y_{n})$; it is Borel by claims 1, 2 and 5 of The Borel Sigma-Algebra of a Euclidean Space as a Product, and Measurability of Projections, Sequentially Continuous Maps, and Open and Closed Sets, and $p_{n}=\pi_{m,n}\circ p_{m}$. For a closed linear subspace $V$ of $L^{2}(\mu)$, $P_{V}$ is the orthogonal projection of Orthogonal Projection onto a Closed Linear Subspace of a Real Hilbert Space; by Nearest-Point Projection onto a Nonempty Closed Convex Subset of a Real Hilbert Space §existence $P_{V}x\in V$ and $\lVert x-P_{V}x\rVert\le\lVert x-v\rVert$ for every $v\in V$, and by Orthogonal Projection onto a Closed Linear Subspace of a Real Hilbert Space §characterisation $x-P_{V}x$ is orthogonal to every element of $V$. We write $\Pi_{n}=P_{V_{n}}$ once claim 3 is proved. For each $n$, $C^{1}_{b}(\mathbb{R}^{n})$ is the set of Bounded Continuously Differentiable Functions with Bounded Partial Derivatives on Euclidean Space §bounded with $q=n$, both in Bounded C^1 Cylindrical Functions on a Hilbert Space with an Orthonormal Basis §cylindrical and in Finite Fisher Information Relative to a Diagonal Gaussian Measure on Euclidean Space as Componentwise Integration by Parts against Bounded C^1 Functions, the latter being read, as the statement prescribes, with $n$ in place of $d$ and the variance vector $c^{(n)}=(c_{1},\dots,c_{n})$ (Variance Sequences and Their Truncations §truncations). So read, and applied to $\mu_{n}$ (which lies in $\mathcal{P}_{2}(\mathbb{R}^{n})$ by claim 1), the identity of Finite Fisher Information Relative to a Diagonal Gaussian Measure on Euclidean Space as Componentwise Integration by Parts against Bounded C^1 Functions §componentwise reads, for $k\in[n]$ and $\psi\in C^{1}_{b}(\mathbb{R}^{n})$,

$$\langle g,\psi\rangle_{L^{2}(\mu_{n})}=\int_{\mathbb{R}^{n}}\Bigl(\frac{y_{k}}{c_{k}}\psi(y)-\partial_{k}\psi(y)\Bigr)\mu_{n}(dy).\qquad(\mathrm{S}_{n,k})$$

Claim 1. By claim 1 of Image Measures, Measures with Densities, and Change of Variables, $\mu_{n}$ is a Borel probability measure on $\mathbb{R}^{n}$. The function $y\mapsto\lVert y\rVert^{2}$ is nonnegative and Borel (The Second Moment of a Probability Measure on Euclidean Space and the Probability Measures with Finite Second Moment §moment), so by claim 2 of Image Measures, Measures with Densities, and Change of Variables, $\int\lVert y\rVert^{2}\,\mu_{n}(dy)=\int\lVert p_{n}(x)\rVert^{2}\,\mu(dx)$. By Borel Sets of a Hilbert Space with an Orthonormal Basis: Coordinates, Determination by Finite-Dimensional Projections, and Pairs §continuity, $\lVert p_{n}(x)\rVert^{2}=|P_{n}x|^{2}\le|P_{n}x|^{2}+|Q_{n}x|^{2}=|x|^{2}$, so by monotonicity of the integral (Linearity and Monotonicity of the Lebesgue Integral §nonnegative) $M_{2}(\mu_{n})\le M_{2}(\mu)<\infty$ (The Second Moment of a Borel Probability Measure on a Hilbert Space and the Probability Measures with Finite Second Moment §space); hence $\mu_{n}\in\mathcal{P}_{2}(\mathbb{R}^{n})$ (The Second Moment of a Probability Measure on Euclidean Space and the Probability Measures with Finite Second Moment §space).

Claim 2. Let $g:\mathbb{R}^{n}\to\mathbb{R}$ be Borel. Then $g\circ p_{n}$ is Borel, since $(g\circ p_{n})^{-1}(B)=p_{n}^{-1}(g^{-1}(B))$, and $|g|^{2}$ is Borel and nonnegative (Power-Integrable Functions and the p-Seminorm §measurable-power) with $|g|^{2}\circ p_{n}=|g\circ p_{n}|^{2}$. Claim 2 of Image Measures, Measures with Densities, and Change of Variables gives $\int|g|^{2}\,d\mu_{n}=\int|g\circ p_{n}|^{2}\,d\mu$ in $[0,\infty]$; so one side is finite exactly when the other is, and then the two norms agree. If $g'$ is another Borel representative of the class of $g$ in $L^{2}(\mu_{n})$, the Borel set $N=\{g\neq g'\}$ (claim 2 of Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions) is contained in a $\mu_{n}$-null set, so $\mu_{n}(N)=0$, and $\{g\circ p_{n}\neq g'\circ p_{n}\}=p_{n}^{-1}(N)$ has $\mu$-measure $\mu_{n}(N)=0$; so the class of $g\circ p_{n}$ depends only on that of $g$. Moreover, for $2$-integrable Borel $g,g'$ on $\mathbb{R}^{n}$, $gg'$ is $\mu_{n}$-integrable (The Lebesgue Space of Square-Integrable Functions is a Real Hilbert Space §inner-product), so by claim 2 of Image Measures, Measures with Densities, and Change of Variables $(gg')\circ p_{n}$ is $\mu$-integrable with the same integral; that is,

$$\langle g\circ p_{n},g'\circ p_{n}\rangle_{L^{2}(\mu)}=\langle g,g'\rangle_{L^{2}(\mu_{n})}.\qquad(\mathrm{IP})$$

Claim 3. Define $J_{n}:L^{2}(\mu_{n})\to L^{2}(\mu)$, $J_{n}h=h\circ p_{n}$, well defined by claim 2. Since $(sg+tg')\circ p_{n}=s(g\circ p_{n})+t(g'\circ p_{n})$ pointwise and the operations of $L^{2}$ are those of representatives (The Lebesgue Space of Power-Integrable Functions §space), $J_{n}$ is linear, and $\lVert J_{n}h\rVert=\lVert h\rVert$ by claim 2. By claim 2 again, $V_{n}$ is exactly the image of $J_{n}$, hence a linear subspace. Let $v$ lie in the closure of $V_{n}$; by Sequential Characterization of the Closure in a Metric Space there are $h_{j}\in L^{2}(\mu_{n})$ with $J_{n}h_{j}\to v$. By The Norm Metric of a Real Inner Product Space: Triangle Inequalities, Limits and Continuity §triangle, $\lVert h_{j}-h_{l}\rVert=\lVert J_{n}h_{j}-J_{n}h_{l}\rVert\le\lVert J_{n}h_{j}-v\rVert+\lVert v-J_{n}h_{l}\rVert$, so $(h_{j})$ is Cauchy and converges to some $h\in L^{2}(\mu_{n})$ by completeness (Real Hilbert Space §hilbert). Then $\lVert J_{n}h-v\rVert\le\lVert h-h_{j}\rVert+\lVert J_{n}h_{j}-v\rVert\to0$, so $v=J_{n}h\in V_{n}$. Thus $V_{n}$ equals its closure and is closed (claims 1 and 4 of The Closure is the Smallest Closed Superset). Finally, if $v\in V_{n}$ is the class of $g\circ p_{n}$, then $g\circ p_{n}=(g\circ\pi_{n+1,n})\circ p_{n+1}$ with $g\circ\pi_{n+1,n}$ Borel, so $v\in V_{n+1}$. By induction $V_{n}\subseteq V_{m}$ whenever $n\le m$.

Claim 4. Let $(\zeta_{k})$ be the relative score of $\mu$, $n\in\mathbb{N}$ and $k\in[n]$. As $\Pi_{n}\zeta_{k}\in V_{n}$, there is $h_{k}\in L^{2}(\mu_{n})$ with $\Pi_{n}\zeta_{k}=J_{n}h_{k}$. Let $\psi\in C^{1}_{b}(\mathbb{R}^{n})$ and $\varphi=\psi\circ p_{n}\in\mathcal{F}C^{1}_{b}(X)$, with representation $(n,\psi)$ (Bounded C^1 Cylindrical Functions on a Hilbert Space with an Orthonormal Basis §cylindrical). By claim 1 and Finite Fisher Information Relative to a Diagonal Gaussian Measure on Euclidean Space as Componentwise Integration by Parts against Bounded C^1 Functions §integrable, $\psi$ is Borel and $2$-integrable for $\mu_{n}$ and $F(y)=y_{k}\psi(y)/c_{k}-\partial_{k}\psi(y)$ is $\mu_{n}$-integrable; so the class of $\varphi$ is $J_{n}\psi\in V_{n}$. Since $\zeta_{k}-\Pi_{n}\zeta_{k}$ is orthogonal to $V_{n}$, by (IP)

$$\langle h_{k},\psi\rangle_{L^{2}(\mu_{n})}=\langle J_{n}h_{k},J_{n}\psi\rangle_{L^{2}(\mu)}=\langle\zeta_{k},\varphi\rangle_{L^{2}(\mu)}=\int_{X}\Bigl(\frac{x_{k}}{c_{k}}\varphi(x)-\partial_{k}\varphi(x)\Bigr)\mu(dx)$$

by The Relative Score with Respect to a Diagonal Gaussian Measure on a Hilbert Space §score. Since $x_{k}=(p_{n}(x))_{k}$ and the partial derivative $\partial_{k}\varphi$ of Partial Derivatives along an Orthonormal Basis of a Function Differentiable on a Hilbert Space §partial equals $(\partial_{k}\psi)\circ p_{n}$ by Bounded C^1 Cylindrical Functions: Linear Structure, the Gradient and the Partial Derivatives, Bounds and Integrability, and Density in the Square-Integrable Functions §partial, the last integrand is $F\circ p_{n}$, and claim 2 of Image Measures, Measures with Densities, and Change of Variables gives $\int F\circ p_{n}\,d\mu=\int F\,d\mu_{n}$. So $h_{k}$ satisfies $(\mathrm{S}_{n,k})$ for every $\psi\in C^{1}_{b}(\mathbb{R}^{n})$, for each $k\in[n]$. By Finite Fisher Information Relative to a Diagonal Gaussian Measure on Euclidean Space as Componentwise Integration by Parts against Bounded C^1 Functions §componentwise, $\mu_{n}$ has finite Fisher information relative to $\gamma_{c^{(n)}}$, and by Finite Fisher Information Relative to a Diagonal Gaussian Measure on Euclidean Space as Componentwise Integration by Parts against Bounded C^1 Functions §agreement $h_{k}=\zeta^{(n)}_{k}$; hence $\Pi_{n}\zeta_{k}=\zeta^{(n)}_{k}\circ p_{n}$.

Claim 5. First, whenever $\mu_{n}$ has finite Fisher information relative to $\gamma_{c^{(n)}}$,

$$\mathcal{I}^{(n)}_{a}(\mu_{n})=\sum_{k=1}^{n}a_{k}\lVert\zeta^{(n)}_{k}\rVert_{L^{2}(\mu_{n})}^{2}=\sum_{k=1}^{n}a_{k}\lVert\zeta^{(n)}_{k}\circ p_{n}\rVert_{L^{2}(\mu)}^{2}.\qquad(\mathrm{E})$$

Indeed, let $\zeta=(\zeta_{1},\dots,\zeta_{n})$ be a representative of the relative score of $\mu_{n}$. By the convention of Finite Fisher Information Relative to a Diagonal Gaussian Measure on Euclidean Space as Componentwise Integration by Parts against Bounded C^1 Functions (read with $n$ in place of $d$), the class in $L^{2}(\mu_{n})$ of the $k$-th component $\zeta_{k}$ is $\zeta^{(n)}_{k}$; and $\zeta_{k}$ is Borel with $\int\zeta_{k}^{2}\,d\mu_{n}<\infty$ (Square-Integrable Random Vectors: Coordinates, Operations, Almost Sure Equality and the Mean-Square Form §coordinates), so $\int\zeta_{k}^{2}\,d\mu_{n}=\lVert\zeta^{(n)}_{k}\rVert_{L^{2}(\mu_{n})}^{2}$ by The Lebesgue Space of Square-Integrable Functions is a Real Hilbert Space §inner-product. Each $a_{k}\zeta_{k}^{2}$ is nonnegative and Borel (claims 2 and 3 of Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions), with $a_{k}>0$ (Weight Sequences and the Weighted Fisher Information Relative to a Diagonal Gaussian Measure on a Hilbert Space §weights). Hence, by the shorthand of the statement, The Weighted Fisher Information Relative to a Diagonal Gaussian Measure on Euclidean Space §information (with weights $a^{(n)}$ and the representative $\zeta$), and additivity and homogeneity of the integral of nonnegative measurable functions (Linearity and Monotonicity of the Lebesgue Integral §nonnegative) applied repeatedly to the finite sum,

$$\mathcal{I}^{(n)}_{a}(\mu_{n})=\int_{\mathbb{R}^{n}}\sum_{k=1}^{n}a_{k}\,\zeta_{k}(y)^{2}\,\mu_{n}(dy)=\sum_{k=1}^{n}a_{k}\int_{\mathbb{R}^{n}}\zeta_{k}^{2}\,d\mu_{n}=\sum_{k=1}^{n}a_{k}\lVert\zeta^{(n)}_{k}\rVert_{L^{2}(\mu_{n})}^{2},$$

which is the first equality of (E); the second follows from claim 2. Now let $\mu$ have a relative score $(\zeta_{k})$ and $n\in\mathbb{N}$; by claim 4 and (E), $\mathcal{I}^{(n)}_{a}(\mu_{n})=\sum_{k=1}^{n}a_{k}\lVert\Pi_{n}\zeta_{k}\rVert^{2}$. For $k\in[n]$ put $w=\Pi_{n+1}\zeta_{k}$. Since $V_{n}\subseteq V_{n+1}$, both $\zeta_{k}-w$ and $\zeta_{k}-\Pi_{n}\zeta_{k}$ are orthogonal to $V_{n}$, hence so is their difference $w-\Pi_{n}\zeta_{k}$; as $\Pi_{n}\zeta_{k}\in V_{n}$, Orthogonal Projection onto a Closed Linear Subspace of a Real Hilbert Space §characterisation gives $\Pi_{n}w=\Pi_{n}\zeta_{k}$, and Orthogonal Projection onto a Closed Linear Subspace of a Real Hilbert Space §pythagoras gives $\lVert\Pi_{n}\zeta_{k}\rVert=\lVert\Pi_{n}w\rVert\le\lVert w\rVert$. As every $a_{k}>0$ (Weight Sequences and the Weighted Fisher Information Relative to a Diagonal Gaussian Measure on a Hilbert Space §weights),

$$\mathcal{I}^{(n)}_{a}(\mu_{n})\le\sum_{k=1}^{n}a_{k}\lVert\Pi_{n+1}\zeta_{k}\rVert^{2}\le\sum_{k=1}^{n+1}a_{k}\lVert\Pi_{n+1}\zeta_{k}\rVert^{2}=\mathcal{I}^{(n+1)}_{a}(\mu_{n+1}).$$

Claim 6. Write $\mathcal{I}_{a}=\mathcal{I}_{a}(\mu\,|\,\gamma_{c})$. By Orthogonal Projection onto a Closed Linear Subspace of a Real Hilbert Space §pythagoras, $\lVert\Pi_{n}\zeta_{k}\rVert\le\lVert\zeta_{k}\rVert$, so, since $\mathcal{I}^{(n)}_{a}(\mu_{n})=\sum_{k=1}^{n}a_{k}\lVert\Pi_{n}\zeta_{k}\rVert^{2}$ by claim 4 and (E), and by Series of Nonnegative Real Numbers, Comparison, and the Geometric Series §dominates, $\mathcal{I}^{(n)}_{a}(\mu_{n})\le\sum_{k=1}^{n}a_{k}\lVert\zeta_{k}\rVert^{2}\le\mathcal{I}_{a}$. Next fix $k$; we show $\Pi_{n}\zeta_{k}\to\zeta_{k}$. Let $\varepsilon>0$. The classes of $\mathcal{F}C^{1}_{b}(X)$ are dense in $L^{2}(\mu)$ (Bounded C^1 Cylindrical Functions: Linear Structure, the Gradient and the Partial Derivatives, Bounds and Integrability, and Density in the Square-Integrable Functions §density), so by Dense Subset of a Topological Space and Characterization of the Closure in a Metric Space by Open Balls there is $\varphi\in\mathcal{F}C^{1}_{b}(X)$ with $\lVert\zeta_{k}-\varphi\rVert<\varepsilon$. If $(m,\psi)$ represents $\varphi$, its class is $J_{m}\psi\in V_{m}$ (as in claim 4), so the class of $\varphi$ lies in $V_{n}$ for $n\ge m$ by claim 3, and then $\lVert\zeta_{k}-\Pi_{n}\zeta_{k}\rVert\le\lVert\zeta_{k}-\varphi\rVert<\varepsilon$. Hence $\lVert\Pi_{n}\zeta_{k}\rVert\to\lVert\zeta_{k}\rVert$ (The Norm Metric of a Real Inner Product Space: Triangle Inequalities, Limits and Continuity §continuity). Now let $\varepsilon>0$; first choose $K$ with $\sum_{k=1}^{K}a_{k}\lVert\zeta_{k}\rVert^{2}\ge\mathcal{I}_{a}-\varepsilon$ (Series of Real Numbers §convergent); then, the finite sum $\sum_{k=1}^{K}a_{k}\lVert\Pi_{n}\zeta_{k}\rVert^{2}$ converging to $\sum_{k=1}^{K}a_{k}\lVert\zeta_{k}\rVert^{2}$, choose $N\ge K$ with $\sum_{k=1}^{K}a_{k}\lVert\Pi_{n}\zeta_{k}\rVert^{2}\ge\mathcal{I}_{a}-2\varepsilon$ for $n\ge N$. For $n\ge N$, dropping the nonnegative terms with $K<k\le n$,

$$\mathcal{I}_{a}-2\varepsilon\le\sum_{k=1}^{K}a_{k}\lVert\Pi_{n}\zeta_{k}\rVert^{2}\le\mathcal{I}^{(n)}_{a}(\mu_{n})\le\mathcal{I}_{a},$$

so $\mathcal{I}^{(n)}_{a}(\mu_{n})\to\mathcal{I}_{a}$.

Claim 7. For $m\in\mathbb{N}$ and $k\in[m]$ put $G^{m}_{k}=\zeta^{(m)}_{k}\circ p_{m}\in V_{m}$. Step 1: for $k\le m$ and every $\varphi\in\mathcal{F}C^{1}_{b}(X)$ with a representation $(n,\psi)$ such that $n\le m$,

$$\langle G^{m}_{k},\varphi\rangle_{L^{2}(\mu)}=\int_{X}\Bigl(\frac{x_{k}}{c_{k}}\varphi(x)-\partial_{k}\varphi(x)\Bigr)\mu(dx).\qquad(*)$$

Put $\pi=\pi_{m,n}$ and $\psi'=\psi\circ\pi$. Write $\varepsilon^{(m)}_{l}$ for the $l$-th standard basis vector of $\mathbb{R}^{m}$ and $\varepsilon^{(n)}_{l}$ for that of $\mathbb{R}^{n}$ (not to be confused with the basis $(e_{k})$ of $X$). For $y\in\mathbb{R}^{m}$, $l\in[m]$ and real $t$, $\pi(y+t\varepsilon^{(m)}_{l})=\pi(y)+t\varepsilon^{(n)}_{l}$ if $l\le n$ and $\pi(y+t\varepsilon^{(m)}_{l})=\pi(y)$ if $l>n$; so the difference quotients of Partial Derivative on a Euclidean Open Set give $\partial_{l}\psi'(y)=\partial_{l}\psi(\pi(y))$ for $l\le n$ and $\partial_{l}\psi'(y)=0$ for $l>n$. Since $\sum_{i=1}^{n}(\pi(y')_{i}-\pi(y)_{i})^{2}\le\sum_{i=1}^{m}(y'_{i}-y_{i})^{2}$, continuity at every point (Continuity at a Point for Maps Between Euclidean Spaces) of $\psi$ and of the $\partial_{l}\psi$ (clause 1 of C^k Maps on a Euclidean Open Set) passes to $\psi'$ and to $\partial_{l}\psi'=(\partial_{l}\psi)\circ\pi$ for $l\le n$, and $\partial_{l}\psi'=0$ is continuous for $l>n$; so $\psi'$ is of class $C^{1}$, and bounded with bounded partial derivatives, i.e. $\psi'\in C^{1}_{b}(\mathbb{R}^{m})$ (Bounded Continuously Differentiable Functions with Bounded Partial Derivatives on Euclidean Space §bounded with $q=m$). As $\psi'\circ p_{m}=\varphi$, the class of $\varphi$ is $J_{m}\psi'$. By (IP), Finite Fisher Information Relative to a Diagonal Gaussian Measure on Euclidean Space as Componentwise Integration by Parts against Bounded C^1 Functions §agreement at level $m$ (which applies since $\mu_{m}$ has finite Fisher information) and claim 2 of Image Measures, Measures with Densities, and Change of Variables (the integrand being $\mu_{m}$-integrable by Finite Fisher Information Relative to a Diagonal Gaussian Measure on Euclidean Space as Componentwise Integration by Parts against Bounded C^1 Functions §integrable),

$$\langle G^{m}_{k},\varphi\rangle_{L^{2}(\mu)}=\langle\zeta^{(m)}_{k},\psi'\rangle_{L^{2}(\mu_{m})}=\int_{X}\Bigl(\frac{x_{k}}{c_{k}}\psi'(p_{m}(x))-\partial_{k}\psi'(p_{m}(x))\Bigr)\mu(dx),$$

using $(p_{m}(x))_{k}=x_{k}$. Here $\psi'(p_{m}(x))=\varphi(x)$, and $\partial_{k}\psi'(p_{m}(x))$ equals $\partial_{k}\psi(p_{n}(x))=\partial_{k}\varphi(x)$ if $k\le n$ and $0=\partial_{k}\varphi(x)$ if $k>n$, by Bounded C^1 Cylindrical Functions: Linear Structure, the Gradient and the Partial Derivatives, Bounds and Integrability, and Density in the Square-Integrable Functions §partial. This proves $(*)$.

Step 2: for $k\le n\le m$, $\Pi_{n}G^{m}_{k}=G^{n}_{k}$. Write $\Pi_{n}G^{m}_{k}=J_{n}h$ with $h\in L^{2}(\mu_{n})$. For $\psi\in C^{1}_{b}(\mathbb{R}^{n})$ and $\varphi=\psi\circ p_{n}$, by (IP), orthogonality of $G^{m}_{k}-\Pi_{n}G^{m}_{k}$ to $V_{n}\ni J_{n}\psi$, $(*)$, Bounded C^1 Cylindrical Functions: Linear Structure, the Gradient and the Partial Derivatives, Bounds and Integrability, and Density in the Square-Integrable Functions §partial and claim 2 of Image Measures, Measures with Densities, and Change of Variables as in claim 4,

$$\langle h,\psi\rangle_{L^{2}(\mu_{n})}=\langle G^{m}_{k},\varphi\rangle_{L^{2}(\mu)}=\int_{\mathbb{R}^{n}}\Bigl(\frac{y_{k}}{c_{k}}\psi(y)-\partial_{k}\psi(y)\Bigr)\mu_{n}(dy).$$

So $h$ satisfies $(\mathrm{S}_{n,k})$, and $h=\zeta^{(n)}_{k}$ by the uniqueness in Finite Fisher Information Relative to a Diagonal Gaussian Measure on Euclidean Space as Componentwise Integration by Parts against Bounded C^1 Functions §agreement. By Orthogonal Projection onto a Closed Linear Subspace of a Real Hilbert Space §pythagoras,

$$\lVert G^{m}_{k}\rVert^{2}=\lVert G^{n}_{k}\rVert^{2}+\lVert G^{m}_{k}-G^{n}_{k}\rVert^{2}\qquad(k\le n\le m).$$

Step 3: limits. Fix $k$ and put $s_{m}=\lVert G^{m}_{k}\rVert^{2}$ for $m\ge k$. By Step 2, $(s_{m})_{m\ge k}$ is nondecreasing, and by (E) and positivity of the weights $a_{k}s_{m}\le\mathcal{I}^{(m)}_{a}(\mu_{m})\le C$; so $(s_{m})$ converges, hence is Cauchy. Given $\varepsilon>0$, choose $N\ge k$ with $|s_{m}-s_{n}|<\varepsilon^{2}$ for $m,n\ge N$; then by Step 2 $\lVert G^{m}_{k}-G^{n}_{k}\rVert^{2}=s_{m}-s_{n}<\varepsilon^{2}$ for $m\ge n\ge N$. So $(G^{m}_{k})_{m\ge k}$ is Cauchy in $L^{2}(\mu)$ and converges to some $\zeta_{k}\in L^{2}(\mu)$ (Real Hilbert Space §hilbert). Let $\varphi\in\mathcal{F}C^{1}_{b}(X)$ with representation $(n,\psi)$. For every $m\ge\max(n,k)$, $(*)$ gives $\langle G^{m}_{k},\varphi\rangle$ equal to the right side of the score identity; by The Norm Metric of a Real Inner Product Space: Triangle Inequalities, Limits and Continuity §continuity, $\langle G^{m}_{k},\varphi\rangle\to\langle\zeta_{k},\varphi\rangle$, so $\zeta_{k}$ satisfies the identity of The Relative Score with Respect to a Diagonal Gaussian Measure on a Hilbert Space §score. Hence $\mu$ (which lies in $\mathcal{P}_{2}(X)$) has the relative score $(\zeta_{k})_{k\in\mathbb{N}}$. Finally let $K\in\mathbb{N}$. For $m\ge K$, (E) and the nonnegativity of the omitted terms give $\sum_{k=1}^{K}a_{k}\lVert G^{m}_{k}\rVert^{2}\le\mathcal{I}^{(m)}_{a}(\mu_{m})\le C$; letting $m\to\infty$, with $\lVert G^{m}_{k}\rVert\to\lVert\zeta_{k}\rVert$ by The Norm Metric of a Real Inner Product Space: Triangle Inequalities, Limits and Continuity §continuity, gives $\sum_{k=1}^{K}a_{k}\lVert\zeta_{k}\rVert^{2}\le C$. The terms being nonnegative, Series of Nonnegative Real Numbers, Comparison, and the Geometric Series §criterion shows that $\sum_{k=1}^{\infty}a_{k}\lVert\zeta_{k}\rVert^{2}$ converges with sum at most $C$; by Weight Sequences and the Weighted Fisher Information Relative to a Diagonal Gaussian Measure on a Hilbert Space §information, $\mu$ has finite Fisher information relative to $\gamma_{c}$ with weights $a$ and $\mathcal{I}_{a}(\mu\,|\,\gamma_{c})\le C$.