Proof of Score Identities and the Mixture-Weight Directional van Trees Inequality

Steps 1--4 below are adapted from the proof of The Multivariate van Trees Inequality (proof version 24faa7bd-c8a3-4670-a59a-137757233010, cited as provenance), whose hypothesis (iv) enters those steps only through the square-integrability of the scores, which is hypothesis (iv$'$) here; the single map $m$ of the present lemma plays the role of each of the maps $m_1,\dots,m_l$ there. Throughout, $\lambda$, $\mathcal{B}(\mathbb{R})$, $\mathcal{B}_l$, $\lambda_l$ and the insertion maps $\Psi_i$ are those of Finite Products of Lebesgue Measure and Coordinate Integration on $\mathbb{R}^l$; write $\kappa=\lambda_l\otimes\mu$. Expectations are those of Expectation, Variance, and Moments, and integrals those of Lebesgue Integral of a Nonnegative Measurable Function and Integrable Function and the Lebesgue Integral. For $l=1$ we use throughout the conventions of Finite Products of Lebesgue Measure and Coordinate Integration on $\mathbb{R}^l$: a pair $(\theta',y)$ reads as $y$, $\lambda_{l-1}\otimes\mu$ as $\mu$, and $\Psi_1(t,y)=(t,y)$; every step below then applies verbatim, the case $i\ne j$ in Step 4(b) being vacuous.

Step 1 (change of variables). $(\Theta,D)$ is measurable and $Q$ is its image measure, and by assumption (i), $Q$ is the measure with density $p$ with respect to $\kappa$. Hence claims 2 and 3 of Image Measures, Measures with Densities, and Change of Variables give: for every $\mathcal{B}_l\otimes\mathcal{G}$-measurable $g:\mathbb{R}^{l}\times Y\to[0,\infty]$,

and for measurable $g:\mathbb{R}^{l}\times Y\to\mathbb{R}$, the random variable $g\circ(\Theta,D)$ is integrable with respect to $P$ if and only if $gp$ is integrable with respect to $\kappa$, in which case $\mathbb{E}[g(\Theta,D)]=\int gp\,d\kappa$. Taking $g\equiv1$: $\int p\,d\kappa=Q(\mathbb{R}^{l}\times Y)=P(\Omega)=1$.

Step 2 (a one-dimensional vanishing lemma). Let $g:\mathbb{R}\to\mathbb{R}$ be differentiable at every point of $\mathbb{R}$, regarded as an interval each of whose points $x$ is an interior point (as $x-1<x<x+1$ with $x-1,x+1\in\mathbb{R}$), let $g$ and its derivative $g'$ be continuous on $\mathbb{R}$ as maps from the real line with the absolute value metric into itself, and let $g$ and $g'$ be integrable with respect to $\lambda$. (They are measurable: for continuous $u:\mathbb{R}\to\mathbb{R}$ and real $a$, each point of $\{u>a\}$ has an open interval around it inside the set, by continuity, so $\{u>a\}$ is open and hence lies in the Borel $\sigma$-algebra.) Then:

(i) $\int_{\mathbb{R}}g'\,d\lambda=0$.

(ii) If moreover $t\mapsto t\,g(t)$ and $t\mapsto t\,g'(t)$ are integrable with respect to $\lambda$, then $\int_{\mathbb{R}}t\,g'(t)\,d\lambda(t)=-\int_{\mathbb{R}}g\,d\lambda$.

Proof of (i). Let $a<b$ be real and write $[a,b]$ for the closed interval they determine, regarded as a subset of the real line with the absolute value metric. The restrictions of $g$ and of $g'$ to $[a,b]$ are continuous on $[a,b]$ by claim 1 of the restriction lemma. By claim 3 of the interval toolkit, the restriction of $g'$ is therefore Riemann integrable on $[a,b]$ with $\int_a^bg'(t)\,dt=\int_{[a,b]}g'\,d\lambda_{[a,b]}$, and by claim 2 of the same lemma the latter integral is $\int_{\mathbb{R}}g'\,\mathbf{1}_{[a,b]}\,d\lambda$, since the zero extension of the restriction of $g'$ is $g'\mathbf{1}_{[a,b]}$. Moreover $\mathbb{R}$ is an interval containing $[a,b]$ and every $x$ with $a<x<b$ is an interior point of $[a,b]$, so claim 2 of the restriction lemma makes the restriction of $g$ differentiable at every such $x$, with derivative $g'(x)$. Applying the second fundamental theorem of calculus to the restrictions of $g$ and $g'$ gives $g(b)-g(a)=\int_a^bg'(t)\,dt$. Thus

As $n\to\infty$ through the natural numbers, $g'\mathbf{1}_{[0,n]}\to g'\mathbf{1}_{[0,\infty)}$ pointwise, dominated by the integrable $|g'|$, so Dominated Convergence Theorem and ($*$) give $g(n)\to c_+:=g(0)+\int g'\mathbf{1}_{[0,\infty)}\,d\lambda$. Moreover, by ($*$) and monotonicity, $\sup_{t\in[n,n+1]}|g(t)-g(n)|\le\int|g'|\mathbf{1}_{[n,\infty)}\,d\lambda\to0$ (dominated convergence again), so $g(t)\to c_+$ as $t\to\infty$. If $c_+\ne0$, there is $R_0>0$ with $|g(t)|\ge|c_+|/2$ for all $t\ge R_0$; then for every natural $n>R_0$, monotonicity, the integral of simple functions, and Existence of Lebesgue Measure on the Real Line give $\int|g|\,d\lambda\ge(|c_+|/2)\,\lambda([R_0,n])=(|c_+|/2)(n-R_0)\to\infty$, contradicting integrability of $g$; so $c_+=0$. Symmetrically, $g(-n)=g(0)-\int g'\mathbf{1}_{[-n,0]}\,d\lambda$ converges to a limit $c_-$, $g(t)\to c_-$ as $t\to-\infty$, and $c_-=0$. Finally, by dominated convergence and ($*$),

Proof of (ii). $h(t)=t\,g(t)$ is differentiable at every point with $h'(t)=g(t)+t\,g'(t)$ by claim 3 (the product rule) of Sum, Constant Multiple, and Product Rules for One-Dimensional Derivatives (the map $t\mapsto t$ has derivative $1$ directly from the definition of the derivative); $h$ and $h'$ are continuous, by claims 2, 3 and 5 of the continuity of sums and products of real-valued functions on a metric space together with the continuity of the identity map $t\mapsto t$ (for which $\delta=\varepsilon$ works); $h$ is integrable by assumption, and $h'$ is integrable as a sum of integrable functions (Linearity and Monotonicity of the Lebesgue Integral). Part (i) applied to $h$ gives $\int(g(t)+t\,g'(t))\,d\lambda(t)=0$, and linearity gives (ii).

Step 3 (slices of the density). Fix $i\in\{1,\dots,l\}$. For $(\theta',y)\in\mathbb{R}^{l-1}\times Y$ and $t\in\mathbb{R}$ put $g_{\theta',y}(t)=p\bigl(\Psi_i(t,(\theta',y))\bigr)$, and write $\Psi_i(t,(\theta',y))=(\theta(t),y)$, so that $\theta(t)\in\mathbb{R}^{l}$ has $i$th coordinate $t$ and all other coordinates independent of $t$, and $g_{\theta',y}(t)=p(\theta(t),y)$. Since $\theta(t)$ and $\theta(t_0)$ differ only in the $i$th coordinate,

By assumption (ii), $g_{\theta',y}$ is strictly positive.

Differentiability. Fix $t_0\in\mathbb{R}$. By assumption (ii) the function $p(\cdot,y)$ is of class $C^1$ on $\mathbb{R}^{l}$, so by clauses 1 and 3 of the definition of a $C^k$ map its partial derivative with respect to the $i$th variable exists at $\theta(t_0)$. Unwinding that definition: for every real $\varepsilon>0$ there is a real $\delta>0$ such that every real $h$ with $0<|h|<\delta$ satisfies

because the point obtained from $\theta(t_0)$ by increasing its $i$th coordinate by $h$ is exactly $\theta(t_0+h)$. The further requirement in that definition, that the incremented point belong to the domain, is automatic here because the domain is all of $\mathbb{R}^{l}$; what remains is verbatim the assertion that $g_{\theta',y}$ is differentiable at $t_0$ with

the value being unambiguous by Uniqueness of the Partial Derivative on a Euclidean Open Set.

Continuity. By clause 1 of the same definition, $p(\cdot,y)$ and $\partial_ip(\cdot,y)$ are continuous at every point of $\mathbb{R}^{l}$. Let $u$ be either of them, let $t_0\in\mathbb{R}$ and let $\varepsilon>0$ be real; continuity of $u$ at $\theta(t_0)$ supplies a real $\delta>0$ such that every $w\in\mathbb{R}^{l}$ with $\sum_k(w_k-\theta(t_0)_k)^2<\delta^2$ satisfies $\bigl(u(w)-u(\theta(t_0))\bigr)^2<\varepsilon^2$. Taking $w=\theta(t)$ and using the displayed identity, $(t-t_0)^2<\delta^2$ implies $\bigl(u(\theta(t))-u(\theta(t_0))\bigr)^2<\varepsilon^2$; by clause 2 of monotonicity of squaring, applied to the nonnegative reals $|t-t_0|,\delta$ and $|u(\theta(t))-u(\theta(t_0))|,\varepsilon$, this says that $|t-t_0|<\delta$ implies $|u(\theta(t))-u(\theta(t_0))|<\varepsilon$. Hence $g_{\theta',y}$ and $g_{\theta',y}'$ are continuous at every point of $\mathbb{R}$, as maps from the real line with the absolute value metric into itself.

Step 4 (the score identities). Fix $i,j\in\{1,\dots,l\}$ and let $\delta_{ij}=1$ if $i=j$ and $\delta_{ij}=0$ otherwise. We show

We use three elementary facts on a measure space, from Lebesgue Integral of a Nonnegative Measurable Function, Simple Function and Its Integral, and Linearity and Monotonicity of the Lebesgue Integral: (F1) a $[0,\infty]$-valued measurable $u$ with finite integral is finite off a set of measure zero (on $N=\{u=\infty\}$ one has $u\ge L\mathbf{1}_N$ for every $L$, so $L\,\nu(N)\le\int u\,d\nu$ for every $L$, $\nu$ denoting the measure of the space in question); (F2) a $[0,\infty]$-valued measurable $u$ vanishing off a set of measure zero has $\int u\,d\nu=0$ (every simple $s$ with $0\le s\le u$ is bounded by a multiple of the indicator of that set, so $\int s\,dm=0$; take the supremum); (F3) consequently, integrable real functions agreeing off a set of measure zero have equal integrals, and likewise $[0,\infty]$-valued measurable functions (split by the exceptional set and use additivity with (F2)).

(a) $\mathbb{E}[m(D)S_i]=0$. The function $G(\theta,y)=m(y)\,\partial_ip(\theta,y)/p(\theta,y)$ is $\mathcal{B}_l\otimes\mathcal{G}$-measurable: $(\theta,y)\mapsto m(y)$ is measurable (the preimage of a Borel set $A$ is the measurable rectangle $\mathbb{R}^{l}\times m^{-1}(A)$), $\partial_ip/p$ is measurable by Sequentially Continuous Functions of Measurable Euclidean Maps are Measurable applied to the continuous map $(u,v)\mapsto u/v$ on $\mathbb{R}\times(0,\infty)$ composed with $(\partial_ip,p)$, whose components are measurable by assumptions (iii) and (i) and which takes values in $\mathbb{R}\times(0,\infty)$ by assumption (ii), and products of real-valued measurable functions are measurable by Sequentially Continuous Functions of Measurable Euclidean Maps are Measurable (the map $(u,v)\mapsto uv$ is continuous on $\mathbb{R}^{2}$). The random variable $G(\Theta,D)=m(D)S_i$ is integrable, being a product of the square-integrable random variables $m(D)$ and $S_i$ (Square-Integrable Random Variables and the Mean-Square Inner Product). By Step 1, $Gp$ is $\kappa$-integrable and

the second equality holding pointwise because $p>0$ everywhere (assumption (ii)). Apply claim 4 of Finite Products of Lebesgue Measure and Coordinate Integration on $\mathbb{R}^l$ (coordinate Fubini at coordinate $i$) to the $\kappa$-integrable $(\theta,y)\mapsto m(y)\partial_ip(\theta,y)$: there is a set $N_1$ of measure zero off which $t\mapsto m(y)\,g_{\theta',y}'(t)$ is $\lambda$-integrable (Step 3 identifies the integrand), and $\int m\partial_ip\,d\kappa$ equals the integral of the function $F$ given off $N_1$ by $F(\theta',y)=\int_{\mathbb{R}}m(y)g_{\theta',y}'(t)\,d\lambda(t)$ and by $0$ on $N_1$. Now apply claim 3 of Finite Products of Lebesgue Measure and Coordinate Integration on $\mathbb{R}^l$ (coordinate Tonelli) to the nonnegative measurable functions $p$ and $|\partial_ip|$: since $\int p\,d\kappa=1<\infty$ (Step 1) and $\int|\partial_ip|\,d\kappa<\infty$ (assumption (iii)), the measurable $[0,\infty]$-valued functions $(\theta',y)\mapsto\int_{\mathbb{R}}g_{\theta',y}\,d\lambda$ and $(\theta',y)\mapsto\int_{\mathbb{R}}|g_{\theta',y}'|\,d\lambda$ have finite integrals, hence by (F1) are finite off sets $N_2$, $N_3$ of measure zero. For $(\theta',y)\notin N_1\cup N_2\cup N_3$, the function $g_{\theta',y}$ satisfies all hypotheses of Step 2(i) (continuity and differentiability from Step 3, integrability off $N_2\cup N_3$), so $\int_{\mathbb{R}}g_{\theta',y}'\,d\lambda=0$ and, pulling out the constant $m(y)$ by linearity (licensed off $N_3$, where $g_{\theta',y}'$ is $\lambda$-integrable), $F(\theta',y)=0$. Thus $F$ vanishes off a set of measure zero, and $\mathbb{E}[m(D)S_i]=\int F\,d(\lambda_{l-1}\otimes\mu)=0$ by (F3).

(b) $\mathbb{E}[\Theta_jS_i]=-\delta_{ij}$. Take $G(\theta,y)=\theta_j\,\partial_ip(\theta,y)/p(\theta,y)$; the coordinate map $(\theta,y)\mapsto\theta_j$ is measurable (preimages are measurable rectangles), $G(\Theta,D)=\Theta_jS_i$ is integrable as a product of square-integrable random variables, and Step 1 gives

the integrand being $\kappa$-integrable also directly from assumption (iii), since $|\theta_j\partial_ip|\le(1+\sum_{j'}|\theta_{j'}|)|\partial_ip|$ pointwise.

Case $i\ne j$. The $j$th coordinate of $\Psi_i(t,(\theta',y))$ does not depend on $t$: it equals $\theta'_j$ if $j<i$ and $\theta'_{j-1}$ if $j>i$; call it $\theta'_{(j)}$. Coordinate Fubini applied to $\theta_j\partial_ip$ represents the integral through inner integrals $\int_{\mathbb{R}}\theta'_{(j)}\,g_{\theta',y}'(t)\,d\lambda(t)=\theta'_{(j)}\int_{\mathbb{R}}g_{\theta',y}'\,d\lambda=0$ off a set of measure zero, exactly as in (a) with the constant $\theta'_{(j)}$ in place of $m(y)$. Hence $\mathbb{E}[\Theta_jS_i]=0$.

Case $i=j$. Note that the $i$th coordinate of $\Psi_i(t,(\theta',y))$ is exactly $t$. Coordinate Fubini applied to the $\kappa$-integrable $\theta_i\partial_ip$ represents $\mathbb{E}[\Theta_iS_i]$ as the integral of the function $F$ equal, off a set $N_1'$ of measure zero, to $F(\theta',y)=\int_{\mathbb{R}}t\,g_{\theta',y}'(t)\,d\lambda(t)$ and to $0$ on $N_1'$. Two further applications of coordinate Tonelli give sets of measure zero off which $\int_{\mathbb{R}}|t|\,g_{\theta',y}(t)\,d\lambda(t)<\infty$ and $\int_{\mathbb{R}}|t|\,|g_{\theta',y}'(t)|\,d\lambda(t)<\infty$: the first because $\int|\theta_i|\,p\,d\kappa=\mathbb{E}[|\Theta_i|]<\infty$ by Step 1 (square-integrable random variables are integrable, Square-Integrable Random Variables and the Mean-Square Inner Product), the second from assumption (iii) with the weight $|\theta_i|$, both followed by (F1). Off the union of all these sets and $N_2$, $N_3$ of part (a), Step 2(ii) applies to $g_{\theta',y}$ and gives

Let $G_0(\theta',y)=\int_{\mathbb{R}}g_{\theta',y}\,d\lambda$, the $[0,\infty]$-valued measurable function of coordinate Tonelli applied to $p$, with $\int G_0\,d(\lambda_{l-1}\otimes\mu)=\int p\,d\kappa=1$. Then $F$ is integrable (claim 4), $F^{+}$ vanishes off a set of measure zero, and $F^{-}=G_0$ off a set of measure zero; so by (F2) and (F3), $\int F\,d(\lambda_{l-1}\otimes\mu)=-\int G_0\,d(\lambda_{l-1}\otimes\mu)=-1$. Hence $\mathbb{E}[\Theta_iS_i]=-1$.

Together, (a) and (b) are the two identities of claim 1:

Step 5 (claim 1, directional form). Fix $u,\alpha\in\mathbb{R}^{l}$. The random variable $(m(D)-\alpha\cdot\Theta)S_u=\sum_{i=1}^{l}u_i\,m(D)S_i-\sum_{i=1}^{l}\sum_{j=1}^{l}u_i\alpha_j\,\Theta_jS_i$ is a finite linear combination of integrable random variables (products of square-integrable ones), so by linearity of the expectation and ($**$),

This completes the proof of claim 1.

Step 6 (claim 2). Measurability and normalization of $\bar p$. Fix $j$ and let $\tau_j:\mathbb{R}^{l}\times Y\to\mathbb{R}^{l}\times Y$, $\tau_j(\theta,y)=(\theta-a_j,y)$. For a Borel rectangle $R=A_1\times\dots\times A_l$ and $G\in\mathcal{G}$, $\tau_j^{-1}(R\times G)=(R+a_j)\times G$ with $R+a_j=(A_1+(a_j)_1)\times\dots\times(A_l+(a_j)_l)$, a Borel rectangle by claim 1 of Translation and Reflection Invariance of Lebesgue Measure on $\mathbb{R}^n$ applied on the real line; since such sets $R\times G$ generate $\mathcal{B}_l\otimes\mathcal{G}$ (claim 2 of Finite Products of Lebesgue Measure and Coordinate Integration on $\mathbb{R}^l$), $\tau_j$ is measurable by the generator criterion of Measurable Function and Real-Valued Measurable Function, and $p\circ\tau_j=p(\theta-a_j,y)$ is measurable. Hence $\bar p$, a nonnegative linear combination of measurable functions, is measurable, and it is strictly positive because $p>0$ (assumption (ii)). By Tonelli and Fubini Theorems (Tonelli, both measures being $\sigma$-finite) and claim 2 of Translation and Reflection Invariance of Lebesgue Measure on $\mathbb{R}^n$ applied to the section $\theta\mapsto p(\theta,y)$ for each fixed $y$ (sections of product-measurable functions are measurable by the section clause of the same theorem),

so $\int\bar p\,d\kappa=\frac12+\frac{1}{2n}\cdot n=1$ by linearity. More generally, the same two steps give, for every $\mathcal{B}_l\otimes\mathcal{G}$-measurable $F:\mathbb{R}^{l}\times Y\to[0,\infty]$,

(apply claim 2 of Translation and Reflection Invariance of Lebesgue Measure on $\mathbb{R}^n$ to the section $\theta\mapsto F(\theta,y)p(\theta-a_j,y)$ with the shift $a_j$ (that claim gives $\int h(\theta+a_j)\,d\lambda_l=\int h\,d\lambda_l$ for $h(\theta)=F(\theta,y)p(\theta-a_j,y)$), noting that $(\theta,y)\mapsto F(\theta+a_j,y)$ is measurable as $F\circ\tau_j^{-1}$, with $\tau_j^{-1}(\theta,y)=(\theta+a_j,y)$ measurable by the same generator argument).

The mixture-weight information. Fix $u$. The function $\partial_up=\sum_iu_i\partial_ip$ is measurable (assumption (iii)), so $(\partial_up)^{2}/\bar p$ is measurable by Sequentially Continuous Functions of Measurable Euclidean Maps are Measurable applied to the continuous map $(s,t)\mapsto s^{2}/t$ on $\mathbb{R}\times(0,\infty)$. Since $\bar p\ge\frac12p$ pointwise, $(\partial_up)^{2}/\bar p\le2(\partial_up)^{2}/p$, and by Step 1 (change of variables, with $g=(\partial_up)^{2}/p^{2}$, so that $gp=(\partial_up)^{2}/p$ and $g(\Theta,D)=S_u^{2}$),

$S_u$ being square-integrable as a linear combination of the square-integrable $S_i$ (hypothesis (iv$'$)).

The shifted errors. Fix $m$ and $\alpha$, put $e(\theta,y)=m(y)-\alpha\cdot\theta$, a measurable real-valued function on $\mathbb{R}^{l}\times Y$ (as in Step 4, $(\theta,y)\mapsto m(y)$ and the coordinate maps are measurable), and put $\epsilon=\lVert e(\Theta,D)\rVert_2=\lVert m(D)-\alpha\cdot\Theta\rVert_2$ and $c_j=\alpha\cdot a_j$. For each $j$, $e(\theta+a_j,y)=e(\theta,y)-c_j$, so by (T) applied to $F=e^{2}$ and then Step 1,

the last step by the triangle inequality (claim 2 of Cauchy-Schwarz and Triangle Inequalities for the Mean-Square Norm; the constant $c_j$ is square-integrable with norm $|c_j|$) and monotonicity of squaring (Monotonicity of Squaring on the Nonnegative Elements of an Ordered Field). Also $\int e^{2}p\,d\kappa=\epsilon^{2}$ by Step 1. Hence, with $c_{\max}=\max_j|c_j|$ and linearity,

Cauchy-Schwarz with the mixture weight. Apply claim 1 of Weighted Cauchy-Schwarz Inequality on a Measure Space and the Symmetrised Score Functional: Bounds and Averaging on the measure space $(\mathbb{R}^{l}\times Y,\mathcal{B}_l\otimes\mathcal{G},\kappa)$ with $\beta=e^{2}\bar p$ (measurable, nonnegative, finite integral by the previous display) and $\alpha_{\mathrm{CS}}=e\,\partial_up$ (measurable). Where $\beta=0$ one has $e=0$ (as $\bar p>0$), hence $\alpha_{\mathrm{CS}}=0$; and on $\{\beta>0\}$, $\alpha_{\mathrm{CS}}^{2}/\beta=(\partial_up)^{2}/\bar p$, so the function $q$ of that claim is at most $(\partial_up)^{2}/\bar p$ pointwise and $\int q\,d\kappa\le\mathcal{I}_u<\infty$. The claim yields that $e\,\partial_up$ is $\kappa$-integrable and

Finally, $e\,\partial_up=g\,p$ with $g=e\,\partial_up/p$, and $g(\Theta,D)=(m(D)-\alpha\cdot\Theta)S_u$ is integrable (Step 5), so by Step 1 and Step 5, $\int e\,\partial_up\,d\kappa=\mathbb{E}[(m(D)-\alpha\cdot\Theta)S_u]=\alpha\cdot u$. Therefore $(\alpha\cdot u)^{2}\le(\epsilon+c_{\max})^{2}\mathcal{I}_u$, and taking nonnegative square roots (monotone by Monotonicity of Squaring on the Nonnegative Elements of an Ordered Field, with $\sqrt{st}=\sqrt s\sqrt t$ for $s,t\ge0$ as the right side is nonnegative with square $st$) gives $|\alpha\cdot u|\le\sqrt{\mathcal{I}_u}\,(\epsilon+c_{\max})$, which is claim 2. $\blacksquare$