Proof of The Multivariate van Trees Inequality

Throughout, $\lambda$, $\mathcal{B}(\mathbb{R})$, $\mathcal{B}_l$, $\lambda_l$ and the insertion maps $\Psi_i$ are those of Finite Products of Lebesgue Measure and Coordinate Integration on $\mathbb{R}^l$; write $\kappa=\lambda_l\otimes\mu$. Expectations are those of Expectation, Variance, and Moments, and integrals those of Lebesgue Integral of a Nonnegative Measurable Function and Integrable Function and the Lebesgue Integral. For $l=1$ we use throughout the conventions of Finite Products of Lebesgue Measure and Coordinate Integration on $\mathbb{R}^l$: a pair $(\theta',y)$ reads as $y$, $\lambda_{l-1}\otimes\mu$ as $\mu$, and $\Psi_1(t,y)=(t,y)$; every step below then applies verbatim, the case $i\ne j$ in Step 4(b) being vacuous.

Step 1 (change of variables). $(\Theta,D)$ is measurable and $Q$ is its image measure, and by assumption (i), $Q$ is the measure with density $p$ with respect to $\kappa$. Hence claims 2 and 3 of Image Measures, Measures with Densities, and Change of Variables give: for every $\mathcal{B}_l\otimes\mathcal{G}$-measurable $g:\mathbb{R}^{l}\times Y\to[0,\infty]$,

and for measurable $g:\mathbb{R}^{l}\times Y\to\mathbb{R}$, the random variable $g\circ(\Theta,D)$ is integrable with respect to $P$ if and only if $gp$ is integrable with respect to $\kappa$, in which case $\mathbb{E}[g(\Theta,D)]=\int gp\,d\kappa$. Taking $g\equiv1$: $\int p\,d\kappa=Q(\mathbb{R}^{l}\times Y)=P(\Omega)=1$.

Step 2 (a one-dimensional vanishing lemma). Let $g:\mathbb{R}\to\mathbb{R}$ be differentiable at every point with $g$ and its derivative $g'$ continuous, and let $g$ and $g'$ be integrable with respect to $\lambda$. (They are measurable: for continuous $u:\mathbb{R}\to\mathbb{R}$ and real $a$, each point of $\{u>a\}$ has an open interval around it inside the set, by continuity, so $\{u>a\}$ is open and hence lies in the Borel $\sigma$-algebra.) Then:

(i) $\int_{\mathbb{R}}g'\,d\lambda=0$.

(ii) If moreover $t\mapsto t\,g(t)$ and $t\mapsto t\,g'(t)$ are integrable with respect to $\lambda$, then $\int_{\mathbb{R}}t\,g'(t)\,d\lambda(t)=-\int_{\mathbb{R}}g\,d\lambda$.

Proof of (i). For real $a<b$, the function $\tilde g(s)=g(a+s)$ on $[0,b-a]$ has difference quotients at $s$ coinciding with those of $g$ at $a+s$, so $\tilde g$ is continuous and differentiable at every point of $[0,b-a]$ with continuous derivative $\tilde g'(s)=g'(a+s)$, i.e. $\tilde g\in C^{1}([0,b-a])$, and the Fundamental Theorem of Calculus Fundamental Theorem of Calculus in One Dimension gives $g(b)-g(a)=\int_0^{b-a}g'(a+s)\,ds$, read as the Riemann integral of the continuous integrand $\tilde g'$, which exists by Continuous Functions on a Closed Interval are Riemann Integrable. By Agreement of the Riemann and Lebesgue Integrals for Continuous Functions on a Closed Interval this equals the Lebesgue integral $\int_{\mathbb{R}}g'(a+s)\,\mathbf{1}_{[0,b-a]}(s)\,d\lambda(s)$, which equals $\int_{\mathbb{R}}g'\,\mathbf{1}_{[a,b]}\,d\lambda$ by claim 2 of Translation Invariance of Lebesgue Measure and the Lebesgue Integral applied to the positive and negative parts of $g'\mathbf{1}_{[a,b]}$ (whose translates by $a$ are the positive and negative parts of $g'(a+\cdot)\mathbf{1}_{[0,b-a]}$; each part is bounded — continuous functions on a compact interval are bounded, Continuous Real-Valued Functions on a Compact Interval are Bounded — and supported in an interval of finite measure, hence of finite integral, so the two nonnegative identities may be subtracted). Thus

As $n\to\infty$ through the natural numbers, $g'\mathbf{1}_{[0,n]}\to g'\mathbf{1}_{[0,\infty)}$ pointwise, dominated by the integrable $|g'|$, so Dominated Convergence Theorem and ($*$) give $g(n)\to c_+:=g(0)+\int g'\mathbf{1}_{[0,\infty)}\,d\lambda$. Moreover, by ($*$) and monotonicity, $\sup_{t\in[n,n+1]}|g(t)-g(n)|\le\int|g'|\mathbf{1}_{[n,\infty)}\,d\lambda\to0$ (dominated convergence again), so $g(t)\to c_+$ as $t\to\infty$. If $c_+\ne0$, there is $R_0>0$ with $|g(t)|\ge|c_+|/2$ for all $t\ge R_0$; then for every natural $n>R_0$, monotonicity, the integral of simple functions, and Existence of Lebesgue Measure on the Real Line give $\int|g|\,d\lambda\ge(|c_+|/2)\,\lambda([R_0,n])=(|c_+|/2)(n-R_0)\to\infty$, contradicting integrability of $g$; so $c_+=0$. Symmetrically, $g(-n)=g(0)-\int g'\mathbf{1}_{[-n,0]}\,d\lambda$ converges to a limit $c_-$, $g(t)\to c_-$ as $t\to-\infty$, and $c_-=0$. Finally, by dominated convergence and ($*$),

Proof of (ii). $h(t)=t\,g(t)$ is differentiable at every point with $h'(t)=g(t)+t\,g'(t)$ by the product rule of Sum and Product Rules for One-Dimensional Derivatives and Continuity (the map $t\mapsto t$ has derivative $1$ from the definition of the derivative); $h$ and $h'$ are continuous, $h$ is integrable by assumption, and $h'$ is integrable as a sum of integrable functions (Linearity and Monotonicity of the Lebesgue Integral). Part (i) applied to $h$ gives $\int(g(t)+t\,g'(t))\,d\lambda(t)=0$, and linearity gives (ii).

Step 3 (slices of the density). Fix $i\in\{1,\dots,l\}$. For $(\theta',y)\in\mathbb{R}^{l-1}\times Y$ and $t\in\mathbb{R}$ put $g_{\theta',y}(t)=p\bigl(\Psi_i(t,(\theta',y))\bigr)$. The map $t\mapsto(\theta'_1,\dots,\theta'_{i-1},t,\theta'_i,\dots,\theta'_{l-1})$ from $\mathbb{R}$ to $\mathbb{R}^{l}$ is continuous, its coordinate functions being constant or the identity (Coordinatewise Characterization of Continuity for Euclidean Maps). By assumption (ii) and Slice Function and the Partial Derivative, applied to the $C^1$ function $p(\cdot,y)$ on $\mathbb{R}^{l}$ at each inserted point in the $i$th variable, $g_{\theta',y}$ is strictly positive and differentiable at every $t\in\mathbb{R}$ with

(The cited lemma yields, at each fixed $t_0$, an interval $I$ around $t_0$ on which its slice function coincides with $g_{\theta',y}$, and equates the slice derivative at the interior point $t_0$ with the partial derivative; differentiability at $t_0$ and the value of the derivative depend only on the restriction to $I$, so the identity holds at every $t_0\in\mathbb{R}$.) Both $g_{\theta',y}$ and $g_{\theta',y}'$ are continuous, being compositions of the continuous insertion with the continuous functions $p(\cdot,y)$ and $\partial_ip(\cdot,y)$ (Composition of Continuous Euclidean Maps; continuity of these is part of the $C^1$ property).

Step 4 (the score identities). Fix $i,j\in\{1,\dots,l\}$ and let $\delta_{ij}=1$ if $i=j$ and $\delta_{ij}=0$ otherwise. We show

We use three elementary facts on a measure space, from Lebesgue Integral of a Nonnegative Measurable Function, Simple Function and Its Integral, and Linearity and Monotonicity of the Lebesgue Integral: (F1) a $[0,\infty]$-valued measurable $u$ with finite integral is finite off a set of measure zero (on $N=\{u=\infty\}$ one has $u\ge L\mathbf{1}_N$ for every $L$, so $L\,m(N)\le\int u\,dm$ for every $L$); (F2) a $[0,\infty]$-valued measurable $u$ vanishing off a set of measure zero has $\int u\,dm=0$ (every simple $s$ with $0\le s\le u$ is bounded by a multiple of the indicator of that set, so $\int s\,dm=0$; take the supremum); (F3) consequently, integrable real functions agreeing off a set of measure zero have equal integrals, and likewise $[0,\infty]$-valued measurable functions (split by the exceptional set and use additivity with (F2)).

(a) $\mathbb{E}[m_j(D)S_i]=0$. The function $G(\theta,y)=m_j(y)\,\partial_ip(\theta,y)/p(\theta,y)$ is $\mathcal{B}_l\otimes\mathcal{G}$-measurable: $(\theta,y)\mapsto m_j(y)$ is measurable (the preimage of a Borel set $A$ is the measurable rectangle $\mathbb{R}^{l}\times m_j^{-1}(A)$), $\partial_ip/p$ is measurable as in assumption (iv), and products of real-valued measurable functions are measurable by Sequentially Continuous Functions of Measurable Euclidean Maps are Measurable (the map $(u,v)\mapsto uv$ is continuous on $\mathbb{R}^{2}$). The random variable $G(\Theta,D)=m_j(D)S_i$ is integrable, being a product of the square-integrable random variables $m_j(D)$ and $S_i$ (Square-Integrable Random Variables and the Mean-Square Inner Product). By Step 1, $Gp$ is $\kappa$-integrable and

the second equality holding pointwise because $p>0$ everywhere (assumption (ii)). Apply claim 4 of Finite Products of Lebesgue Measure and Coordinate Integration on $\mathbb{R}^l$ (coordinate Fubini at coordinate $i$) to the $\kappa$-integrable $(\theta,y)\mapsto m_j(y)\partial_ip(\theta,y)$: there is a set $N_1$ of measure zero off which $t\mapsto m_j(y)\,g_{\theta',y}'(t)$ is $\lambda$-integrable (Step 3 identifies the integrand), and $\int m_j\partial_ip\,d\kappa$ equals the integral of the function $F$ given off $N_1$ by $F(\theta',y)=\int_{\mathbb{R}}m_j(y)g_{\theta',y}'(t)\,d\lambda(t)$ and by $0$ on $N_1$. Now apply claim 3 of Finite Products of Lebesgue Measure and Coordinate Integration on $\mathbb{R}^l$ (coordinate Tonelli) to the nonnegative measurable functions $p$ and $|\partial_ip|$: since $\int p\,d\kappa=1<\infty$ (Step 1) and $\int|\partial_ip|\,d\kappa<\infty$ (assumption (iii)), the measurable $[0,\infty]$-valued functions $(\theta',y)\mapsto\int_{\mathbb{R}}g_{\theta',y}\,d\lambda$ and $(\theta',y)\mapsto\int_{\mathbb{R}}|g_{\theta',y}'|\,d\lambda$ have finite integrals, hence by (F1) are finite off sets $N_2$, $N_3$ of measure zero. For $(\theta',y)\notin N_1\cup N_2\cup N_3$, the function $g_{\theta',y}$ satisfies all hypotheses of Step 2(i) (continuity and differentiability from Step 3, integrability off $N_2\cup N_3$), so $\int_{\mathbb{R}}g_{\theta',y}'\,d\lambda=0$ and, pulling out the constant $m_j(y)$ by linearity (licensed off $N_3$, where $g_{\theta',y}'$ is $\lambda$-integrable), $F(\theta',y)=0$. Thus $F$ vanishes off a set of measure zero, and $\mathbb{E}[m_j(D)S_i]=\int F\,d(\lambda_{l-1}\otimes\mu)=0$ by (F3).

(b) $\mathbb{E}[\Theta_jS_i]=-\delta_{ij}$. Take $G(\theta,y)=\theta_j\,\partial_ip(\theta,y)/p(\theta,y)$; the coordinate map $(\theta,y)\mapsto\theta_j$ is measurable (preimages are measurable rectangles), $G(\Theta,D)=\Theta_jS_i$ is integrable as a product of square-integrable random variables, and Step 1 gives

the integrand being $\kappa$-integrable also directly from assumption (iii), since $|\theta_j\partial_ip|\le(1+\sum_{j'}|\theta_{j'}|)|\partial_ip|$ pointwise.

Case $i\ne j$. The $j$th coordinate of $\Psi_i(t,(\theta',y))$ does not depend on $t$: it equals $\theta'_j$ if $j<i$ and $\theta'_{j-1}$ if $j>i$; call it $\theta'_{(j)}$. Coordinate Fubini applied to $\theta_j\partial_ip$ represents the integral through inner integrals $\int_{\mathbb{R}}\theta'_{(j)}\,g_{\theta',y}'(t)\,d\lambda(t)=\theta'_{(j)}\int_{\mathbb{R}}g_{\theta',y}'\,d\lambda=0$ off a set of measure zero, exactly as in (a) with the constant $\theta'_{(j)}$ in place of $m_j(y)$. Hence $\mathbb{E}[\Theta_jS_i]=0$.

Case $i=j$. Note that the $i$th coordinate of $\Psi_i(t,(\theta',y))$ is exactly $t$. Coordinate Fubini applied to the $\kappa$-integrable $\theta_i\partial_ip$ represents $\mathbb{E}[\Theta_iS_i]$ as the integral of the function $F$ equal, off a set $N_1'$ of measure zero, to $F(\theta',y)=\int_{\mathbb{R}}t\,g_{\theta',y}'(t)\,d\lambda(t)$ and to $0$ on $N_1'$. Two further applications of coordinate Tonelli give sets of measure zero off which $\int_{\mathbb{R}}|t|\,g_{\theta',y}(t)\,d\lambda(t)<\infty$ and $\int_{\mathbb{R}}|t|\,|g_{\theta',y}'(t)|\,d\lambda(t)<\infty$: the first because $\int|\theta_i|\,p\,d\kappa=\mathbb{E}[|\Theta_i|]<\infty$ by Step 1 (square-integrable random variables are integrable, Square-Integrable Random Variables and the Mean-Square Inner Product), the second from assumption (iii) with the weight $|\theta_i|$, both followed by (F1). Off the union of all these sets and $N_2$, $N_3$ of part (a), Step 2(ii) applies to $g_{\theta',y}$ and gives

Let $G_0(\theta',y)=\int_{\mathbb{R}}g_{\theta',y}\,d\lambda$, the $[0,\infty]$-valued measurable function of coordinate Tonelli applied to $p$, with $\int G_0\,d(\lambda_{l-1}\otimes\mu)=\int p\,d\kappa=1$. Then $F$ is integrable (claim 4), $F^{+}$ vanishes off a set of measure zero, and $F^{-}=G_0$ off a set of measure zero; so by (F2) and (F3), $\int F\,d(\lambda_{l-1}\otimes\mu)=-\int G_0\,d(\lambda_{l-1}\otimes\mu)=-1$. Hence $\mathbb{E}[\Theta_iS_i]=-1$.

Combining (a) and (b) with linearity of the expectation (Linearity and Monotonicity of the Lebesgue Integral),

Step 5 (Cauchy-Schwarz step). Each $m_j(D)-\Theta_j$ is square-integrable (Square-Integrable Random Variables and the Mean-Square Inner Product), so every product $(m_i(D)-\Theta_i)(m_j(D)-\Theta_j)$ is integrable, $R$ is well defined, and $R_{ij}=R_{ji}$ by commutativity of pointwise multiplication. $J$ is symmetric positive definite by assumption (iv), so $J^{-1}$ exists and is symmetric positive definite by Invertibility of Symmetric Positive Definite Matrices. We use the dot product and the matrix-vector product on $\mathbb{R}^{l}$; componentwise, $a\cdot(Ma)=\sum_{i,j}a_iM_{ij}a_j$ for any $l\times l$ matrix $M$; $A(Bx)=(AB)x$ for $l\times l$ matrices, since $(A(Bx))_i=\sum_jA_{ij}\sum_mB_{jm}x_m=\sum_m(AB)_{im}x_m$ with the matrix product; and $Ix=x$ for the identity matrix $I$ of Inverse Matrix and Invertible Real Square Matrix, since $(Ix)_i=\sum_jI_{ij}x_j=x_i$.

Fix $a\in\mathbb{R}^{l}$ and set $b=J^{-1}a$, and

both square-integrable as linear combinations of square-integrable random variables. Expanding the products and using linearity of the expectation:

by ($**$); and

since $Jb=J(J^{-1}a)=(JJ^{-1})a=Ia=a$ by the identities above and the definition of the inverse, and the dot product is symmetric. Since $(X-W)^{2}\ge0$ pointwise, monotonicity and linearity of the expectation give

the last step by componentwise bilinearity. As $a$ was arbitrary and $R-J^{-1}$ is symmetric, $R-J^{-1}$ is positive semidefinite; that is, $R\succeq J^{-1}$ in the semidefinite order. $\blacksquare$