Proof of The Multivariate van Trees Inequality

Throughout, $\lambda$, $\mathcal{B}(\mathbb{R})$, $\mathcal{B}_l$, $\lambda_l$ and the insertion maps $\Psi_i$ are those of Finite Products of Lebesgue Measure and Coordinate Integration on $\mathbb{R}^l$; write $\kappa=\lambda_l\otimes\mu$. Expectations are those of Expectation, Variance, and Moments, and integrals those of Lebesgue Integral of a Nonnegative Measurable Function and Integrable Function and the Lebesgue Integral. For $l=1$ we use throughout the conventions of Finite Products of Lebesgue Measure and Coordinate Integration on $\mathbb{R}^l$: a pair $(\theta',y)$ reads as $y$, $\lambda_{l-1}\otimes\mu$ as $\mu$, and $\Psi_1(t,y)=(t,y)$; every step below then applies verbatim, the case $i\ne j$ in Step 4(b) being vacuous.

Step 1 (change of variables). $(\Theta,D)$ is measurable and $Q$ is its image measure, and by assumption (i), $Q$ is the measure with density $p$ with respect to $\kappa$. Hence claims 2 and 3 of Image Measures, Measures with Densities, and Change of Variables give: for every $\mathcal{B}_l\otimes\mathcal{G}$-measurable $g:\mathbb{R}^{l}\times Y\to[0,\infty]$,

and for measurable $g:\mathbb{R}^{l}\times Y\to\mathbb{R}$, the random variable $g\circ(\Theta,D)$ is integrable with respect to $P$ if and only if $gp$ is integrable with respect to $\kappa$, in which case $\mathbb{E}[g(\Theta,D)]=\int gp\,d\kappa$. Taking $g\equiv1$: $\int p\,d\kappa=Q(\mathbb{R}^{l}\times Y)=P(\Omega)=1$.

Step 2 (a one-dimensional vanishing lemma). Let $g:\mathbb{R}\to\mathbb{R}$ be differentiable at every point of $\mathbb{R}$, regarded as an interval each of whose points $x$ is an interior point (as $x-1<x<x+1$ with $x-1,x+1\in\mathbb{R}$), let $g$ and its derivative $g'$ be continuous on $\mathbb{R}$ as maps from the real line with the absolute value metric into itself, and let $g$ and $g'$ be integrable with respect to $\lambda$. (They are measurable: for continuous $u:\mathbb{R}\to\mathbb{R}$ and real $a$, each point of $\{u>a\}$ has an open interval around it inside the set, by continuity, so $\{u>a\}$ is open and hence lies in the Borel $\sigma$-algebra.) Then:

(i) $\int_{\mathbb{R}}g'\,d\lambda=0$.

(ii) If moreover $t\mapsto t\,g(t)$ and $t\mapsto t\,g'(t)$ are integrable with respect to $\lambda$, then $\int_{\mathbb{R}}t\,g'(t)\,d\lambda(t)=-\int_{\mathbb{R}}g\,d\lambda$.

Proof of (i). Let $a<b$ be real and write $[a,b]$ for the closed interval they determine, regarded as a subset of the real line with the absolute value metric. The restrictions of $g$ and of $g'$ to $[a,b]$ are continuous on $[a,b]$ by claim 1 of the restriction lemma. By claim 3 of the interval toolkit, the restriction of $g'$ is therefore Riemann integrable on $[a,b]$ with $\int_a^bg'(t)\,dt=\int_{[a,b]}g'\,d\lambda_{[a,b]}$, and by claim 2 of the same lemma the latter integral is $\int_{\mathbb{R}}g'\,\mathbf{1}_{[a,b]}\,d\lambda$, since the zero extension of the restriction of $g'$ is $g'\mathbf{1}_{[a,b]}$. Moreover $\mathbb{R}$ is an interval containing $[a,b]$ and every $x$ with $a<x<b$ is an interior point of $[a,b]$, so claim 2 of the restriction lemma makes the restriction of $g$ differentiable at every such $x$, with derivative $g'(x)$. Applying the second fundamental theorem of calculus to the restrictions of $g$ and $g'$ gives $g(b)-g(a)=\int_a^bg'(t)\,dt$. Thus

As $n\to\infty$ through the natural numbers, $g'\mathbf{1}_{[0,n]}\to g'\mathbf{1}_{[0,\infty)}$ pointwise, dominated by the integrable $|g'|$, so Dominated Convergence Theorem and ($*$) give $g(n)\to c_+:=g(0)+\int g'\mathbf{1}_{[0,\infty)}\,d\lambda$. Moreover, by ($*$) and monotonicity, $\sup_{t\in[n,n+1]}|g(t)-g(n)|\le\int|g'|\mathbf{1}_{[n,\infty)}\,d\lambda\to0$ (dominated convergence again), so $g(t)\to c_+$ as $t\to\infty$. If $c_+\ne0$, there is $R_0>0$ with $|g(t)|\ge|c_+|/2$ for all $t\ge R_0$; then for every natural $n>R_0$, monotonicity, the integral of simple functions, and Existence of Lebesgue Measure on the Real Line give $\int|g|\,d\lambda\ge(|c_+|/2)\,\lambda([R_0,n])=(|c_+|/2)(n-R_0)\to\infty$, contradicting integrability of $g$; so $c_+=0$. Symmetrically, $g(-n)=g(0)-\int g'\mathbf{1}_{[-n,0]}\,d\lambda$ converges to a limit $c_-$, $g(t)\to c_-$ as $t\to-\infty$, and $c_-=0$. Finally, by dominated convergence and ($*$),

Proof of (ii). $h(t)=t\,g(t)$ is differentiable at every point with $h'(t)=g(t)+t\,g'(t)$ by claim 3 (the product rule) of Sum, Constant Multiple, and Product Rules for One-Dimensional Derivatives (the map $t\mapsto t$ has derivative $1$ directly from the definition of the derivative); $h$ and $h'$ are continuous, by claims 2, 3 and 5 of the continuity of sums and products of real-valued functions on a metric space together with the continuity of the identity map $t\mapsto t$ (for which $\delta=\varepsilon$ works); $h$ is integrable by assumption, and $h'$ is integrable as a sum of integrable functions (Linearity and Monotonicity of the Lebesgue Integral). Part (i) applied to $h$ gives $\int(g(t)+t\,g'(t))\,d\lambda(t)=0$, and linearity gives (ii).

Step 3 (slices of the density). Fix $i\in\{1,\dots,l\}$. For $(\theta',y)\in\mathbb{R}^{l-1}\times Y$ and $t\in\mathbb{R}$ put $g_{\theta',y}(t)=p\bigl(\Psi_i(t,(\theta',y))\bigr)$, and write $\Psi_i(t,(\theta',y))=(\theta(t),y)$, so that $\theta(t)\in\mathbb{R}^{l}$ has $i$th coordinate $t$ and all other coordinates independent of $t$, and $g_{\theta',y}(t)=p(\theta(t),y)$. Since $\theta(t)$ and $\theta(t_0)$ differ only in the $i$th coordinate,

By assumption (ii), $g_{\theta',y}$ is strictly positive.

Differentiability. Fix $t_0\in\mathbb{R}$. By assumption (ii) the function $p(\cdot,y)$ is of class $C^1$ on $\mathbb{R}^{l}$, so by clauses 1 and 3 of the definition of a $C^k$ map its partial derivative with respect to the $i$th variable exists at $\theta(t_0)$. Unwinding that definition: for every real $\varepsilon>0$ there is a real $\delta>0$ such that every real $h$ with $0<|h|<\delta$ satisfies

because the point obtained from $\theta(t_0)$ by increasing its $i$th coordinate by $h$ is exactly $\theta(t_0+h)$. The further requirement in that definition, that the incremented point belong to the domain, is automatic here because the domain is all of $\mathbb{R}^{l}$; what remains is verbatim the assertion that $g_{\theta',y}$ is differentiable at $t_0$ with

the value being unambiguous by Uniqueness of the Partial Derivative on a Euclidean Open Set.

Continuity. By clause 1 of the same definition, $p(\cdot,y)$ and $\partial_ip(\cdot,y)$ are continuous at every point of $\mathbb{R}^{l}$. Let $u$ be either of them, let $t_0\in\mathbb{R}$ and let $\varepsilon>0$ be real; continuity of $u$ at $\theta(t_0)$ supplies a real $\delta>0$ such that every $w\in\mathbb{R}^{l}$ with $\sum_k(w_k-\theta(t_0)_k)^2<\delta^2$ satisfies $\bigl(u(w)-u(\theta(t_0))\bigr)^2<\varepsilon^2$. Taking $w=\theta(t)$ and using the displayed identity, $(t-t_0)^2<\delta^2$ implies $\bigl(u(\theta(t))-u(\theta(t_0))\bigr)^2<\varepsilon^2$; by clause 2 of monotonicity of squaring, applied to the nonnegative reals $|t-t_0|,\delta$ and $|u(\theta(t))-u(\theta(t_0))|,\varepsilon$, this says that $|t-t_0|<\delta$ implies $|u(\theta(t))-u(\theta(t_0))|<\varepsilon$. Hence $g_{\theta',y}$ and $g_{\theta',y}'$ are continuous at every point of $\mathbb{R}$, as maps from the real line with the absolute value metric into itself.

Step 4 (the score identities). Fix $i,j\in\{1,\dots,l\}$ and let $\delta_{ij}=1$ if $i=j$ and $\delta_{ij}=0$ otherwise. We show

We use three elementary facts on a measure space, from Lebesgue Integral of a Nonnegative Measurable Function, Simple Function and Its Integral, and Linearity and Monotonicity of the Lebesgue Integral: (F1) a $[0,\infty]$-valued measurable $u$ with finite integral is finite off a set of measure zero (on $N=\{u=\infty\}$ one has $u\ge L\mathbf{1}_N$ for every $L$, so $L\,m(N)\le\int u\,dm$ for every $L$); (F2) a $[0,\infty]$-valued measurable $u$ vanishing off a set of measure zero has $\int u\,dm=0$ (every simple $s$ with $0\le s\le u$ is bounded by a multiple of the indicator of that set, so $\int s\,dm=0$; take the supremum); (F3) consequently, integrable real functions agreeing off a set of measure zero have equal integrals, and likewise $[0,\infty]$-valued measurable functions (split by the exceptional set and use additivity with (F2)).

(a) $\mathbb{E}[m_j(D)S_i]=0$. The function $G(\theta,y)=m_j(y)\,\partial_ip(\theta,y)/p(\theta,y)$ is $\mathcal{B}_l\otimes\mathcal{G}$-measurable: $(\theta,y)\mapsto m_j(y)$ is measurable (the preimage of a Borel set $A$ is the measurable rectangle $\mathbb{R}^{l}\times m_j^{-1}(A)$), $\partial_ip/p$ is measurable as in assumption (iv), and products of real-valued measurable functions are measurable by Sequentially Continuous Functions of Measurable Euclidean Maps are Measurable (the map $(u,v)\mapsto uv$ is continuous on $\mathbb{R}^{2}$). The random variable $G(\Theta,D)=m_j(D)S_i$ is integrable, being a product of the square-integrable random variables $m_j(D)$ and $S_i$ (Square-Integrable Random Variables and the Mean-Square Inner Product). By Step 1, $Gp$ is $\kappa$-integrable and

the second equality holding pointwise because $p>0$ everywhere (assumption (ii)). Apply claim 4 of Finite Products of Lebesgue Measure and Coordinate Integration on $\mathbb{R}^l$ (coordinate Fubini at coordinate $i$) to the $\kappa$-integrable $(\theta,y)\mapsto m_j(y)\partial_ip(\theta,y)$: there is a set $N_1$ of measure zero off which $t\mapsto m_j(y)\,g_{\theta',y}'(t)$ is $\lambda$-integrable (Step 3 identifies the integrand), and $\int m_j\partial_ip\,d\kappa$ equals the integral of the function $F$ given off $N_1$ by $F(\theta',y)=\int_{\mathbb{R}}m_j(y)g_{\theta',y}'(t)\,d\lambda(t)$ and by $0$ on $N_1$. Now apply claim 3 of Finite Products of Lebesgue Measure and Coordinate Integration on $\mathbb{R}^l$ (coordinate Tonelli) to the nonnegative measurable functions $p$ and $|\partial_ip|$: since $\int p\,d\kappa=1<\infty$ (Step 1) and $\int|\partial_ip|\,d\kappa<\infty$ (assumption (iii)), the measurable $[0,\infty]$-valued functions $(\theta',y)\mapsto\int_{\mathbb{R}}g_{\theta',y}\,d\lambda$ and $(\theta',y)\mapsto\int_{\mathbb{R}}|g_{\theta',y}'|\,d\lambda$ have finite integrals, hence by (F1) are finite off sets $N_2$, $N_3$ of measure zero. For $(\theta',y)\notin N_1\cup N_2\cup N_3$, the function $g_{\theta',y}$ satisfies all hypotheses of Step 2(i) (continuity and differentiability from Step 3, integrability off $N_2\cup N_3$), so $\int_{\mathbb{R}}g_{\theta',y}'\,d\lambda=0$ and, pulling out the constant $m_j(y)$ by linearity (licensed off $N_3$, where $g_{\theta',y}'$ is $\lambda$-integrable), $F(\theta',y)=0$. Thus $F$ vanishes off a set of measure zero, and $\mathbb{E}[m_j(D)S_i]=\int F\,d(\lambda_{l-1}\otimes\mu)=0$ by (F3).

(b) $\mathbb{E}[\Theta_jS_i]=-\delta_{ij}$. Take $G(\theta,y)=\theta_j\,\partial_ip(\theta,y)/p(\theta,y)$; the coordinate map $(\theta,y)\mapsto\theta_j$ is measurable (preimages are measurable rectangles), $G(\Theta,D)=\Theta_jS_i$ is integrable as a product of square-integrable random variables, and Step 1 gives

the integrand being $\kappa$-integrable also directly from assumption (iii), since $|\theta_j\partial_ip|\le(1+\sum_{j'}|\theta_{j'}|)|\partial_ip|$ pointwise.

Case $i\ne j$. The $j$th coordinate of $\Psi_i(t,(\theta',y))$ does not depend on $t$: it equals $\theta'_j$ if $j<i$ and $\theta'_{j-1}$ if $j>i$; call it $\theta'_{(j)}$. Coordinate Fubini applied to $\theta_j\partial_ip$ represents the integral through inner integrals $\int_{\mathbb{R}}\theta'_{(j)}\,g_{\theta',y}'(t)\,d\lambda(t)=\theta'_{(j)}\int_{\mathbb{R}}g_{\theta',y}'\,d\lambda=0$ off a set of measure zero, exactly as in (a) with the constant $\theta'_{(j)}$ in place of $m_j(y)$. Hence $\mathbb{E}[\Theta_jS_i]=0$.

Case $i=j$. Note that the $i$th coordinate of $\Psi_i(t,(\theta',y))$ is exactly $t$. Coordinate Fubini applied to the $\kappa$-integrable $\theta_i\partial_ip$ represents $\mathbb{E}[\Theta_iS_i]$ as the integral of the function $F$ equal, off a set $N_1'$ of measure zero, to $F(\theta',y)=\int_{\mathbb{R}}t\,g_{\theta',y}'(t)\,d\lambda(t)$ and to $0$ on $N_1'$. Two further applications of coordinate Tonelli give sets of measure zero off which $\int_{\mathbb{R}}|t|\,g_{\theta',y}(t)\,d\lambda(t)<\infty$ and $\int_{\mathbb{R}}|t|\,|g_{\theta',y}'(t)|\,d\lambda(t)<\infty$: the first because $\int|\theta_i|\,p\,d\kappa=\mathbb{E}[|\Theta_i|]<\infty$ by Step 1 (square-integrable random variables are integrable, Square-Integrable Random Variables and the Mean-Square Inner Product), the second from assumption (iii) with the weight $|\theta_i|$, both followed by (F1). Off the union of all these sets and $N_2$, $N_3$ of part (a), Step 2(ii) applies to $g_{\theta',y}$ and gives

Let $G_0(\theta',y)=\int_{\mathbb{R}}g_{\theta',y}\,d\lambda$, the $[0,\infty]$-valued measurable function of coordinate Tonelli applied to $p$, with $\int G_0\,d(\lambda_{l-1}\otimes\mu)=\int p\,d\kappa=1$. Then $F$ is integrable (claim 4), $F^{+}$ vanishes off a set of measure zero, and $F^{-}=G_0$ off a set of measure zero; so by (F2) and (F3), $\int F\,d(\lambda_{l-1}\otimes\mu)=-\int G_0\,d(\lambda_{l-1}\otimes\mu)=-1$. Hence $\mathbb{E}[\Theta_iS_i]=-1$.

Combining (a) and (b) with linearity of the expectation (Linearity and Monotonicity of the Lebesgue Integral),

Step 5 (Cauchy-Schwarz step). Each $m_j(D)-\Theta_j$ is square-integrable (Square-Integrable Random Variables and the Mean-Square Inner Product), so every product $(m_i(D)-\Theta_i)(m_j(D)-\Theta_j)$ is integrable, $R$ is well defined, and $R_{ij}=R_{ji}$ by commutativity of pointwise multiplication. $J$ is symmetric positive definite by assumption (iv), so $J^{-1}$ exists and is symmetric positive definite by Invertibility of Symmetric Positive Definite Matrices. We use the dot product and the matrix-vector product on $\mathbb{R}^{l}$; componentwise, $a\cdot(Ma)=\sum_{i,j}a_iM_{ij}a_j$ for any $l\times l$ matrix $M$; $A(Bx)=(AB)x$ for $l\times l$ matrices, since $(A(Bx))_i=\sum_jA_{ij}\sum_mB_{jm}x_m=\sum_m(AB)_{im}x_m$ with the matrix product; and $Ix=x$ for the identity matrix $I$ of Inverse Matrix and Invertible Real Square Matrix, since $(Ix)_i=\sum_jI_{ij}x_j=x_i$.

Fix $a\in\mathbb{R}^{l}$ and set $b=J^{-1}a$, and

both square-integrable as linear combinations of square-integrable random variables. Expanding the products and using linearity of the expectation:

by ($**$); and

since $Jb=J(J^{-1}a)=(JJ^{-1})a=Ia=a$ by the identities above and the definition of the inverse, and the dot product is symmetric. Since $(X-W)^{2}\ge0$ pointwise, monotonicity and linearity of the expectation give

the last step by componentwise bilinearity. As $a$ was arbitrary and $R-J^{-1}$ is symmetric, $R-J^{-1}$ is positive semidefinite; that is, $R\succeq J^{-1}$ in the semidefinite order. $\blacksquare$