Proof of The Diagonal Gaussian Density on Euclidean Space: Regularity, Gradient, Normalization, Second Moments and Exponential Moments of Diagonal Quadratic Forms

Each result cited below is universally quantified over the data in its own statement.

Throughout, $\lambda$ denotes Lebesgue measure on the Borel sets of $\mathbb{R}$; by Lebesgue Measure on $\mathbb{R}^n$ (its case $n=1$) it is $\lambda_{1}$, and $\mathbb{R}$ is read as $\mathbb{R}^{1}$ as in One-Dimensional Derivatives, Partial Derivatives, and Smoothness on the Real Line. Points of $\mathbb{R}^{m}$ are tuples, as in Probability Measures on Euclidean Space and Random Vectors: Standing Notation §spaces. A nonnegative Borel function into $\mathbb{R}$ is integrated as a $[0,\infty]$-valued map, and a Borel $f:\mathbb{R}^{m}\to\mathbb{R}$ is integrable exactly when $\int|f|\,d\lambda_{m}<\infty$, by Measure Spaces and the Lebesgue Integral: Standing Notation §measurable and Measure Spaces and the Lebesgue Integral: Standing Notation §integral. By The Diagonal Gaussian Density on Euclidean Space and Its Notation §scaling and The Diagonal Gaussian Density on Euclidean Space and Its Notation §density, $\rho_{c}(x)=\exp(q_{c}(x))$ with $q_{c}(x)=-Z_{c}-\sum_{i=1}^{d}x_{i}^{2}/(2c_{i})$, and $\kappa$ is a positive real number with $\int_{\mathbb{R}}\psi\,d\lambda=\kappa$, where $\psi(s)=\exp(-s^{2}/2)$, by claim 2 of The Gaussian Weight Defines a Probability Distribution. For a positive real $v$ write $e_{v}(s)=\exp(-s^{2}/(2v))$ for $s\in\mathbb{R}$; thus $\psi=e_{1}$.

Step 1 (Gaussian-type functions are smooth, positive and Borel). Let $m\ge1$, let $w=(w_{1},\dots,w_{m})$ have positive entries and let $b\in\mathbb{R}$. Put $q_{w,b}(y)=b+\sum_{l=1}^{m}\bigl(-(2w_{l})^{-1}\bigr)\pi_{l}(y)\pi_{l}(y)$ for $y\in\mathbb{R}^{m}$, where $\pi_{l}(y)=y_{l}$, and $E_{w,b}=\exp\circ q_{w,b}$. The sets $\mathbb{R}^{m}$ and $\mathbb{R}^{1}$ are open by claim 1 of Euclidean Space is Open in Itself, and $C^k$ Maps are Continuous (with $n=m$ and with $n=1$). Constants and the coordinate functions $\pi_{l}$ are smooth on $\mathbb{R}^{m}$ by claim 2 of Constants, Coordinate Functions, Sums and Products of $C^k$ Functions on a Euclidean Open Set, so $q_{w,b}$ is smooth by claim 3 there (applied finitely often to the products, scalar multiples and sums). By claim 3 of Basic Properties of the Exponential Function, $\exp$ is differentiable at every point with $\exp'=\exp$; hence the one-element set $D=\{\exp\}$ satisfies the hypothesis of claim 3 of One-Dimensional Derivatives, Partial Derivatives, and Smoothness on the Real Line, and $\exp$ is smooth on $\mathbb{R}^{1}$ in the sense of Smooth Map on a Euclidean Open Set. By claim 3 of A Composition of $C^k$ Maps Between Euclidean Open Sets is of Class $C^k$ (with $U=\mathbb{R}^{m}$, $V=\mathbb{R}^{1}$, $F=q_{w,b}$, $G=\exp$), $E_{w,b}$ is smooth on $\mathbb{R}^{m}$. It is positive by claim 2 of Basic Properties of the Exponential Function. By claim 3 of Euclidean Space is Open in Itself, and $C^k$ Maps are Continuous it is continuous from $(\mathbb{R}^{m},d_{E})$ into $(\mathbb{R},d_{\mathbb{R}})$, hence Borel by Probability Measures on Euclidean Space and Random Vectors: Standing Notation §borel-maps.

For the partial derivatives, fix $y$ and $i\in[m]$. For every real $h\ne0$ the difference quotient of Partial Derivative on a Euclidean Open Set for $\pi_{l}$ at $y$ in the $i$th variable equals $1$ if $l=i$ and $0$ if $l\ne i$, and that of a constant equals $0$; so $\partial_{i}\pi_{l}(y)$ is $1$ or $0$ accordingly and constants have partial derivative $0$. By claim 1 of Constants, Coordinate Functions, Sums and Products of $C^k$ Functions on a Euclidean Open Set (product, scalar-multiple and sum rules) $\partial_{i}q_{w,b}(y)=-(2w_{i})^{-1}\cdot2y_{i}=-y_{i}/w_{i}$. Since $q_{w,b}$ and $\exp$ are smooth, they are of class $C^{1}$ by Smooth Map on a Euclidean Open Set, so claim 1 of A Composition of $C^k$ Maps Between Euclidean Open Sets is of Class $C^k$ gives $\partial_{i}E_{w,b}(y)=\partial_{1}\exp(q_{w,b}(y))\,\partial_{i}q_{w,b}(y)$; and $\partial_{1}\exp(u)=\exp'(u)=\exp(u)$ by claim 2 of One-Dimensional Derivatives, Partial Derivatives, and Smoothness on the Real Line and claim 3 of Basic Properties of the Exponential Function. Hence

$$\partial_{i}E_{w,b}(y)=-\frac{y_{i}}{w_{i}}\,E_{w,b}(y)\qquad(y\in\mathbb{R}^{m},\ i\in[m]).$$

In particular ($m=1$, $w=(v)$, $b=0$) each $e_{v}$ is a positive Borel function on $\mathbb{R}$, and $s\mapsto s^{2}e_{v}(s)$ is Borel by claim 1 of The Borel Sigma-Algebra of a Euclidean Space as a Product, and Measurability of Projections, Sequentially Continuous Maps, and Open and Closed Sets (the coordinate $s\mapsto s$ is Borel, the $\sigma$-algebra there being $\mathcal{B}(\mathbb{R})$ by claim 5) and claim 3 of Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions.

Step 2 (Claims 1 and 2). With $m=d$, $w=c$ (a variance vector, so $c_{i}>0$ by The Diagonal Gaussian Density on Euclidean Space and Its Notation §variances) and $b=-Z_{c}$ we have $q_{c}=q_{c,-Z_{c}}$ and $\rho_{c}=E_{c,-Z_{c}}$. Step 1 shows that $\rho_{c}$ is smooth, positive and Borel, which is claim 1. Being smooth, $\rho_{c}$ is of class $C^{1}$, and by Differential Calculus and Convexity on Euclidean Open Sets: Standing Notation §derivatives and Gradient of a Real-Valued Function on a Euclidean Open Set its gradient is $D\rho_{c}(x)=(\partial_{1}\rho_{c}(x),\dots,\partial_{d}\rho_{c}(x))$. By Step 1 the $i$th entry is $-\rho_{c}(x)\,x_{i}/c_{i}$, which is the $i$th entry of the scalar multiple $-\rho_{c}(x)S_{c}(x)$ of $S_{c}(x)=(x_{1}/c_{1},\dots,x_{d}/c_{d})$ (The Diagonal Gaussian Density on Euclidean Space and Its Notation §scaling). This is claim 2.

Step 3 (Factorization). Let $w\in\mathbb{R}^{m}$ have positive entries and $b\in\mathbb{R}$. Since $q_{w,b}(y)=b+\sum_{i=1}^{m}\bigl(-y_{i}^{2}/(2w_{i})\bigr)$, claim 1 of Basic Properties of the Exponential Function ($\exp(u+u')=\exp(u)\exp(u')$, applied by induction on the number of summands) gives

$$E_{w,b}(y)=\exp(b)\prod_{i=1}^{m}e_{w_{i}}(y_{i}).$$

Moreover, $\kappa\sqrt{c_{i}}>0$, so by claim 1 and claim 2 of Basic Properties of the Exponential Function and the identity $\exp(\log t)=t$ of The Natural Logarithm,

$$\exp(-Z_{c})=\prod_{i=1}^{d}\exp\bigl(-\log(\kappa\sqrt{c_{i}})\bigr)=\prod_{i=1}^{d}\frac{1}{\kappa\sqrt{c_{i}}} .$$

Hence, for every $x\in\mathbb{R}^{d}$,

$$\rho_{c}(x)=\prod_{i=1}^{d}\frac{e_{c_{i}}(x_{i})}{\kappa\sqrt{c_{i}}}.\tag{F1}$$

Now let $t$ be as in claim 5 and put $u_{i}=1-2t_{i}c_{i}>0$ and $v_{i}=c_{i}/u_{i}>0$. Since $t_{i}-1/(2c_{i})=-u_{i}/(2c_{i})=-1/(2v_{i})$, we have $\sum_{i}t_{i}x_{i}^{2}+q_{c}(x)=q_{v,-Z_{c}}(x)$ with $v=(v_{1},\dots,v_{d})$, so by claim 1 of Basic Properties of the Exponential Function and the two displays above,

$$\exp\Bigl(\sum_{i=1}^{d}t_{i}x_{i}^{2}\Bigr)\rho_{c}(x)=E_{v,-Z_{c}}(x)=\prod_{i=1}^{d}\frac{e_{v_{i}}(x_{i})}{\kappa\sqrt{c_{i}}}.\tag{F2}$$

Step 4 (One-dimensional integrals). Let $v>0$. We show

$$\int_{\mathbb{R}}e_{v}\,d\lambda=\kappa\sqrt{v},\qquad\int_{\mathbb{R}}s^{2}e_{v}(s)\,\lambda(ds)=v\,\kappa\sqrt{v}.\tag{F3}$$

First, $\int_{\mathbb{R}}s^{2}\psi(s)\,\lambda(ds)=\kappa$. Indeed, apply The Gaussian Kernels on Euclidean Space: Scaling, Derivatives up to Order Three, the Convolution Identity, Moments and Tails §moments in dimension $q=1$ with $s=1$: for $z\in\mathbb{R}^{1}$ we have $\lVert z\rVert^{2}=z^{2}$ by Elementary Properties of the Euclidean Norm on $\mathbb{R}^n$ §square, so the kernel there is $g_{1}=k_{1}\psi$ with $k_{1}$ (the normalizing constant written $c_{1}$ in that lemma, renamed here to avoid a clash with the variance $c_{1}$) the unique positive real making $\int g_{1}\,d\lambda_{1}=1$; since $\int\kappa^{-1}\psi\,d\lambda=\kappa^{-1}\kappa=1$ by Linearity and Monotonicity of the Lebesgue Integral §nonnegative, uniqueness gives $k_{1}=\kappa^{-1}$. That clause gives $\int z^{2}g_{1}(z)\,dz=1\cdot1=1$ (the nonnegative function $z\mapsto z^{2}g_{1}(z)$ being integrated as a $[0,\infty]$-valued map, as fixed in the preamble of The Gaussian Kernels on Euclidean Space: Scaling, Derivatives up to Order Three, the Convolution Identity, Moments and Tails), and then $\int s^{2}\psi(s)\,\lambda(ds)=\int\kappa\,s^{2}g_{1}(s)\,\lambda(ds)=\kappa$ by Linearity and Monotonicity of the Lebesgue Integral §nonnegative.

Next put $r=(\sqrt{v})^{-1}$, a nonzero real with $|r|^{-1}=\sqrt{v}$ and $r^{2}=1/v$. Apply claim 2 of Scaling of Lebesgue Measure and the Lebesgue Integral on $\mathbb{R}^n$ with $n=1$ and the scalar written $c$ there equal to $r$, first to the nonnegative Borel function $\psi$ (Step 1) and then to $s\mapsto s^{2}\psi(s)$ (Step 1). Since $\psi(rs)=\exp(-s^{2}/(2v))=e_{v}(s)$ and $(rs)^{2}\psi(rs)=v^{-1}s^{2}e_{v}(s)$, we obtain $\int e_{v}\,d\lambda=\sqrt{v}\,\kappa$ and $\int v^{-1}s^{2}e_{v}(s)\,\lambda(ds)=\sqrt{v}\,\kappa$. Multiplying the latter integrand by the constant $v$ and using Linearity and Monotonicity of the Lebesgue Integral §nonnegative gives the second identity in (F3).

Step 5 (Integrals of products of one-variable functions). Let $m\ge1$ and let $h_{1},\dots,h_{m}:\mathbb{R}\to\mathbb{R}$ be nonnegative Borel functions with $a_{i}=\int_{\mathbb{R}}h_{i}\,d\lambda<\infty$. Put $H_{m}(y)=\prod_{i=1}^{m}h_{i}(y_{i})$ for $y\in\mathbb{R}^{m}$. We show that $H_{m}$ is Borel and $\int_{\mathbb{R}^{m}}H_{m}\,d\lambda_{m}=\prod_{i=1}^{m}a_{i}$. Each coordinate map $y\mapsto y_{i}$ is measurable for $\mathcal{B}_{m}=\mathcal{B}(\mathbb{R}^{m})$ by claims 1 and 5 of The Borel Sigma-Algebra of a Euclidean Space as a Product, and Measurability of Projections, Sequentially Continuous Maps, and Open and Closed Sets, so $y\mapsto h_{i}(y_{i})$ is Borel as a composition of Borel maps (Probability Measures on Euclidean Space and Random Vectors: Standing Notation §borel-maps) and $H_{m}$ is Borel by claim 3 of Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions.

We prove the integral formula by induction on $m$. For $m=1$, $H_{1}=h_{1}$ and $\lambda_{1}=\lambda$. Let $m\ge2$ and assume the formula in dimension $m-1$. Identify $\mathbb{R}^{m}$ with $\mathbb{R}^{m-1}\times\mathbb{R}$ via $(y_{1},\dots,y_{m})\leftrightarrow((y_{1},\dots,y_{m-1}),y_{m})$ as in Finite Products of Lebesgue Measure and Coordinate Integration on $\mathbb{R}^l$. Under this identification $\mathcal{B}_{m}=\mathcal{B}_{m-1}\otimes\mathcal{B}(\mathbb{R})$ and the measure constructed there is $\lambda_{m-1}\otimes\lambda$, the product of Existence and Uniqueness of the Product Measure; by Lebesgue Measure on $\mathbb{R}^n$ (with claim 1 of Finite Products of Lebesgue Measure and Coordinate Integration on $\mathbb{R}^l$ and claim 5 of The Borel Sigma-Algebra of a Euclidean Space as a Product, and Measurability of Projections, Sequentially Continuous Maps, and Open and Closed Sets) that measure is Lebesgue measure $\lambda_{m}$ on $\mathcal{B}(\mathbb{R}^{m})$, and likewise the factor on $\mathbb{R}^{m-1}$ is $\lambda_{m-1}$. Both factors are $\sigma$-finite by claim 1 of Finite Products of Lebesgue Measure and Coordinate Integration on $\mathbb{R}^l$ (with $l=m-1$ and $l=1$). Write $y=(y',y_{m})$; then $H_{m}(y',y_{m})=H_{m-1}(y')\,h_{m}(y_{m})$, where $H_{m-1}$ is built from $h_{1},\dots,h_{m-1}$ and is Borel on $\mathbb{R}^{m-1}$ by the first paragraph. Since $H_{m}$ is a nonnegative $\mathcal{B}_{m-1}\otimes\mathcal{B}(\mathbb{R})$-measurable function, the Tonelli part of Tonelli and Fubini Theorems (with $X=\mathbb{R}^{m-1}$, $\mu=\lambda_{m-1}$, $Y=\mathbb{R}$, $\nu=\lambda$) gives

$$\int_{\mathbb{R}^{m}}H_{m}\,d\lambda_{m}=\int_{\mathbb{R}^{m-1}}\Bigl(\int_{\mathbb{R}}H_{m-1}(y')\,h_{m}(s)\,\lambda(ds)\Bigr)\lambda_{m-1}(dy').$$

For fixed $y'$ the inner integral equals $H_{m-1}(y')\,a_{m}$ by Linearity and Monotonicity of the Lebesgue Integral §nonnegative (constant factor $H_{m-1}(y')\in[0,\infty)$), and by the same clause (constant factor $a_{m}\in[0,\infty)$) and the induction hypothesis the outer integral equals $a_{m}\prod_{i=1}^{m-1}a_{i}=\prod_{i=1}^{m}a_{i}$.

Step 6 (Claim 3). By (F1), $\rho_{c}=H_{d}$ with $h_{i}=(\kappa\sqrt{c_{i}})^{-1}e_{c_{i}}$, which is nonnegative and Borel by Step 1 and claim 2 of Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions, with $a_{i}=(\kappa\sqrt{c_{i}})^{-1}\kappa\sqrt{c_{i}}=1$ by (F3) and Linearity and Monotonicity of the Lebesgue Integral §nonnegative. Step 5 gives $\int_{\mathbb{R}^{d}}\rho_{c}\,d\lambda_{d}=1$.

Step 7 (Claim 4). Fix $j\in[d]$. By (F1), $x_{j}^{2}\rho_{c}(x)=H_{d}(x)$ with $h_{i}$ as in Step 6 for $i\ne j$ and $h_{j}(s)=(\kappa\sqrt{c_{j}})^{-1}s^{2}e_{c_{j}}(s)$, which is nonnegative and Borel by Step 1 and claim 2 of Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions. By (F3) and Linearity and Monotonicity of the Lebesgue Integral §nonnegative, $a_{j}=(\kappa\sqrt{c_{j}})^{-1}c_{j}\kappa\sqrt{c_{j}}=c_{j}$, and $a_{i}=1$ for $i\ne j$. Step 5 shows that $x\mapsto x_{j}^{2}\rho_{c}(x)$ is a nonnegative Borel function with integral $c_{j}<\infty$; hence it is integrable (Measure Spaces and the Lebesgue Integral: Standing Notation §integral) with integral $c_{j}$. Here the integral of the integrable function is its $[0,\infty]$-valued integral: for a nonnegative Borel $f$ with $\int f\,d\lambda_{d}<\infty$ we have $f^{+}=f$ and $f^{-}=0=0\cdot f$, so $\int f^{-}\,d\lambda_{d}=0\cdot\int f\,d\lambda_{d}=0$ by Linearity and Monotonicity of the Lebesgue Integral §nonnegative, and Integrable Function and the Lebesgue Integral shows that $f$ is integrable with integral $\int f^{+}\,d\lambda_{d}-0=\int f\,d\lambda_{d}$. Finally $\lVert x\rVert^{2}=\sum_{j=1}^{d}x_{j}^{2}$ by Elementary Properties of the Euclidean Norm on $\mathbb{R}^n$ §square, so $\lVert x\rVert^{2}\rho_{c}(x)=\sum_{j=1}^{d}x_{j}^{2}\rho_{c}(x)$, and Linearity and Monotonicity of the Lebesgue Integral §integrable, applied by induction on the number of summands, shows that this function is integrable with $\int_{\mathbb{R}^{d}}\lVert x\rVert^{2}\rho_{c}(x)\,\lambda_{d}(dx)=\sum_{j=1}^{d}c_{j}$.

Step 8 (Claim 5). With $u_{i}$ and $v_{i}$ as in Step 3, (F2) exhibits $x\mapsto\exp(\sum_{i}t_{i}x_{i}^{2})\rho_{c}(x)$ as $H_{d}$ with $h_{i}=(\kappa\sqrt{c_{i}})^{-1}e_{v_{i}}$, nonnegative and Borel as in Step 6. By (F3) and Linearity and Monotonicity of the Lebesgue Integral §nonnegative, $a_{i}=\sqrt{v_{i}}/\sqrt{c_{i}}=1/\sqrt{u_{i}}$, because $\sqrt{v_{i}}\sqrt{u_{i}}=\sqrt{v_{i}u_{i}}=\sqrt{c_{i}}$. Step 5 shows that the function is nonnegative and Borel with integral $\prod_{i=1}^{d}u_{i}^{-1/2}$, where $u_{i}^{-1/2}=1/\sqrt{u_{i}}$; being finite, it is integrable (Measure Spaces and the Lebesgue Integral: Standing Notation §integral), and its integral as an integrable function is this same number, as observed in Step 7. It remains to identify the value. For $u>0$ the number $p=\exp(\tfrac12\log u)$ is positive and satisfies $p^{2}=\exp(\log u)=u$ by claims 1 and 2 of Basic Properties of the Exponential Function and The Natural Logarithm, so $p=\sqrt{u}$ by uniqueness of the nonnegative square root, and $\exp(-\tfrac12\log u)=1/\sqrt{u}$ by claim 2 of Basic Properties of the Exponential Function. Hence, by claim 1 of Basic Properties of the Exponential Function applied by induction on the number of summands,

$$\exp\Bigl(-\tfrac12\sum_{i=1}^{d}\log(1-2t_{i}c_{i})\Bigr)=\prod_{i=1}^{d}\exp\bigl(-\tfrac12\log u_{i}\bigr)=\prod_{i=1}^{d}\frac{1}{\sqrt{u_{i}}},$$

which is the integral just computed. This proves claim 5.