Proof of The Gaussian Smoothing Weight: Normalization, Derivatives, Exponential Tilting, Moments, and First-Order Remainder

Throughout, $z=(z_1,\dots,z_m)$, and $\lVert z\rVert^{2}=z\cdot z=\sum_{i=1}^{m}z_i^{2}$ by the definitions of the Euclidean norm and the dot product; the dot product is symmetric and linear in each argument, being a finite sum of products of coordinates, so $\lVert z-a\rVert^{2}=\lVert z\rVert^{2}-2\,a\cdot z+\lVert a\rVert^{2}$ for all $z,a\in\mathbb{R}^m$. We use claims 1, 2 and 4 of Basic Properties of the Exponential Function: $\exp(u+v)=\exp(u)\exp(v)$, $\exp(0)=1$, $\exp>0$, $\exp$ is strictly increasing, and $\exp(u)\ge1+u$ for $u\ge0$. Limits of sums and products of convergent real sequences are handled by Arithmetic of Limits of Real Sequences, and inequalities are passed to the limit by claim 1 (comparison) and claim 4 (absolute values) of Order Properties of Limits of Real Sequences; finite sums are compared termwise and bounded by the sum of absolute values via Comparison and Absolute Value Bounds for Finite Sums of Real Numbers. Some elementary consequences of the series definition of The Real Exponential Function are used repeatedly; they are proved for the partial sums $E_K(u)=\sum_{k=0}^{K}u^{k}/k!$ ($K\ge0$, with $E_{-1}=E_{-2}=0$), which converge to $\exp(u)$ as $K\to\infty$ by that definition, and then passed to the limit. A subsequence of a convergent sequence converges to the same limit (an index bound for the sequence serves for the subsequence), and prepending or dropping finitely many terms does not affect convergence or the limit.

(E1) For every real $h$, $|\exp(h)-1|\le|h|\exp(|h|)$: indeed, since $k!\ge(k-1)!$ and all terms are nonnegative, $|E_K(h)-1|\le\sum_{k=1}^{K}|h|^{k}/k!=|h|\sum_{k=1}^{K}|h|^{k-1}/k!\le|h|\sum_{k=1}^{K}|h|^{k-1}/(k-1)!=|h|E_{K-1}(|h|)\le|h|E_K(|h|)$, and letting $K\to\infty$ gives the claim. Consequently $\exp$ is sequentially continuous on $\mathbb{R}$: if $s_k\to s$, choose $K_0$ with $|s_k-s|\le1$ for all $k\ge K_0$; for such $k$, $|\exp(s_k)-\exp(s)|=\exp(s)\,|\exp(s_k-s)-1|\le\exp(s+1)\,|s_k-s|$, which tends to $0$, so $\exp(s_k)\to\exp(s)$ by claim 3 of Order Properties of Limits of Real Sequences. Likewise, for $s>0$ and $K\ge2$, $\frac{E_K(s)-1}{s}-1=\sum_{k=2}^{K}s^{k-1}/k!$ lies between $0$ and $s\sum_{k=2}^{K}s^{k-2}/(k-2)!=sE_{K-2}(s)\le sE_K(s)$ (as $k!\ge(k-2)!$), so in the limit $0\le\frac{\exp(s)-1}{s}-1\le s\exp(s)$.

(E2) For every real $u$ and $K\ge0$, since $(-u)^{k}=u^{k}$ for even $k$ and $(-u)^{k}=-u^{k}$ for odd $k$,

$$E_{2K+1}(u)+E_{2K+1}(-u)=2\sum_{j=0}^{K}\frac{u^{2j}}{(2j)!},$$

all terms on the right being nonnegative, so the right side is at least $2+u^{2}$ for $K\ge1$. Letting $K\to\infty$ along the subsequence $(E_{2K+1})_K$, $\exp(u)+\exp(-u)\ge2+u^{2}$. Hence $u^{2}\le\exp(u)+\exp(-u)-2$ and, as $2+u^{2}\ge2|u|$, also $|u|\le\tfrac12\bigl(\exp(u)+\exp(-u)\bigr)$. Moreover, for $u\ge0$, $\exp(u)\ge1+u\ge2\sqrt{u}$ because $(1-\sqrt u)^{2}\ge0$, where $\sqrt u$ is the nonnegative square root of Existence and Uniqueness of the Nonnegative Square Root.

A sequence $z^k\to z$ in $\mathbb{R}^m$ (that is, $d(z^k,z)\to0$) converges coordinatewise, since $|z^k_i-z_i|\le d(z^k,z)$ by the definition of the Euclidean distance; hence, by the limit laws, every polynomial in the coordinates is sequentially continuous, and by (E1) so is $\exp$ composed with such a polynomial. Products and sums of sequentially continuous functions are sequentially continuous, again by the limit laws, and so are quotients with a nowhere-vanishing denominator (claim 4 of Arithmetic of Limits of Real Sequences); in particular every function of the form (polynomial in the coordinates)$\,\times\,\exp$(polynomial in the coordinates), and every quotient of such functions by $\varphi_\eta$, is sequentially continuous. Every sequentially continuous $f:\mathbb{R}^m\to\mathbb{R}$ is measurable by claim 3(a) of The Borel Sigma-Algebra of a Euclidean Space as a Product, and Measurability of Projections, Sequentially Continuous Maps, and Open and Closed Sets together with claim 5 there ($\mathcal{B}_m=\mathcal{B}(\mathbb{R}^m)$). In particular $\psi_\eta$, $\varphi_\eta$, $Z_a$, and all the functions appearing in claims 2 to 5 are sequentially continuous and measurable; we shall not repeat this. Linearity and monotonicity of integrals refer to Linearity and Monotonicity of the Lebesgue Integral, claim 1 for $[0,\infty]$-valued and claim 2 for integrable functions; a measurable $f$ is integrable if and only if $\int|f|\,d\lambda_m<\infty$ by Integrable Function and the Lebesgue Integral, and then the integral of $|f|$ dominates that of any measurable function bounded by $|f|$ in absolute value. Translation and reflection of integrands refer to claims 2 and 3 of Translation and Reflection Invariance of Lebesgue Measure on $\mathbb{R}^n$; a nonnegative real-valued measurable function is measurable as a $[0,\infty]$-valued function (the criterion of Measurable Function and Real-Valued Measurable Function is unchanged), so claim 2 there applies to such functions.

Step 1: claim 1. Since $-\lVert z\rVert^{2}/(2\eta)\le0$ and $\exp$ is positive and increasing, $0<\psi_\eta(z)\le\exp(0)=1$.

Finiteness of the integral. Let $g(t)=\exp(-t^{2}/2)$ and $c=\int_{\mathbb{R}}g\,d\lambda\in(0,\infty)$ as in claim 2 of The Gaussian Weight Defines a Probability Distribution, where $\lambda$ is Lebesgue measure on the real line. For $l\ge1$ put $G_l(z)=\prod_{i=1}^{l}g(z_i)$ on $\mathbb{R}^{l}$; it is sequentially continuous, hence measurable with respect to $\mathcal{B}_l=\mathcal{B}(\mathbb{R}^l)$. We show $\int_{\mathbb{R}^l}G_l\,d\lambda_l=c^{l}$ by induction on $l$. For $l=1$, $G_1=g$. For $l\ge2$, by Lebesgue Measure on $\mathbb{R}^n$ and the recursive construction in Finite Products of Lebesgue Measure and Coordinate Integration on $\mathbb{R}^l$, $\lambda_l$ is the product measure $\lambda_{l-1}\otimes\lambda$ on $\mathcal{B}_{l-1}\otimes\mathcal{B}(\mathbb{R})=\mathcal{B}_l$ under the identification of $\mathbb{R}^{l}$ with $\mathbb{R}^{l-1}\times\mathbb{R}$, the factors being $\sigma$-finite by claim 1 of that lemma, and $G_l(z',t)=G_{l-1}(z')\,g(t)$. The Tonelli theorem of Tonelli and Fubini Theorems and linearity give

$$\int_{\mathbb{R}^l}G_l\,d\lambda_l=\int_{\mathbb{R}^{l-1}}\Bigl(G_{l-1}(z')\int_{\mathbb{R}}g\,d\lambda\Bigr)d\lambda_{l-1}(z')=c\int_{\mathbb{R}^{l-1}}G_{l-1}\,d\lambda_{l-1}=c^{l}.$$

Since $\eta\le1$, $-z_i^{2}/(2\eta)\le-z_i^{2}/2$ for each $i$, so by the functional equation and monotonicity of $\exp$, $\psi_\eta(z)=\prod_{i=1}^{m}\exp(-z_i^{2}/(2\eta))\le G_m(z)$ (a product of positive factors is at most the product of termwise larger positive factors, by induction on the number of factors), and by monotonicity of the integral $\int\psi_\eta\,d\lambda_m\le c^{m}<\infty$.

Positivity of the integral. Let $B$ be the open ball of centre $0$ and radius $1$ for $d$; by claim 1 of Balls Have Positive Lebesgue Measure and Bounded Sets Have Finite Lebesgue Measure it is a Borel set with $\lambda_m(B)>0$. For $z\in B$ we have $\lVert z\rVert<1$, hence $\psi_\eta(z)\ge\exp(-1/(2\eta))$. Thus $\psi_\eta\ge\exp(-1/(2\eta))\mathbf{1}_B$, a nonnegative simple function with integral $\exp(-1/(2\eta))\lambda_m(B)>0$, and monotonicity gives $\int\psi_\eta\,d\lambda_m>0$.

Normalization. Put $I_\eta=\int\psi_\eta\,d\lambda_m\in(0,\infty)$. For a real $c>0$, linearity gives $\int c\psi_\eta\,d\lambda_m=cI_\eta$, which equals $1$ if and only if $c=1/I_\eta$. So $c_\eta=1/I_\eta$ is the unique positive normalizer, and $\varphi_\eta=c_\eta\psi_\eta$ satisfies $0<\varphi_\eta\le c_\eta$ and $\int\varphi_\eta\,d\lambda_m=1$. Since $\lVert-z\rVert=\lVert z\rVert$, $\varphi_\eta(-z)=\varphi_\eta(z)$. Finally, for $a\in\mathbb{R}^m$, claim 2 of Translation and Reflection Invariance of Lebesgue Measure on $\mathbb{R}^n$ applied to $\varphi_\eta$ with the shift $-a$ shows that $z\mapsto\varphi_\eta(z+(-a))$ is measurable with integral $\int\varphi_\eta\,d\lambda_m=1$.

Step 2: claim 2. Fix $i$ and $z\in\mathbb{R}^m$. In the notation of Slice Function and the Partial Derivative (with $U=\mathbb{R}^m$, the point $z$ in the role of the base point of that lemma, and admissible radius $\rho=1$, every point of $\mathbb{R}^m$ lying in $U$), the slice function $g_i(s)=\varphi_\eta(z[s])$, $s\in I=(z_i-1,z_i+1)$, equals $C\exp(-s^{2}/(2\eta))$ with the constant $C=c_\eta\exp\bigl(-\sum_{k\ne i}z_k^{2}/(2\eta)\bigr)>0$, by the functional equation. The identity function $s\mapsto s$ is differentiable at every interior point of $I$ with derivative $1$ (its difference quotients are identically $1$), so by the product and constant-multiple rules (claims 3 and 2 of Sum, Constant Multiple, and Product Rules for One-Dimensional Derivatives) the function $\gamma(s)=-s^{2}/(2\eta)$ is differentiable on $I$ with $\gamma'(s)=-s/\eta$. The exponential function is differentiable at every real number with $\exp'=\exp$ (claim 3 of Basic Properties of the Exponential Function), so by Chain Rule for One-Dimensional Derivatives (with $J=\mathbb{R}$) and the constant-multiple rule, $g_i=C\,(\exp\circ\gamma)$ is differentiable at $z_i$ with

$$g_i'(z_i)=C\exp(\gamma(z_i))\,\gamma'(z_i)=-\frac{z_i}{\eta}\,g_i(z_i)=-\frac{z_i}{\eta}\,\varphi_\eta(z).$$

By claim 2 of Slice Function and the Partial Derivative, $\partial_i\varphi_\eta(z)$ exists and equals $g_i'(z_i)$. The function $z\mapsto-(z_i/\eta)\varphi_\eta(z)$ is sequentially continuous as a product of such functions. For the bound, if $z_i=0$ there is nothing to prove; otherwise put $u=z_i^{2}/(2\eta)>0$, so that by (E2) $\exp(u)\ge2\sqrt u=2|z_i|/\sqrt{2\eta}$ and

$$|\partial_i\varphi_\eta(z)|=c_\eta\frac{|z_i|}{\eta}\exp\Bigl(-\frac{\lVert z\rVert^{2}}{2\eta}\Bigr)\le c_\eta\frac{|z_i|}{\eta}\exp(-u)\le c_\eta\frac{|z_i|}{\eta}\cdot\frac{\sqrt{2\eta}}{2|z_i|}=\frac{c_\eta}{\sqrt{2\eta}},$$

where we used $\lVert z\rVert^{2}\ge z_i^{2}$ and the monotonicity of $\exp$.

Step 3: claim 3. By the expansion of $\lVert z-a\rVert^{2}$ recorded above and the functional equation,

$$\varphi_\eta(z-a)=c_\eta\exp\Bigl(-\frac{\lVert z\rVert^{2}}{2\eta}\Bigr)\exp\Bigl(\frac{a\cdot z}{\eta}\Bigr)\exp\Bigl(-\frac{\lVert a\rVert^{2}}{2\eta}\Bigr)=\varphi_\eta(z)\exp\bigl(Z_a(z)\bigr)\exp(-\kappa_a/2),$$

which is the tilting identity. Multiplying by $\exp(\kappa_a/2)$, integrating, and using linearity and the last part of claim 1 gives $\int\varphi_\eta\exp(Z_a)\,d\lambda_m=\exp(\kappa_a/2)$. Next, $Z_{2a}=2Z_a$ and $\kappa_{2a}=4\kappa_a$, so the tilting identity for $2a$ gives

$$\frac{\varphi_\eta(z-a)^{2}}{\varphi_\eta(z)}=\varphi_\eta(z)\exp\bigl(2Z_a(z)\bigr)\exp(-\kappa_a)=\exp(-\kappa_a)\exp(2\kappa_a)\,\varphi_\eta(z-2a)=\exp(\kappa_a)\,\varphi_\eta(z-2a),$$

and integrating with claim 1 yields $\exp(\kappa_a)$.

Step 4: claim 4. Write $Z=Z_a$ and $\kappa=\kappa_a$; note $Z_{-a}=-Z$, $\kappa_{-a}=\kappa$, $Z_{ta}=tZ$ and $\kappa_{ta}=t^{2}\kappa$ for real $t$.

Integrability. By (E2), $|Z|\varphi_\eta\le\tfrac12\varphi_\eta(\exp(Z)+\exp(-Z))$, and the right side has integral $\exp(\kappa/2)$ by claim 3 applied to $a$ and to $-a$; so $Z\varphi_\eta$ is integrable. Similarly $Z^{2}\varphi_\eta\le\varphi_\eta(\exp(Z)+\exp(-Z)-2)$, whose integral is $2\exp(\kappa/2)-2$, so $Z^{2}\varphi_\eta$ is integrable; and $|Z|\exp(Z)\varphi_\eta\le\tfrac12(\exp(2Z)+1)\varphi_\eta$ (from $2|Z|\le\exp(Z)+\exp(-Z)$ multiplied by $\exp(Z)$), whose integral is $\tfrac12(\exp(2\kappa)+1)$ by claim 3 for $2a$, so $Z\exp(Z)\varphi_\eta$ is integrable.

First moment. The integrable function $f=Z\varphi_\eta$ satisfies $f(-z)=-f(z)$, because $Z(-z)=-Z(z)$ and $\varphi_\eta(-z)=\varphi_\eta(z)$. Claim 3 of Translation and Reflection Invariance of Lebesgue Measure on $\mathbb{R}^n$ with the reflection $z\mapsto0-z$ gives $\int f\,d\lambda_m=\int f(-z)\,d\lambda_m(z)=-\int f\,d\lambda_m$, so $\int Z\varphi_\eta\,d\lambda_m=0$.

Second moment. If $a=0$ then $Z=0=\kappa$ and there is nothing to prove, so let $a\ne0$. For $t\in(0,1]$ put $h_t=\bigl(\exp(tZ)+\exp(-tZ)-2\bigr)/t^{2}$. By the partial-sum identity in (E2), for $K\ge2$ and every point,

$$\frac{E_{2K+1}(tZ)+E_{2K+1}(-tZ)-2}{t^{2}}=2\sum_{j=1}^{K}\frac{t^{2j-2}Z^{2j}}{(2j)!}=Z^{2}+2\sum_{j=2}^{K}\frac{t^{2j-2}Z^{2j}}{(2j)!},$$

and since $0\le t^{2j-2}\le t^{2}$ for $j\ge2$, the last sum lies between $0$ and $t^{2}\bigl(E_{2K+1}(Z)+E_{2K+1}(-Z)-2-Z^{2}\bigr)$. Letting $K\to\infty$,

$$Z^{2}\le h_t\le Z^{2}+t^{2}\,(h_1-Z^{2})\qquad\text{pointwise.}$$

Multiplying by $\varphi_\eta$ and integrating (monotonicity and linearity for the integrable functions involved; $h_1\varphi_\eta$ is integrable by claim 3) gives

$$\int Z^{2}\varphi_\eta\,d\lambda_m\le\int h_t\varphi_\eta\,d\lambda_m\le\int Z^{2}\varphi_\eta\,d\lambda_m+t^{2}M,\qquad M=\int(h_1-Z^{2})\varphi_\eta\,d\lambda_m\in[0,\infty).$$

By claim 3 applied to $ta$ and $-ta$, $\int h_t\varphi_\eta\,d\lambda_m=2\bigl(\exp(t^{2}\kappa/2)-1\bigr)/t^{2}=\kappa\,\frac{\exp(s)-1}{s}$ with $s=t^{2}\kappa/2\in(0,\kappa/2]$, and by (E1) this lies between $\kappa$ and $\kappa(1+s\exp(s))\le\kappa+t^{2}\tfrac{\kappa^{2}}{2}\exp(\kappa/2)$. Combining, $\bigl|\int Z^{2}\varphi_\eta\,d\lambda_m-\kappa\bigr|\le t^{2}\max\bigl(M,\tfrac{\kappa^{2}}{2}\exp(\kappa/2)\bigr)$ for every $t\in(0,1]$; a nonnegative real number bounded by $t^{2}C$ for every $t\in(0,1]$ (with a fixed real $C\ge0$) is $0$, whence $\int Z^{2}\varphi_\eta\,d\lambda_m=\kappa$.

Tilted first moment. By the tilting identity, $Z(z)\exp(Z(z))\varphi_\eta(z)=\exp(\kappa/2)\,Z(z)\varphi_\eta(z-a)$. Put $f(z)=Z(z)\varphi_\eta(z-a)$. Since $Z(x+a)=(a\cdot x+a\cdot a)/\eta=Z(x)+\kappa$, we have $f(x+a)=(Z(x)+\kappa)\varphi_\eta(x)$, which is integrable by the above and claim 1; so by claim 3 of Translation and Reflection Invariance of Lebesgue Measure on $\mathbb{R}^n$ $f$ is integrable and

$$\int f\,d\lambda_m=\int f(x+a)\,d\lambda_m(x)=\int Z\varphi_\eta\,d\lambda_m+\kappa\int\varphi_\eta\,d\lambda_m=\kappa.$$

Hence $\int Z\exp(Z)\varphi_\eta\,d\lambda_m=\kappa\exp(\kappa/2)$.

Step 5: claim 5. By claim 2, $\sum_{i}a_i\partial_i\varphi_\eta(z)=-\frac{a\cdot z}{\eta}\varphi_\eta(z)=-Z(z)\varphi_\eta(z)$, and by the tilting identity $\varphi_\eta(z-a)=\varphi_\eta(z)\exp(Z(z)-\kappa/2)$; so $R_a=\varphi_\eta\bigl(\exp(Z-\kappa/2)-1-Z\bigr)$, which is sequentially continuous. Expanding the square,

$$\frac{R_a^{2}}{\varphi_\eta}=\varphi_\eta\Bigl(\exp(2Z-\kappa)+1+Z^{2}-2\exp(Z-\kappa/2)-2Z\exp(Z-\kappa/2)+2Z\Bigr).$$

Each of the six terms is integrable: $\exp(2Z)\varphi_\eta$ and $\exp(Z)\varphi_\eta$ by claim 3 (for $2a$ and $a$), $\varphi_\eta$ by claim 1, and $Z^{2}\varphi_\eta$, $Z\exp(Z)\varphi_\eta$, $Z\varphi_\eta$ by claim 4. Hence $R_a^{2}/\varphi_\eta$ is integrable, and by linearity together with the values $\int\exp(2Z)\varphi_\eta=\exp(2\kappa)$, $\int\exp(Z)\varphi_\eta=\exp(\kappa/2)$ and those of claim 4,

$$\int\frac{R_a^{2}}{\varphi_\eta}\,d\lambda_m=\exp(-\kappa)\exp(2\kappa)+1+\kappa-2\exp(-\kappa/2)\exp(\kappa/2)-2\exp(-\kappa/2)\,\kappa\exp(\kappa/2)+0=\exp(\kappa)-1-\kappa.$$

Finally, for $K\ge2$, $E_K(\kappa)-1-\kappa=\sum_{k=2}^{K}\kappa^{k}/k!$, and $k!\ge2\,(k-2)!$ for $k\ge2$, so this is at most $\frac{\kappa^{2}}{2}\sum_{k=2}^{K}\kappa^{k-2}/(k-2)!=\frac{\kappa^{2}}{2}E_{K-2}(\kappa)$; letting $K\to\infty$ gives $\exp(\kappa)-1-\kappa\le\frac{\kappa^{2}}{2}\exp(\kappa)$. $\blacksquare$