Proof of The Multiscale Gaussian Kernel: Definition by a Series, Regularity and Bounds, the Smoothed Potential of a Probability Measure and the Kernel Pairing of Two Measures

Each result cited is universally quantified over the data in its own statement. Let $A_{0},\dots,A_{3}$ be the constants of The Gaussian Kernels on Euclidean Space: Scaling, Derivatives up to Order Three, the Convolution Identity, Moments and Tails §bounds, and let the notation $s^{-m/2}$ be as in The Gaussian Kernels on Euclidean Space: Scaling, Derivatives up to Order Three, the Convolution Identity, Moments and Tails.

Step 0 (the weights beat the bounds). For $k\in\mathbb{N}$ one has $4^{k}=(2\cdot2)^{k}=2^{k}2^{k}$ by claim 3 of Properties of Natural Number Powers in a Field, so $(2^{-k})^{2}=4^{-k}=s_{k}$ and $\sqrt{s_{k}}=2^{-k}$ by the uniqueness in Existence and Uniqueness of the Nonnegative Square Root, and $\sqrt{2s_{k}}=\sqrt{2}\,\sqrt{s_{k}}=\sqrt{2}\,2^{-k}$ by claim 5 of Real Powers Through the Exponential, and Elementary Asymptotic Tools: Monotonicity, Null Sequences of Negative Powers, Exponential Domination, Integer Rounding, and Square-Root and Exponential Inequalities. Hence, for a natural number $m$, $(2s_{k})^{-m/2}$, the inverse of $(\sqrt{2}\,2^{-k})^{m}=(\sqrt{2})^{m}2^{-km}$ (claim 3 of Properties of Natural Number Powers in a Field and claim 1 of Real Powers Through the Exponential, and Elementary Asymptotic Tools: Monotonicity, Null Sequences of Negative Powers, Exponential Domination, Integer Rounding, and Square-Root and Exponential Inequalities), equals $(\sqrt{2})^{-m}2^{km}$, and

by the algebra of powers in claim 1 of Real Powers Through the Exponential, and Elementary Asymptotic Tools: Monotonicity, Null Sequences of Negative Powers, Exponential Domination, Integer Rounding, and Square-Root and Exponential Inequalities. For $m\in\{0,1,2,3\}$ put $a^{(m)}_{k}=A_{m}(\sqrt{2})^{-(q+m)}(2^{-(8-m)})^{k}$. Since $0<2^{-(8-m)}<1$, the series $\sum_{k}a^{(m)}_{k}$ converges by Series of Nonnegative Real Numbers, Comparison, and the Geometric Series §geometric together with the homogeneity of Elementary Properties of Series of Real Numbers §linearity; let $M_{m}=\sum_{k=1}^{\infty}a^{(m)}_{k}$, a nonnegative real number depending only on $q$. By The Gaussian Kernels on Euclidean Space: Scaling, Derivatives up to Order Three, the Convolution Identity, Moments and Tails §bounds applied with $s=2s_{k}\le1$, for all $z$ and $i,j,l\in[q]$,

Claim 1. Each $g_{2s_{k}}$ is smooth (The Gaussian Kernels on Euclidean Space: Scaling, Derivatives up to Order Three, the Convolution Identity, Moments and Tails §derivatives), hence of class $C^{1}$, and so is $w_{k}g_{2s_{k}}$ with $\partial_{i}(w_{k}g_{2s_{k}})=w_{k}\partial_{i}g_{2s_{k}}$ (claims 1 and 3 of Constants, Coordinate Functions, Sums and Products of $C^k$ Functions on a Euclidean Open Set); likewise $w_{k}\partial_{i}g_{2s_{k}}$ and $w_{k}\partial_{j}\partial_{i}g_{2s_{k}}$ are of class $C^{1}$ with partial derivatives $w_{k}\partial_{j}\partial_{i}g_{2s_{k}}$ and $w_{k}\partial_{l}\partial_{j}\partial_{i}g_{2s_{k}}$. Apply Term-by-Term Differentiation of a Series of Continuously Differentiable Functions with Summable Uniform Bounds three times: to $f_{k}=w_{k}g_{2s_{k}}$ with the bounds $a^{(0)}_{k},a^{(1)}_{k}$; to $f_{k}=w_{k}\partial_{i}g_{2s_{k}}$ with $a^{(1)}_{k},a^{(2)}_{k}$; and to $f_{k}=w_{k}\partial_{j}\partial_{i}g_{2s_{k}}$ with $a^{(2)}_{k},a^{(3)}_{k}$. By Term-by-Term Differentiation of a Series of Continuously Differentiable Functions with Summable Uniform Bounds §convergence all the series in the statement converge absolutely, and by Term-by-Term Differentiation of a Series of Continuously Differentiable Functions with Summable Uniform Bounds §differentiation: $K$ is of class $C^{1}$ with $\partial_{i}K=\sum_{k}w_{k}\partial_{i}g_{2s_{k}}$; this function is of class $C^{1}$ with $\partial_{j}\partial_{i}K=\sum_{k}w_{k}\partial_{j}\partial_{i}g_{2s_{k}}$; and this is of class $C^{1}$ with $\partial_{l}\partial_{j}\partial_{i}K=\sum_{k}w_{k}\partial_{l}\partial_{j}\partial_{i}g_{2s_{k}}$; together with the bounds $|K|\le M_{0}$, $|\partial_{i}K|\le M_{1}$, $|\partial_{j}\partial_{i}K|\le M_{2}$, $|\partial_{l}\partial_{j}\partial_{i}K|\le M_{3}$. By clause 2 of C^k Maps on a Euclidean Open Set, applied twice, $K$ is of class $C^{3}$. Its terms being nonnegative, $K(z)\ge0$ by Series of Nonnegative Real Numbers, Comparison, and the Geometric Series §criterion; and $K(-z)=K(z)$ because the two series $\sum_{k}w_{k}g_{2s_{k}}(-z)$ and $\sum_{k}w_{k}g_{2s_{k}}(z)$ have the same terms, by the evenness in The Gaussian Kernels on Euclidean Space: Scaling, Derivatives up to Order Three, the Convolution Identity, Moments and Tails §derivatives.

Claim 2. Let $\nu\in\mathcal{P}(\mathbb{R}^{q})$, and let $M=\max\{M_{0},M_{1},M_{2},M_{3}\}$. By claim 1, $K$, $\partial_{i}K$ and $\partial_{j}\partial_{i}K$ are of class $C^{1}$ and bounded, together with their partial derivatives, by $M$. By The Convolution of a Bounded Continuous Function with a Probability Measure on Euclidean Space: Continuity, Boundedness and Differentiation Under the Integral §continuous the function $y\mapsto K(x-y)$ is Borel and bounded for each $x$, so $K*\nu$ is defined, and by The Convolution of a Bounded Continuous Function with a Probability Measure on Euclidean Space: Continuity, Boundedness and Differentiation Under the Integral §derivative applied to $G=K$, then to $G=\partial_{i}K$, then to $G=\partial_{j}\partial_{i}K$, the function $K*\nu$ is of class $C^{1}$ with $\partial_{i}(K*\nu)=(\partial_{i}K)*\nu$, this is of class $C^{1}$ with $\partial_{j}\partial_{i}(K*\nu)=(\partial_{j}\partial_{i}K)*\nu$, and this is of class $C^{1}$ with $\partial_{l}\partial_{j}\partial_{i}(K*\nu)=(\partial_{l}\partial_{j}\partial_{i}K)*\nu$, which are the displayed formulas; $K*\nu$ is of class $C^{3}$ by clause 2 of C^k Maps on a Euclidean Open Set. The bounds by $M_{0},\dots,M_{3}$ follow from The Convolution of a Bounded Continuous Function with a Probability Measure on Euclidean Space: Continuity, Boundedness and Differentiation Under the Integral §continuous applied with $C=M_{m}$ to the respective derivative of $K$, and $K*\nu\ge0$ since its integrand is nonnegative (claim 1 of Linearity and Monotonicity of the Lebesgue Integral). For the series representation fix $x$ and let $F_{m}(y)=\sum_{k=1}^{m}w_{k}g_{2s_{k}}(x-y)$, the $m$th partial sum of the series defining $K(x-y)$. Each $F_{m}$ is a nonnegative Borel function of $y$ (The Convolution of a Bounded Continuous Function with a Probability Measure on Euclidean Space: Continuity, Boundedness and Differentiation Under the Integral §continuous for each term, claim 2 of Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions), the sequence $(F_{m}(y))_{m}$ is nondecreasing with limit $K(x-y)$ (Series of Real Numbers §convergent), and $\int F_{m}\,d\nu=\sum_{k=1}^{m}w_{k}(g_{2s_{k}}*\nu)(x)$ by the additivity and homogeneity in claim 1 of Linearity and Monotonicity of the Lebesgue Integral, applied inductively over the summands (Principle of Induction for the Natural Numbers, claim 1 of Properties of Finite Sums). By Monotone Convergence Theorem, $\int K(x-y)\,\nu(dy)$ is the supremum of these partial sums; the partial sums being nondecreasing and bounded by this finite supremum, they converge to it by A Bounded Monotone Sequence of Real Numbers Converges, which is the statement that $\sum_{k}w_{k}(g_{2s_{k}}*\nu)(x)$ converges, absolutely since its terms are nonnegative, with sum $(K*\nu)(x)$.

Claim 3. The map $z\mapsto\mathrm{pr}_{1}(z)-\mathrm{pr}_{2}(z)$ on $\mathbb{R}^{q+q}$ is Borel (its components are differences of the Borel components of the projections, Pairs of Euclidean Points: Coordinate Projections, Pairings, the Product Measure on a Euclidean Space, Borel Norm Functions and Finite Sets §projections and claim 2 of Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions), $K$ is continuous (clause 1 of C^k Maps on a Euclidean Open Set) hence Borel (Probability Measures on Euclidean Space and Random Vectors: Standing Notation §borel-maps), so $z\mapsto K(\mathrm{pr}_{1}(z)-\mathrm{pr}_{2}(z))$ is Borel (claim 4 of Borel Measurability and Bounded Integration on a Metric Space), with values in $[0,M_{0}]$; hence $\mathcal{K}(\mu,\nu)$ is a real number with $0\le\mathcal{K}(\mu,\nu)\le M_{0}$ by claim 6(b) of Borel Measurability and Bounded Integration on a Metric Space and claim 1 of Linearity and Monotonicity of the Lebesgue Integral. By Pairs of Euclidean Points: Coordinate Projections, Pairings, the Product Measure on a Euclidean Space, Borel Norm Functions and Finite Sets §product and Tonelli and Fubini Theorems (Tonelli; $\mu$ and $\nu$ are finite, the integrand nonnegative and measurable for the product $\sigma$-algebra as the composition with $\iota$ of a Borel function),

and the last expression equals $\int\bigl(\int K(y-x)\,\mu(dx)\bigr)\nu(dy)$ by the evenness of $K$, which is the corresponding iterated integral for $\mathcal{K}(\nu,\mu)$; hence $\mathcal{K}(\mu,\nu)=\mathcal{K}(\nu,\mu)$. For the series, let $x\in\mathbb{R}^{q}$ and let $F_{m}(x)=\sum_{k=1}^{m}w_{k}(g_{2s_{k}}*\nu)(x)$; these are nonnegative Borel functions of $x$ (The Convolution of a Bounded Continuous Function with a Probability Measure on Euclidean Space: Continuity, Boundedness and Differentiation Under the Integral §continuous), nondecreasing in $m$, with limit $(K*\nu)(x)$ by claim 2. As in claim 2, Monotone Convergence Theorem and A Bounded Monotone Sequence of Real Numbers Converges give that $\sum_{k}w_{k}\int g_{2s_{k}}*\nu\,d\mu$ converges, with nonnegative terms, to $\int K*\nu\,d\mu=\mathcal{K}(\mu,\nu)$. Finally, for each $k$, Tonelli as above gives $\int g_{2s_{k}}*\nu\,d\mu=\int_{\mathbb{R}^{q+q}}g_{2s_{k}}(\mathrm{pr}_{1}(z)-\mathrm{pr}_{2}(z))\,(\mu\boxtimes\nu)(dz)$, which equals $\int_{\mathbb{R}^{q}}(g_{s_{k}}*\mu)(g_{s_{k}}*\nu)\,d\lambda_{q}$ by Gaussian Smoothing of a Probability Measure on Euclidean Space: Regularity, Mass, Duality, Approximation of a Bounded Function with a Modulus of Continuity, and the Pairing Identity §pairing, applicable since $0<s_{k}\le\tfrac12$; that clause also gives the integrability of each product.