Proof of Mollification at a Point of Twice Differentiability: Convergence of the Mollified Gradient and Hessian, and a Hessian Bound for Lipschitz Functions

Each result cited is universally quantified over the data in its own statement. Integrals are with respect to $\lambda_{n}$, written $\int\cdots dz$; $\delta_{ij}$ is the entry of $I_{n}$ in row $i$ and column $j$; and $z_{k}$ is the $k$th coordinate of $z\in\mathbb{R}^{n}$.

Step 0: a matrix bound. Let $X\in\mathcal{S}(n)$ and $c\ge0$ with $|X_{ij}|\le c$ for all $i,j$. For $z\in\mathbb{R}^{n}$, claim 4 of Linearity of the Matrix-Vector Product and the Quadratic Form as a Double Sum and $2|z_{i}||z_{j}|\le z_{i}^{2}+z_{j}^{2}$ give

using $\lVert z\rVert^{2}=\sum_{i}z_{i}^{2}$ (Euclidean Norm on $\mathbb{R}^n$). Hence the norm satisfies $\lVert X\rVert\le nc$, being the least upper bound of the numbers $|\xi\cdot(X\xi)|$ with $\lVert\xi\rVert\le1$, and $-(nc)I_{n}\preceq X\preceq(nc)I_{n}$ by claim 3 of Properties of the Norm of a Symmetric Real Matrix.

Step 1: derivatives of $f_{\varepsilon}$ and of the kernel. Fix $\varepsilon>0$ and write $\kappa=\rho_{\varepsilon}$, a mollifier kernel of radius $r=\varepsilon\delta$, smooth by Mollifier Kernel of Radius $\delta$ on $\mathbb{R}^n$. By Differentiating a Convolution through the Kernel (claim 1 for the kernel, claim 2 for the convolution), applied first to $\kappa$ and then to the kernel $\partial_{i}\kappa$, which is again of class $C^{1}$ and vanishes off $\bar B(0_{\mathbb{R}^{n}},r)$, we get for all $x\in\mathbb{R}^{n}$ and $i,j\in[n]$, by Convolution of a Continuous Function with a Compactly Supported Continuous Kernel,

with $\partial_{i}\kappa$ and $\partial_{j}\partial_{i}\kappa$ continuous and vanishing off that ball. By claim 2 of Partial Derivatives, Continuity and $C^k$ Regularity under a Scaling Substitution, applied twice with $c=0_{\mathbb{R}^{n}}$, $\lambda=\varepsilon^{-1}$ and $\mu=(\varepsilon^{-1})^{n}$, we have $\partial_{i}\kappa(z)=(\varepsilon^{-1})^{n+1}\partial_{i}\rho(\varepsilon^{-1}z)$ and $\partial_{j}\partial_{i}\kappa(z)=(\varepsilon^{-1})^{n+2}\partial_{j}\partial_{i}\rho(\varepsilon^{-1}z)$. The functions $\rho$ and $\partial_{i}\rho$ are of class $C^{1}$ and vanish off the compact ball $\bar B(0_{\mathbb{R}^{n}},\delta)$, hence are compactly supported, so $\partial_{i}\rho$ and $\partial_{j}\partial_{i}\rho$ are integrable by Integration by Parts on Euclidean Space Against a Compactly Supported Function of Class $C^{1}$ §vanishing; put $a_{i}=\int|\partial_{i}\rho|\,dz$ and $a_{ij}=\int|\partial_{j}\partial_{i}\rho|\,dz$. Claim 2 of Scaling of Lebesgue Measure and the Lebesgue Integral on $\mathbb{R}^n$ with $c=\varepsilon^{-1}$ then gives

Step 2: moments of the kernel. The same argument shows that $\kappa$ and $\partial_{i}\kappa$ are of class $C^{1}$ and compactly supported, and the coordinate functions $z\mapsto z_{k}$ and $z\mapsto z_{k}z_{l}$ are of class $C^{1}$ with $\partial_{j}z_{k}=\delta_{jk}$ and $\partial_{j}(z_{k}z_{l})=\delta_{jk}z_{l}+\delta_{jl}z_{k}$. Hence Integration by Parts on Euclidean Space Against a Compactly Supported Function of Class $C^{1}$ §vanishing and Integration by Parts on Euclidean Space Against a Compactly Supported Function of Class $C^{1}$ §parts, together with $\int\kappa\,dz=1$, gives

(All integrands are continuous and vanish off a compact ball, hence are integrable.)

Claim 1. Let $\beta=\lVert B\rVert$, so $|z\cdot(Bz)|\le\beta\lVert z\rVert^{2}$ by claim 2 of Properties of the Norm of a Symmetric Real Matrix, and put $S(z)=\tfrac12\,z\cdot(Bz)=\tfrac12\sum_{k,l}B_{kl}z_{k}z_{l}$ (claim 4 of Linearity of the Matrix-Vector Product and the Quadratic Form as a Double Sum). Let $\eta>0$ and let $\delta_{\eta}>0$ be as in the definition of twice differentiability at $x$ for $\eta$; writing $R(z)=f(x-z)-f(x)+p\cdot z-S(z)$, and noting $(-z)\cdot(B(-z))=z\cdot(Bz)$, we have $|R(z)|\le\eta\lVert z\rVert^{2}$ whenever $\lVert z\rVert<\delta_{\eta}$. Let $\varepsilon>0$ with $r=\varepsilon\delta<\delta_{\eta}$. Substituting $f(x-z)=f(x)-p\cdot z+S(z)+R(z)$ into (1) and using (3) and the linearity of the integral (claim 2 of Linearity and Monotonicity of the Lebesgue Integral):

On $\bar B(0_{\mathbb{R}^{n}},r)$, outside which the kernels vanish, $|S|\le\tfrac12\beta r^{2}$ and $|R|\le\eta r^{2}$; with (2) and monotonicity of the integral this gives

Now let $(\varepsilon_{m})$ be positive and converge to $0$. The first estimate (with $\eta=1$) shows that each coordinate of $Df_{\varepsilon_{m}}(x)$ converges to the corresponding coordinate of $p$, hence $Df_{\varepsilon_{m}}(x)\to p$ in $\mathbb{R}^{n}$, since $\lVert v\rVert^{2}=\sum_{i}v_{i}^{2}\le n\max_{i}v_{i}^{2}$. The second shows that for every $\eta>0$ there is $m_{0}$ with $|(D^{2}f_{\varepsilon_{m}}(x)-B)_{ij}|\le\eta\,\delta^{2}\max_{k,l}a_{kl}$ for all $i,j$ and $m\ge m_{0}$, the entries of the Hessian matrix being $\partial_{j}\partial_{i}f_{\varepsilon}$ (Hessian Matrix of a C^2 Function); by Step 0, $\lVert D^{2}f_{\varepsilon_{m}}(x)-B\rVert\le n\,\eta\,\delta^{2}\max_{k,l}a_{kl}$, so $D^{2}f_{\varepsilon_{m}}(x)\to B$ in $\mathcal{S}(n)$.

Claim 2. Put $K=n\,\delta\max_{i,j}a_{ij}$, which depends only on $n$, $\delta$ and $\rho$. Let $f$ be Lipschitz with constant $L$, $\varepsilon>0$ and $x\in\mathbb{R}^{n}$. By (1) and $\int\partial_{j}\partial_{i}\kappa=0$,

and $|f(x-z)-f(x)|\le L\lVert z\rVert\le L\varepsilon\delta$ where the kernel does not vanish, so by (2) $|\partial_{j}\partial_{i}f_{\varepsilon}(x)|\le L\varepsilon\delta\,\varepsilon^{-2}a_{ij}\le n^{-1}KL\varepsilon^{-1}\le KL\varepsilon^{-1}$ (as $1\le n$). Step 0 with $c=n^{-1}KL\varepsilon^{-1}$ gives $-(KL\varepsilon^{-1})I_{n}\preceq D^{2}f_{\varepsilon}(x)\preceq(KL\varepsilon^{-1})I_{n}$.