Proof of Quadratic Test Functions: a Global $C^2$ Majorant and Minorant Matching a $C^2$ Function to Second Order at a Point

Each result cited below is universally quantified over the data appearing in its own statement, and is applied here to the data named at the point of use. Write $p=D\varphi(\hat{x})\in H$ and $b=D^{2}\varphi(\hat{x})\in\mathrm{Sym}(H)$, and let $T_{b}\in\mathcal{L}(H)$ be the operator representing $b$, so that $b(z,y)=\langle T_{b}z,y\rangle$ for all $z,y\in H$, by claim 1 of The Bounded Symmetric Operator Represented by a Bounded Symmetric Bilinear Form on a Real Hilbert Space. The set $H$ is open in $(H,d)$, since every open ball of $(H,d)$ is a subset of $H$, so Open Subset of a Metric Space is satisfied. Recall that $d(x,y)=|x-y|$ for $x,y\in H$, by Real Inner Product Spaces (iii).

Claim 1. Let $x\in H$ and put $z=x-\hat{x}$. By bilinearity and symmetry of $b$ (Bounded Symmetric Bilinear Forms on a Real Inner Product Space: Norm, Order, Identity Form and Restriction (i)),

$$b(z,z)=b(x,x)-b(x,\hat{x})-b(\hat{x},x)+b(\hat{x},\hat{x})=b(x,x)-2\,b(\hat{x},x)+b(\hat{x},\hat{x}),$$

and $b(\hat{x},x)=\langle T_{b}\hat{x},x\rangle$. Likewise $\langle p,z\rangle=\langle p,x\rangle-\langle p,\hat{x}\rangle$ by bilinearity of the inner product (Elementary Identities in a Real Inner Product Space (i)). Hence, with

$$c_{0}=\varphi(\hat{x})-\langle p,\hat{x}\rangle+\tfrac{1}{2}b(\hat{x},\hat{x})\in\mathbb{R},$$

we have $T=Q+a+q$, where $Q,a,q:H\to\mathbb{R}$ are given by

$$Q(x)=\tfrac{1}{2}b(x,x),\qquad a(x)=\langle p-T_{b}\hat{x},x\rangle+c_{0},\qquad q(x)=\tfrac{2\eta}{2}\,|x-\hat{x}|^{2}.$$

By claim 2 of Affine and Quadratic Functions on a Real Hilbert Space are of Class $C^2$, $Q\in C^{2}(H)$ with $DQ(x)=T_{b}x$ and $D^{2}Q(x)=b$; by claim 1 of that lemma, $a\in C^{2}(H)$ with $Da(x)=p-T_{b}\hat{x}$ and $D^{2}a(x)=0_{\mathrm{Sym}}$; and by claim 3 of that lemma, applied with $\alpha=2\eta$ and $y_{0}=\hat{x}$, $q\in C^{2}(H)$ with $Dq(x)=2\eta\,(x-\hat{x})$ and $D^{2}q(x)=2\eta I$. Applying claim 2 of Constants, Sums, Scalar Multiples and Differences of Differentiable Functions on an Open Subset of a Real Inner Product Space twice, $T\in C^{2}(H)$ and, for every $x\in H$,

$$DT(x)=T_{b}x+(p-T_{b}\hat{x})+2\eta\,(x-\hat{x}),\qquad D^{2}T(x)=b+0_{\mathrm{Sym}}+2\eta I=b+2\eta I,$$

the last equality because $0_{\mathrm{Sym}}$ is the zero vector of the vector space $\mathrm{Sym}(H)$, by claim 1 of Elementary Properties of Bounded Symmetric Bilinear Forms: Norm, Quadratic Form, Order and Continuity.

Taking $x=\hat{x}$ gives $DT(\hat{x})=T_{b}\hat{x}+p-T_{b}\hat{x}+2\eta\,0_{H}=p=D\varphi(\hat{x})$. Evaluating the defining formula for $T$ at $\hat{x}$, the last three terms vanish: $\hat{x}-\hat{x}=0_{H}$, $\langle p,0_{H}\rangle=0$ by Elementary Identities in a Real Inner Product Space (ii), $|b(0_{H},0_{H})|\le\lVert b\rVert\,|0_{H}|\,|0_{H}|=0$ by claim 2 of Elementary Properties of Bounded Symmetric Bilinear Forms: Norm, Quadratic Form, Order and Continuity so that $b(0_{H},0_{H})=0$ by claim 1 of Properties of the Absolute Value in an Ordered Field, and $|0_{H}|^{2}=0$. Hence $T(\hat{x})=\varphi(\hat{x})$.

For the norm bound, $D^{2}T(x)-D^{2}\varphi(\hat{x})=(b+2\eta I)-b=2\eta I$ in the vector space $\mathrm{Sym}(H)$. For all $x',y'\in H$,

$$|(2\eta I)(x',y')|=|2\eta\,\langle x',y'\rangle|=|2\eta|\,|\langle x',y'\rangle|\le|2\eta|\,|x'|\,|y'|$$

by Bounded Symmetric Bilinear Forms on a Real Inner Product Space: Norm, Order, Identity Form and Restriction (iv), claim 4 of Properties of the Absolute Value in an Ordered Field and The Cauchy–Schwarz Inequality in a Real Inner Product Space; moreover $|2\eta|=2\,|\eta|$ by claim 4 of Properties of the Absolute Value in an Ordered Field, and $0\le 2|\eta|$ by claim 1 of that lemma and claim 2 of Elementary Arithmetic in an Ordered Field. Claim 2 of Elementary Properties of Bounded Symmetric Bilinear Forms: Norm, Quadratic Form, Order and Continuity therefore gives $\lVert 2\eta I\rVert\le 2|\eta|$, which is the asserted bound.

Finally, let $U'\subseteq H$ be open in $(H,d)$. By claim 4 of Basic Properties of Differentiability on an Open Subset of a Real Inner Product Space, applied with the open sets $U'\subseteq H$, the restriction of $T$ to $U'$ belongs to $C^{2}(U')$ and has the gradients and Hessians of $T$ at the points of $U'$.

Claim 2. Suppose $0<\eta$. By claim 2 of Segment Derivatives, the Second-Order Taylor Expansion, and the Second-Order Condition at a Local Extremum, applied to $\varphi\in C^{2}(U)$ at the point $\hat{x}$ with the positive number $\eta$ in the role of its $\varepsilon$, there is a positive $\rho\in\mathbb{R}$ such that every $z\in H$ with $|z|<\rho$ satisfies $\hat{x}+z\in U$ and

$$\Bigl|\varphi(\hat{x}+z)-\varphi(\hat{x})-\langle p,z\rangle-\tfrac{1}{2}b(z,z)\Bigr|\le\eta\,|z|^{2}.$$

Let $x\in H$ satisfy $|x-\hat{x}|<\rho$ and put $z=x-\hat{x}$, so that $x=\hat{x}+z$ and $|z|<\rho$; then $x\in U$, and by claim 3 of Properties of the Absolute Value in an Ordered Field together with claim 6 of that lemma,

$$\varphi(x)-\varphi(\hat{x})-\langle p,z\rangle-\tfrac{1}{2}b(z,z)\le\eta\,|z|^{2},$$

which, rearranged by claim 3 of Elementary Arithmetic in an Ordered Field, is exactly $\varphi(x)\le T(x)$.

Claim 3. Suppose $\eta<0$, so that $-\eta$ is positive by claim 4 of Elementary Order Arithmetic in an Ordered Field. Applying claim 2 of Segment Derivatives, the Second-Order Taylor Expansion, and the Second-Order Condition at a Local Extremum as in claim 2 above, but with $-\eta$ in the role of its $\varepsilon$, we obtain a positive $\rho$ such that every $x\in H$ with $|x-\hat{x}|<\rho$ lies in $U$ and satisfies, with $z=x-\hat{x}$,

$$-(-\eta)\,|z|^{2}\le\varphi(x)-\varphi(\hat{x})-\langle p,z\rangle-\tfrac{1}{2}b(z,z),$$

again by claim 3 of Properties of the Absolute Value in an Ordered Field and claim 6 of that lemma. Since $-(-\eta)=\eta$, rearranging by claim 3 of Elementary Arithmetic in an Ordered Field gives $T(x)\le\varphi(x)$.

Claim 4. Suppose $0<\eta$ and let $\rho$ be as furnished by claim 2. Let $\delta_{0}$ be a positive real number witnessing, as in Local Maximum of a Function Relative to a Subset of a Metric Space, that the function with value $g(x)-\varphi(x)$ at $x$ has a local maximum at $\hat{x}$ relative to $A$. Put $\delta_{1}=\min\{\delta_{0},\rho\}$, which is positive and satisfies $\delta_{1}\le\delta_{0}$ and $\delta_{1}\le\rho$ by claims 1 and 2 of Elementary Properties of the Minimum of Two Elements.

Let $y\in A$ satisfy $d(\hat{x},y)<\delta_{1}$. Then $d(\hat{x},y)<\rho$ and $d(\hat{x},y)<\delta_{0}$ by claim 2 of Elementary Order Arithmetic in an Ordered Field. Now $|y-\hat{x}|=d(y,\hat{x})=d(\hat{x},y)<\rho$ by the symmetry axiom, so $\varphi(y)\le T(y)$ by claim 2; hence $0\le T(y)-\varphi(y)$ by claim 3 of Elementary Arithmetic in an Ordered Field, and since

$$\bigl(g(y)-\varphi(y)\bigr)-\bigl(g(y)-T(y)\bigr)=T(y)-\varphi(y),$$

the same claim gives $g(y)-T(y)\le g(y)-\varphi(y)$. From $d(\hat{x},y)<\delta_{0}$ we get $g(y)-\varphi(y)\le g(\hat{x})-\varphi(\hat{x})$, and $g(\hat{x})-\varphi(\hat{x})=g(\hat{x})-T(\hat{x})$ because $T(\hat{x})=\varphi(\hat{x})$ by claim 1. Combining these by transitivity of the order gives $g(y)-T(y)\le g(\hat{x})-T(\hat{x})$. Thus $\delta_{1}$ witnesses that the function with value $g(x)-T(x)$ at $x$ has a local maximum at $\hat{x}$ relative to $A$.

Claim 5. Suppose $\eta<0$ and let $\rho$ be as furnished by claim 3. The argument of claim 4 applies verbatim with Local Minimum of a Function Relative to a Subset of a Metric Space in place of Local Maximum of a Function Relative to a Subset of a Metric Space, using $T(y)\le\varphi(y)$ from claim 3 in place of $\varphi(y)\le T(y)$, which gives $g(y)-\varphi(y)\le g(y)-T(y)$, and the inequality $g(\hat{x})-\varphi(\hat{x})\le g(y)-\varphi(y)$ in place of its reverse.