Proof of First- and Second-Order Conditions at a Local Extremum of a $C^2$ Function Penalised by $h$ on the Small Space

Throughout, vector identities are those of Elementary Identities in a Vector Space, real arithmetic and order facts are those of The Real Numbers: Standing Notation and Background, and $\tfrac12$ is $2^{-1}$. For $x\in V$ we use $|x|_{H}\le|x|_{V}$ (Hilbert Triples: Standing Notation and Background (i)), and for $t\in\mathbb{R}$ and $z\in V$ we use $|tz|_{V}=|t|\,|z|_{V}$ and $|tz|_{H}=|t|\,|z|_{H}$ (Elementary Identities in a Real Inner Product Space (iv)), together with $h(tz)=\tfrac12|tz|_{V}^{2}=\tfrac12 t^{2}|z|_{V}^{2}$, since $|t|^{2}=|t\cdot t|=|t^{2}|=t^{2}$ by claim 4 of Properties of the Absolute Value in an Ordered Field, $0\le t^{2}$ (claim 5 of Properties of Natural Number Powers in a Field) and the definition of the absolute value.

Claim 1. By Local Maximum of a Function Relative to a Subset of a Metric Space, applied in the metric space $(H,d_{H})$ to the function $\psi-\lambda h$ on the subset $V\cap U$, there is a real $\rho_{1}>0$ such that

$$\psi(y)-\lambda h(y)\le\psi(\hat{x})-\lambda h(\hat{x})\qquad\text{for every }y\in V\cap U\text{ with }d_{H}(\hat{x},y)<\rho_{1}. \tag{1}$$

Fix $z\in V$ with $z\ne0_{H}$, so that $|z|_{V}>0$ by Elementary Identities in a Real Inner Product Space (iii), and put

$$a=\langle D\psi(\hat{x}),z\rangle_{H}-\lambda\langle\hat{x},z\rangle_{V},\qquad b=D^{2}\psi(\hat{x})(z,z)-\lambda|z|_{V}^{2}.$$

Step A: the basic inequality. Let $\varepsilon>0$. By Segment Derivatives, the Second-Order Taylor Expansion, and the Second-Order Condition at a Local Extremum (ii), applied to $\psi\in C^{2}(U)$ at $\hat{x}$ in the space $H$, there is a real $\eta>0$ such that every $w\in H$ with $|w|_{H}<\eta$ satisfies $\hat{x}+w\in U$ and

$$\bigl|\psi(\hat{x}+w)-\psi(\hat{x})-\langle D\psi(\hat{x}),w\rangle_{H}-\tfrac12 D^{2}\psi(\hat{x})(w,w)\bigr|\le\varepsilon|w|_{H}^{2}.$$

Let $\theta$ be the lesser of $\rho_{1}$ and $\eta$ (claim 9 of Elementary Order Arithmetic in an Ordered Field) and put $\tau_{\varepsilon}=\theta/|z|_{V}$, a positive real number. Let $t\in\mathbb{R}$ satisfy $0<|t|<\tau_{\varepsilon}$, and put $w=tz$. Then $w\in V$ and $\hat{x}+w\in V$, because $V$ is a linear subspace of $H$ (Hilbert Triples: Standing Notation and Background (i)), and $|w|_{H}\le|w|_{V}=|t|\,|z|_{V}<\theta$ (multiplying $|t|<\tau_{\varepsilon}$ by $|z|_{V}>0$, claim 10 of Elementary Order Arithmetic in an Ordered Field). Hence $\hat{x}+tz\in U$ by the choice of $\eta$; moreover $d_{H}(\hat{x},\hat{x}+tz)=|tz|_{H}<\theta\le\rho_{1}$ by Real Inner Product Spaces (iii) and Elementary Identities in a Real Inner Product Space (iv). So (1) applies to $y=\hat{x}+tz$. By The Penalty Function $h=\tfrac12|\cdot|_V^2$ of a Hilbert Triple: Expansion Identities, Closed Sublevel Sets, Density and Local Bounds (ii) with $x=\hat{x}$ and $y=\hat{x}+tz$, for which $y-x=tz$,

$$h(\hat{x}+tz)=h(\hat{x})+t\langle\hat{x},z\rangle_{V}+\tfrac12 t^{2}|z|_{V}^{2},$$

using Elementary Identities in a Real Inner Product Space (i) for $\langle\hat{x},tz\rangle_{V}=t\langle\hat{x},z\rangle_{V}$. By the Taylor bound with $w=tz$, claim 3 of Properties of the Absolute Value in an Ordered Field, and the bilinearity of the inner product and of the form $D^{2}\psi(\hat{x})$ (Bounded Symmetric Bilinear Forms on a Real Inner Product Space: Norm, Order, Identity Form and Restriction (i)), which give $\langle D\psi(\hat{x}),tz\rangle_{H}=t\langle D\psi(\hat{x}),z\rangle_{H}$ and $D^{2}\psi(\hat{x})(tz,tz)=t^{2}D^{2}\psi(\hat{x})(z,z)$,

$$\psi(\hat{x}+tz)-\psi(\hat{x})\ge t\langle D\psi(\hat{x}),z\rangle_{H}+\tfrac12 t^{2}D^{2}\psi(\hat{x})(z,z)-\varepsilon t^{2}|z|_{H}^{2}.$$

Subtracting $\lambda$ times the expansion of $h$ and using (1), we obtain

$$ta+\tfrac12 t^{2}b-\varepsilon t^{2}|z|_{H}^{2}\le\bigl(\psi(\hat{x}+tz)-\lambda h(\hat{x}+tz)\bigr)-\bigl(\psi(\hat{x})-\lambda h(\hat{x})\bigr)\le0,$$

that is,

$$ta+\tfrac12 t^{2}b\le\varepsilon t^{2}|z|_{H}^{2}\qquad\text{whenever }0<|t|<\tau_{\varepsilon}. \tag{2}$$

Step B: $a=0$. Suppose $a\ne0$. By the trichotomy of the order, either $a>0$ or $a<0$; let $\sigma$ be $1$ in the first case and $-1$ in the second, so that $\sigma a=|a|>0$ by the definition of the absolute value and claim 1 of Properties of the Absolute Value in an Ordered Field. Apply Step A with $\varepsilon=1$ and put $K=|z|_{H}^{2}-\tfrac12 b$. For every real $\tau$ with $0<\tau<\tau_{1}$, the number $t=\sigma\tau$ satisfies $0<|t|=\tau<\tau_{1}$ and $t^{2}=\tau^{2}$, so (2) gives $\tau|a|+\tfrac12\tau^{2}b\le\tau^{2}|z|_{H}^{2}$, that is, $\tau|a|\le\tau^{2}K$ (claim 3 of Elementary Arithmetic in an Ordered Field); multiplying by $\tau^{-1}\ge0$ (claim 5 of Elementary Arithmetic in an Ordered Field, the inverse of a positive number being positive by claim 4 of Elementary Arithmetic in an Ordered Field) yields $|a|\le\tau K$. If $K\le0$, take $\tau=\tfrac12\tau_{1}$, positive and less than $\tau_{1}$ by claim 8 of Elementary Order Arithmetic in an Ordered Field; then $|a|\le\tau K\le0$ (claim 5 of Elementary Arithmetic in an Ordered Field), contradicting $|a|>0$. If $K>0$, let $m$ be the lesser of $\tau_{1}$ and $|a|/K$ (claim 9 of Elementary Order Arithmetic in an Ordered Field) and take $\tau=\tfrac12 m$, so that $0<\tau<\tau_{1}$ and $\tau<|a|/K$; then $\tau K<|a|$ by claim 10 of Elementary Order Arithmetic in an Ordered Field, again a contradiction. Hence $a=0$.

Step C: $b\le0$. Let $\varepsilon'>0$ and put $\varepsilon=\varepsilon'/(2|z|_{H}^{2}+1)$, which is positive because $2|z|_{H}^{2}+1\ge1>0$. Apply Step A with this $\varepsilon$ and the number $t=\tfrac12\tau_{\varepsilon}$, which satisfies $0<|t|<\tau_{\varepsilon}$ by claim 8 of Elementary Order Arithmetic in an Ordered Field. Since $a=0$, (2) reads $\tfrac12 t^{2}b\le\varepsilon t^{2}|z|_{H}^{2}$; as $t^{2}>0$ (claim 1 of Monotonicity of Squaring on the Nonnegative Elements of an Ordered Field), multiplying by the nonnegative number $2t^{-2}$ (claim 5 of Elementary Arithmetic in an Ordered Field) gives $b\le2\varepsilon|z|_{H}^{2}\le\varepsilon(2|z|_{H}^{2}+1)=\varepsilon'$, the middle inequality because $0\le\varepsilon$. Since $\varepsilon'>0$ was arbitrary, Comparison of Real Numbers with Arbitrary Positive Slack (i) gives $b\le0$.

Step D: conclusion. Steps B and C show that for every $z\in V$ with $z\ne0_{H}$,

$$\langle D\psi(\hat{x}),z\rangle_{H}=\lambda\langle\hat{x},z\rangle_{V}\qquad\text{and}\qquad D^{2}\psi(\hat{x})(z,z)\le\lambda|z|_{V}^{2}. \tag{3}$$

Both relations also hold for $z=0_{H}$: each side of the equality is $0$ by Elementary Identities in a Real Inner Product Space (ii), and $D^{2}\psi(\hat{x})(0_{H},0_{H})=0$ by the bilinearity of the form (write $0_{H}=0\cdot0_{H}$ and use homogeneity), while $\lambda|0_{H}|_{V}^{2}=0$. So (3) holds for every $z\in V$.

Dividing the equality in (3) by $\lambda>0$ and using Elementary Identities in a Real Inner Product Space (i) together with the symmetry of the inner product, $\langle\hat{x},z\rangle_{V}=\langle\lambda^{-1}D\psi(\hat{x}),z\rangle_{H}$ for every $z\in V$. By Hilbert Triple: a Densely and Continuously Embedded Hilbert Space and Its Form Operator (iv), this says precisely that $\hat{x}\in D(A)$ with $A\hat{x}=\lambda^{-1}D\psi(\hat{x})$, whence $\lambda A\hat{x}=D\psi(\hat{x})$. As $\hat{x}\in U$, also $\hat{x}\in D(A)\cap U=W$.

For the second assertion, $D^{2}\psi(\hat{x})|_{V}(z,z)=D^{2}\psi(\hat{x})(z,z)$ for $z\in V$ by Bounded Symmetric Bilinear Forms on a Real Inner Product Space: Norm, Order, Identity Form and Restriction (v), and $(\lambda I_{V})(z,z)=\lambda\langle z,z\rangle_{V}=\lambda|z|_{V}^{2}$ by Bounded Symmetric Bilinear Forms on a Real Inner Product Space: Norm, Order, Identity Form and Restriction (iv) and Real Inner Product Spaces (ii). Thus the inequality in (3) says $D^{2}\psi(\hat{x})|_{V}(z,z)\le(\lambda I_{V})(z,z)$ for every $z\in V$, which is $D^{2}\psi(\hat{x})|_{V}\preceq\lambda I_{V}$ by Bounded Symmetric Bilinear Forms on a Real Inner Product Space: Norm, Order, Identity Form and Restriction (iii).

Claim 2. Put $\psi'=-\psi$, the function $(-1)\psi$. By Constants, Sums, Scalar Multiples and Differences of Differentiable Functions on an Open Subset of a Real Inner Product Space (iii), $\psi'\in C^{2}(U)$ with $D\psi'(\hat{x})=-D\psi(\hat{x})$ and $D^{2}\psi'(\hat{x})=-D^{2}\psi(\hat{x})$. By Local Minimum of a Function Relative to a Subset of a Metric Space there is a real $\rho>0$ with $\psi(\hat{x})+\lambda h(\hat{x})\le\psi(y)+\lambda h(y)$ for every $y\in V\cap U$ with $d_{H}(\hat{x},y)<\rho$; multiplying by $-1$ (claim 4 of Elementary Order Arithmetic in an Ordered Field) gives $\psi'(y)-\lambda h(y)\le\psi'(\hat{x})-\lambda h(\hat{x})$ for the same $y$, so $\psi'-\lambda h$ has a local maximum at $\hat{x}$ relative to $V\cap U$. Since claim 1 holds for every member of $C^{2}(U)$ in the role of $\psi$, it applies to $\psi'$ and gives $\hat{x}\in D(A)$, hence $\hat{x}\in W$; $\lambda A\hat{x}=D\psi'(\hat{x})=-D\psi(\hat{x})$; and $(-D^{2}\psi(\hat{x}))|_{V}\preceq\lambda I_{V}$. By Elementary Properties of Bounded Symmetric Bilinear Forms: Norm, Quadratic Form, Order and Continuity (x), $(-D^{2}\psi(\hat{x}))|_{V}=-(D^{2}\psi(\hat{x})|_{V})$, and multiplying the last relation by $-1$ (Elementary Properties of Bounded Symmetric Bilinear Forms: Norm, Quadratic Form, Order and Continuity (viii)), with $(-1)(-(D^{2}\psi(\hat{x})|_{V}))=D^{2}\psi(\hat{x})|_{V}$ by Elementary Identities in a Vector Space in $\mathrm{Sym}(V)$, gives $-\lambda I_{V}\preceq D^{2}\psi(\hat{x})|_{V}$, where $-\lambda I_{V}=(-1)(\lambda I_{V})$ by Bounded Symmetric Bilinear Forms on a Real Inner Product Space: Norm, Order, Identity Form and Restriction (iv) and the vector space structure of $\mathrm{Sym}(V)$ (Elementary Properties of Bounded Symmetric Bilinear Forms: Norm, Quadratic Form, Order and Continuity (i)).