Proof of Basic Properties of Twice Differentiability at a Point

We use the notation of the statement. Algebraic manipulations of dot products use Bilinearity and Symmetry of the Dot Product on $\mathbb{R}^n$, and $\lVert v\rVert^{2}=v\cdot v$ is claim 1 of Elementary Properties of the Euclidean Norm on $\mathbb{R}^n$.

Claim 1. By claim 3 of Linearity, Compatibility with the Matrix Product, and a Norm Bound for the Matrix-Vector Product there is $C\in\mathbb{R}$ with $0\le C$ and $\lVert Bv\rVert\le C\lVert v\rVert$ for every $v$; put $K=1+\tfrac{1}{2}C$, a positive real number. Applying Twice Differentiability at a Point §twice-differentiable with the value $1$ in place of $\varepsilon$ gives $\delta_{1}>0$ such that every $h$ with $\lVert h\rVert<\delta_{1}$ satisfies $y+h\in U$ and

using claim 5 of Properties of the Absolute Value in an Ordered Field and Cauchy-Schwarz Inequality for the Euclidean Dot Product.

Let $\varepsilon\in\mathbb{R}$ with $0<\varepsilon$ and let $\delta$ be the smaller of $\delta_{1}$ and $\varepsilon/K$. If $0<\lVert h\rVert<\delta$ then $y+h\in U$ and

Let $A$ be the real matrix with one row and $n$ columns given by $A_{1i}=p_{i}$. By Matrix-Vector Product the single coordinate of $Ah$ is $\sum_{i=1}^{n}p_{i}h_{i}=p\cdot h$, and by claim 1 of Elementary Properties of the Euclidean Norm on $\mathbb{R}^n$ the Euclidean norm of a point of $\mathbb{R}^{1}$ is the absolute value of its coordinate, both being the unique nonnegative real number whose square is the square of that coordinate. So the display reads $\lVert f(y+h)-f(y)-Ah\rVert\le\varepsilon\lVert h\rVert$, and Differentiability at a Point for Maps Between Euclidean Spaces is satisfied: $f$ is differentiable at $y$ with derivative matrix $A$. By claim 1 of A Derivative Matrix is the Jacobian Matrix, and is Unique the partial derivatives of $f$ exist at $y$ with $\partial_{i}f(y)=A_{1i}=p_{i}$, so $p=Df(y)$ by Gradient of a Real-Valued Function on a Euclidean Open Set.

Claim 2. Let $y\in U$ and let $\varepsilon>0$. By Second-Order Taylor Expansion with Peano Remainder there is $\delta>0$ such that every $h$ with $\lVert h\rVert<\delta$ satisfies $y+h\in U$ and

Now $\sum_{i}\partial_{i}f(y)h_{i}=Df(y)\cdot h$ by Gradient of a Real-Valued Function on a Euclidean Open Set and Difference, Dot Product, and Orthogonality in $\mathbb{R}^n$. Writing $D^{2}f(y)$ for the Hessian matrix, whose entry in row $i$ and column $j$ is $\partial_{i}\partial_{j}f(y)$, claim 4 of Linearity of the Matrix-Vector Product and the Quadratic Form as a Double Sum gives

which equals $\sum_{i}\sum_{j}\partial_{j}\partial_{i}f(y)h_{i}h_{j}$ by claim 1 of Equality of Mixed Second Partial Derivatives and Symmetry of the Hessian. Since $D^{2}f(y)\in\mathcal{S}(n)$ by claim 2 of Equality of Mixed Second Partial Derivatives and Symmetry of the Hessian, the display is exactly the condition of Twice Differentiability at a Point §twice-differentiable with first-order coefficient $Df(y)$ and Hessian $D^{2}f(y)$. By the uniqueness recorded in Twice Differentiability at a Point §hessian, the Hessian in the sense of that definition is the Hessian matrix, so the two uses of the notation agree.

Claim 3. By Local Maximum of a Function Relative to a Subset of a Metric Space, applied in the metric space $(\mathbb{R}^{n},d_{E})$ with the subset $U$, there is $\delta_{0}>0$ such that every $z\in U$ with $d_{E}(y,z)<\delta_{0}$ satisfies $f(z)\le f(y)$.

Let $u\in\mathbb{R}^{n}$ with $\lVert u\rVert=1$, let $\varepsilon>0$, and let $\delta$ be as in Twice Differentiability at a Point §twice-differentiable for this $\varepsilon$. Let $t\in\mathbb{R}$ with $0<t$ and $t<\delta$ and $t<\delta_{0}$, and put $h=tu$, so $\lVert h\rVert=t$ and $d_{E}(y,y+h)=\lVert h\rVert=t<\delta_{0}$; thus $y+h\in U$ and $f(y+h)-f(y)\le0$. Expanding by bilinearity, $p\cdot h=t\,p\cdot u$ and $h\cdot(Bh)=t^{2}\,u\cdot(Bu)$, so

using claim 6 of Properties of the Absolute Value in an Ordered Field. Dividing by the positive number $t$,

The bracket does not depend on $t$, so by The Archimedean Property of the Real Numbers the right-hand side is smaller than any prescribed positive real for $t$ small enough; hence $p\cdot u\le0$. Applying this with $-u$, which also has norm $1$ by claim 5 of Elementary Properties of the Euclidean Norm on $\mathbb{R}^n$, gives $-p\cdot u\le0$, so $p\cdot u=0$. If $p\neq0$, taking $u=p/\lVert p\rVert$ gives $0=p\cdot u=\lVert p\rVert>0$, a contradiction; hence $p=0$.

With $p=0$ the displayed inequality becomes $\tfrac{1}{2}t^{2}u\cdot(Bu)\le\varepsilon t^{2}$, so dividing by the positive number $t^{2}$ gives $u\cdot(Bu)\le2\varepsilon$. As $\varepsilon>0$ was arbitrary, $u\cdot(Bu)\le0$ for every $u$ with $\lVert u\rVert=1$. For general $z\neq0$, writing $z=\lVert z\rVert u$ with $u=z/\lVert z\rVert$ of norm $1$ gives $z\cdot(Bz)=\lVert z\rVert^{2}u\cdot(Bu)\le0$, and for $z=0$ both sides vanish. Since every entry of $0_{n}$ is $0$, Matrix-Vector Product gives $0_{n}z=0$ and hence $z\cdot(0_{n}z)=0$; therefore $z\cdot(Bz)\le z\cdot(0_{n}z)$ for every $z\in\mathbb{R}^{n}$, which is $B\preceq0_{n}$ by The Positive Semidefinite Ordering on Symmetric Matrices.