Proof of A Convex Function of Class $C^2$ has Positive Semidefinite Hessian

Write $\lVert\,\cdot\,\rVert$ for the Euclidean norm and $|\,\cdot\,|$ for the absolute value. Claims 1, 2 and 5 of Elementary Properties of the Euclidean Norm on $\mathbb{R}^n$ give $\lVert u\rVert^2=u\cdot u$, $d_E(u,v)=\lVert u-v\rVert$ for the Euclidean distance, and $\lVert\mu u\rVert=|\mu|\,\lVert u\rVert$; claims 2 and 3 of Properties of Finite Sums are used as additivity and homogeneity of finite sums. Order arithmetic is taken from Elementary Order Arithmetic in an Ordered Field (claim 4 sign reversal, claim 5 product of positive elements, claim 7 inverse of a positive element, claim 8 halving) and from Elementary Arithmetic in an Ordered Field (claim 3 translation, claim 5 multiplication by a nonnegative element), and the field axioms of the field $\mathbb{R}$ are used throughout. Two field identities are used repeatedly: $0\,a=0$, which follows from $0\,a+0\,a=(0+0)\,a=0\,a+0$ and claim 2 of Additive Cancellation and Elementary Additive Identities in a Field; and $(-a)\,b=-(a\,b)$, which follows from $a\,b+(-a)\,b=(a+(-a))\,b=0$ and claim 1 of that lemma.

Fix $x\in U$ and $z\in\mathbb{R}^n$, and write $M=D^2f(x)$, so that $M_{ij}=\dfrac{\partial^2f}{\partial x_i\,\partial x_j}(x)$ by Hessian Matrix of a C^2 Function. By The Positive Semidefinite Ordering on Symmetric Matrices the assertion to be proved is

Step 1: the left-hand side vanishes. By Matrix-Vector Product and homogeneity of finite sums with the factor $0$, the $i$th coordinate of $0_nz$ is $\sum_{j=1}^{n}0\,z_j=0\sum_{j=1}^{n}z_j=0$, so $0_nz$ is the origin of $\mathbb{R}^n$; the same computation applied to $z\cdot(0_nz)=\sum_{i=1}^{n}z_i\,0$ gives $z\cdot(0_nz)=0$. So it suffices to prove $0\le Q$, where $Q=z\cdot(Mz)$.

By claim 4 of Linearity of the Matrix-Vector Product and the Quadratic Form as a Double Sum,

If $z$ is the origin, then $z_i=0$ for every $i$ and the same computation gives $Q=0$, so the conclusion holds. Assume therefore that $z$ is not the origin. Then $0<\lVert z\rVert$, since $\lVert z\rVert=d_E(z,0_{\mathbb{R}^n})$ is nonnegative by Metric Space, a metric by Euclidean Distance is a Metric on $\mathbb{R}^n$, and vanishes only when $z$ is the origin; hence $N=\lVert z\rVert^2$ is positive as a product of positive elements.

Step 2: the two Taylor estimates. Let $\varepsilon\in\mathbb{R}$ satisfy $0<\varepsilon$, and let $\delta\in\mathbb{R}$ with $0<\delta$ be as in Second-Order Taylor Expansion with Peano Remainder for $f$, $x$ and $\varepsilon$: every $h\in\mathbb{R}^n$ with $\lVert h\rVert<\delta$ satisfies $x+h\in U$ and

Put $t=(\delta/2)\,\lVert z\rVert^{-1}$, which is positive, and consider the two points $h=t\,z$ and $h'=(-t)\,z$. By homogeneity of the norm and $|t|=|-t|=t$, which holds by Absolute Value in an Ordered Field and claim 2 of Properties of the Absolute Value in an Ordered Field,

using halving, so both estimates are available and $x+h,\,x+h'\in U$.

Write $G=\sum_{i=1}^{n}\frac{\partial f}{\partial x_i}(x)\,z_i$. Since $h_i=t\,z_i$ and $h'_i=(-t)\,z_i$, homogeneity of finite sums gives the linear terms $t\,G$ and $(-t)\,G=-(t\,G)$. For the quadratic terms, $h_ih_j=(t\,t)(z_iz_j)$ and, using $(-a)\,b=-(a\,b)$ twice together with claim 5 of Additive Cancellation and Elementary Additive Identities in a Field, also $h'_ih'_j=(t\,t)(z_iz_j)$; applying homogeneity of finite sums to the inner and then to the outer sum gives

Finally $\lVert h\rVert^{2}=(t\,\lVert z\rVert)(t\,\lVert z\rVert)=(t\,t)\,N$, and likewise for $h'$. Reading the two estimates through the two-sided form of claim 6 of Properties of the Absolute Value in an Ordered Field and keeping only the upper bounds,

Step 3: midpoint convexity. For each index $i$ we have $h_i+h'_i=t\,z_i+(-t)\,z_i=(t+(-t))\,z_i=0$, so $\tfrac12\,(x+h)+\tfrac12\,(x+h')=x$, the two points agreeing in every coordinate. Since $0<\tfrac12$ and $\tfrac12<1$ by halving, and $1-\tfrac12=\tfrac12$, convexity of $f$ applied to the points $x+h$ and $x+h'$ of $U$ gives

Adding the two estimates of step 2, multiplying the result by the nonnegative element $\tfrac12$, and using transitivity,

Translating by $-f(x)$ gives $0\le (t\,t)\bigl(\tfrac12\,Q+\varepsilon\,N\bigr)$, and multiplying by the positive element $(t\,t)^{-1}$ gives

Step 4: conclusion. The last inequality holds for every $\varepsilon\in\mathbb{R}$ with $0<\varepsilon$. Suppose, for contradiction, that $0\le Q$ fails. By the totality of the order of Ordered Field this means $Q\le 0$ and $Q\ne 0$, so $0<-Q$ by sign reversal. Choose

which is positive as a product of positive elements. Then $\varepsilon\,N=\tfrac12\cdot\tfrac12\,(-Q)$, so $\tfrac12\,Q+\varepsilon\,N=\tfrac12\,Q-\tfrac12\cdot\tfrac12\,Q=\tfrac12\cdot\tfrac12\,Q$, using $(-a)\,b=-(a\,b)$ and $\tfrac12=\tfrac12\cdot\tfrac12+\tfrac12\cdot\tfrac12$. Step 3 therefore gives $0\le\tfrac12\cdot\tfrac12\,Q$, and multiplying by the positive element $(\tfrac12\cdot\tfrac12)^{-1}$ gives $0\le Q$, contradicting $Q\ne0$ and $Q\le0$.

Hence $0\le Q$, that is, $z\cdot(0_nz)\le z\cdot(D^2f(x)z)$. As $z\in\mathbb{R}^n$ was arbitrary, $0_n\preceq D^2f(x)$.