Proof of Equality of Mixed Second Partial Derivatives and Symmetry of the Hessian

Let $|\cdot|$ be the absolute value. Claim numbers refer to Elementary Order Arithmetic in an Ordered Field and to Properties of the Absolute Value in an Ordered Field as indicated. For $z\in\mathbb{R}^{n}$, an index $m$ and $u\in\mathbb{R}$, write $z[m{:}u]$ for the point of $\mathbb{R}^{n}$ whose $m$th coordinate is $u$ and whose other coordinates are those of $z$.

If $i=j$ the two sides of claim 1 are the same expression, so assume $i\ne j$ throughout Steps 1 to 5.

Step 0 (squares are strictly monotone on nonnegative reals). If $0\le\alpha$, $0\le\beta$ and $\alpha<\beta$, then $0<\beta$ by claim 2, so $\beta\alpha<\beta\beta$ by claim 10, while $\alpha\alpha\le\beta\alpha$ (an equality when $\alpha=0$, and claim 10 with multiplier $\alpha$ when $0<\alpha$); claim 2 gives $\alpha^{2}<\beta^{2}$. Consequently, if $0\le\alpha$, $0\le\beta$ and $\alpha^{2}<\beta^{2}$, then $\alpha<\beta$: otherwise $\beta\le\alpha$, and either $\beta=\alpha$, giving $\alpha^{2}=\beta^{2}$, or $\beta<\alpha$, giving $\beta^{2}<\alpha^{2}$; both contradict $\alpha^{2}<\beta^{2}$.

Step 1 (a common radius). Since $x\in U$ and $U$ is open, there is $r$ with $0<r$ such that every point of $\mathbb{R}^{n}$ whose Euclidean distance to $x$ is less than $r$ lies in $U$. Put $\rho=r\cdot2^{-1}$, so $0<\rho$ and $\rho<r$ by claim 8.

Let $s,t\in\mathbb{R}$ satisfy $|s|<\rho$ and $|t|<\rho$, and let $w=x[i{:}x_{i}+s][j{:}x_{j}+t]$. Since $i\ne j$, the point $w$ agrees with $x$ in every coordinate other than the $i$th and $j$th, and differs there by $s$ and by $t$; so by the definition of the Euclidean distance the distance from $w$ to $x$ is the nonnegative number whose square is $s^{2}+t^{2}$. Now $s^{2}=|s|^{2}<\rho^{2}$ and $t^{2}=|t|^{2}<\rho^{2}$ by claim 4 of Properties of the Absolute Value in an Ordered Field and Step 0, so $s^{2}+t^{2}<\rho^{2}+\rho^{2}$ by claim 3. Also $\rho^{2}+\rho^{2}=r^{2}\cdot2^{-1}$, which is less than $r^{2}$ by claim 8 applied to $0<r^{2}$ (positive by claim 5). Hence the square of the distance is less than $r^{2}$, so by Step 0 the distance is less than $r$ and $w\in U$. Taking $t=0$ or $s=0$ shows likewise that $x[i{:}x_{i}+s]\in U$ and $x[j{:}x_{j}+t]\in U$.

Write $J_{i}=(x_{i}-\rho,\;x_{i}+\rho)$ and $J_{j}=(x_{j}-\rho,\;x_{j}+\rho)$ for the open intervals; by claim 9 of Properties of the Absolute Value in an Ordered Field and claim 1, $u\in J_{i}$ holds exactly when $|u-x_{i}|<\rho$, and similarly for $J_{j}$.

Step 2 (slices are differentiable). Let $z\in U$ and let $m$ be an index. Real-valued functions on $U$ are regarded as maps into $\mathbb{R}^1$ with themselves as single coordinate function, as in clause 3 of the definition of class $C^k$. Since $f$ is of class $C^{2}$ on $U$, clause 2 of that definition applied with $k=1$ shows that $f$ is of class $C^{1}$ on $U$, so by clause 1 the partial derivative $\partial f/\partial x_{m}$ exists at every point of $U$; the same clause 2 says that each $\partial f/\partial x_{m}$ is itself of class $C^1$ on $U$, so the same holds with $\partial f/\partial x_{m}$ in place of $f$.

Let $V\subseteq\mathbb{R}$ be an open interval such that $z[m{:}u]\in U$ for every $u\in V$, and let $G:V\to\mathbb{R}$ be given by $G(u)=f(z[m{:}u])$. We claim $G$ is differentiable at every $u_{0}\in V$ with $G'(u_{0})=\frac{\partial f}{\partial x_{m}}(z[m{:}u_{0}])$.

Indeed, apply claims 1 and 2 of Slice Function and the Partial Derivative to $f$ at the point $a=z[m{:}u_{0}]$ of $U$ in the $m$th variable: it supplies $\rho_{0}$ with $0<\rho_{0}$ and identifies the slice function $g_{0}$ on $I_{0}=(u_{0}-\rho_{0},u_{0}+\rho_{0})$, given by $g_{0}(u)=f(a[m{:}u])=f(z[m{:}u])$, as differentiable at $u_{0}$ with $g_{0}'(u_{0})=\frac{\partial f}{\partial x_{m}}(a)$, the partial derivative existing as noted above. The functions $G$ and $g_{0}$ agree on $V\cap I_{0}$. By An Open Interval is an Interval All of Whose Points Are Interior both $V$ and $I_{0}$ are intervals all of whose points are interior, so there is $\eta$ with $0<\eta$ such that every $u$ with $|u-u_{0}|<\eta$ lies in $V\cap I_{0}$: take $\eta$ to be the least, by claim 9, of $\rho_{0}$ and of a radius witnessing that $u_{0}$ is interior to $V$. The defining condition of differentiability at $u_{0}$ involves only values at points $u_{0}+k$ with $|k|$ smaller than the chosen $\delta$, and shrinking $\delta$ to be at most $\eta$ (claim 9) confines those points to $V\cap I_{0}$, where $G$ and $g_{0}$ agree. Hence the same value satisfies the condition for $G$ on $V$ as for $g_{0}$ on $I_{0}$, which proves the claim. The identical argument applies with $\partial f/\partial x_{m'}$ in place of $f$ for any index $m'$, since it too has partial derivatives everywhere on $U$.

Step 3 (the double difference, computed in one order). Fix $s,t$ with $0<|s|<\rho$ and $0<|t|<\rho$, and set

Put $y=x[j{:}x_{j}+t]$ and define $\varphi:J_{i}\to\mathbb{R}$ by

Both arguments lie in $U$ for $u\in J_{i}$, by Step 1. By Step 2 each term is differentiable at every $u\in J_{i}$, so by Derivative of a Sum and of a Difference $\varphi$ is differentiable at every $u\in J_{i}$ with

Evaluating, $\varphi(x_{i}+s)-\varphi(x_{i})=\Delta$, and both $x_{i}$ and $x_{i}+s$ lie in $J_{i}$.

Apply Mean Value Theorem on an Open Interval to $\varphi$ on $J_{i}$, with the two points $x_{i}$ and $x_{i}+s$ taken in increasing order: there is $\bar u$ strictly between them with $\Delta=\varphi'(\bar u)\,s$. In either sign case $|\bar u-x_{i}|<|s|$, by claim 1 and claim 9 of Properties of the Absolute Value in an Ordered Field.

Now put $z=x[i{:}\bar u]$ and define $\psi:J_{j}\to\mathbb{R}$ by $\psi(v)=\frac{\partial f}{\partial x_{i}}\bigl(z[j{:}v]\bigr)$; the arguments lie in $U$ by Step 1. By Step 2, applied with $\partial f/\partial x_{i}$ in place of $f$ and the index $j$, $\psi$ is differentiable at every $v\in J_{j}$ with $\psi'(v)=\frac{\partial^{2}f}{\partial x_{j}\,\partial x_{i}}\bigl(z[j{:}v]\bigr)$, in the second-order partial derivative notation of Hessian Matrix of a C^2 Function, that is the iterated partial derivative of clause 4 of C^k Maps on a Euclidean Open Set. Since $y[i{:}\bar u]=z[j{:}x_{j}+t]$ and $x[i{:}\bar u]=z[j{:}x_{j}]$, we have $\varphi'(\bar u)=\psi(x_{j}+t)-\psi(x_{j})$, and Mean Value Theorem on an Open Interval gives $\bar v$ strictly between $x_{j}$ and $x_{j}+t$, so with $|\bar v-x_{j}|<|t|$, such that $\varphi'(\bar u)=\psi'(\bar v)\,t$. Therefore, with $w=x[i{:}\bar u][j{:}\bar v]$,

Step 4 (the other order). Regroup the same $\Delta$ as

which is the expression of Step 3 with the roles of the indices $i$ and $j$, and of $s$ and $t$, interchanged. Step 3 applied in that form produces $\tilde u$ with $|\tilde u-x_{j}|<|t|$ and $\tilde v$ with $|\tilde v-x_{i}|<|s|$ such that, with $w'=x[j{:}\tilde u][i{:}\tilde v]$,

Since $s\,t=t\,s$ is nonzero, multiplying both expressions by $(st)^{-1}$ gives

Moreover $w$ and $w'$ each differ from $x$ only in the $i$th and $j$th coordinates, by amounts of absolute value less than $|s|$ and $|t|$ respectively; so, exactly as in Step 1, the Euclidean distance from each of $w,w'$ to $x$ has square less than $|s|^{2}+|t|^{2}$.

Step 5 (passing to the limit). Write $P=\frac{\partial^{2}f}{\partial x_{j}\,\partial x_{i}}$ and $Q=\frac{\partial^{2}f}{\partial x_{i}\,\partial x_{j}}$, functions on $U$. By clause 2 of C^k Maps on a Euclidean Open Set applied with $k=1$, each of $\partial f/\partial x_{i}$ and $\partial f/\partial x_{j}$ is of class $C^{1}$ on $U$, so $P$ and $Q$ are continuous at every point of $U$.

Let $\varepsilon\in\mathbb{R}$ with $0<\varepsilon$, and put $\eta_{\varepsilon}=\varepsilon\cdot2^{-1}$, positive by claim 8. Continuity of $P$ and of $Q$ at $x$ gives $\theta$ with $0<\theta$, obtained as the least of the two radii by claim 9, such that every $z\in U$ whose Euclidean distance to $x$ is less than $\theta$ satisfies $|P(z)-P(x)|<\eta_{\varepsilon}$ and $|Q(z)-Q(x)|<\eta_{\varepsilon}$.

Choose $\sigma$ to be the least of $\rho$ and $\theta\cdot2^{-1}$ by claim 9, and put $s=t=\sigma\cdot2^{-1}$, so $0<s<\rho$ and $0<|s|=s$, $0<|t|=t$. Then $|s|^{2}+|t|^{2}=s^{2}+s^{2}$, and $s<\theta\cdot2^{-1}$ gives $s^{2}<\theta^{2}\cdot4^{-1}$ by Step 0, so $s^{2}+s^{2}<\theta^{2}\cdot2^{-1}<\theta^{2}$ by claims 3 and 8. By Step 4 and Step 0, the distances from $w$ and from $w'$ to $x$ are therefore less than $\theta$, and both points lie in $U$ by Step 1.

Hence $|P(w)-P(x)|<\eta_{\varepsilon}$ and $|Q(w')-Q(x)|<\eta_{\varepsilon}$, while $P(w)=Q(w')$ by Step 4. Using claim 5 of Properties of the Absolute Value in an Ordered Field twice,

and by claim 2 of Properties of the Absolute Value in an Ordered Field the first summand equals $|P(w)-P(x)|$; so by claim 3, $|P(x)-Q(x)|<\eta_{\varepsilon}+\eta_{\varepsilon}=\varepsilon$.

This holds for every $\varepsilon$ with $0<\varepsilon$. If $P(x)\ne Q(x)$ then $|P(x)-Q(x)|$ is positive by claim 1 of Properties of the Absolute Value in an Ordered Field, and taking $\varepsilon=|P(x)-Q(x)|$ gives $|P(x)-Q(x)|<|P(x)-Q(x)|$, which is impossible. Hence $P(x)=Q(x)$, which is claim 1.

Step 6 (claim 2). By the definition of the Hessian, $\bigl(D^{2}f(x)\bigr)_{ij}=\frac{\partial^{2}f}{\partial x_{i}\,\partial x_{j}}(x)$ for all $i,j$. By claim 1 this equals $\frac{\partial^{2}f}{\partial x_{j}\,\partial x_{i}}(x)=\bigl(D^{2}f(x)\bigr)_{ji}$. By the definition of the transpose, a square real matrix $A$ with $A_{ij}=A_{ji}$ for all $i,j$ satisfies $A=A^{\top}$ and so is symmetric; and $D^{2}f(x)$ is a square real $n\times n$ matrix. Hence $D^{2}f(x)$ belongs to $\mathcal{S}(n)$, the set of symmetric real $n\times n$ matrices.