Proof of Continuous Partial Derivatives Imply Differentiability

Throughout, the arithmetic and order of $\mathbb{R}$ are used through The Structure of ℝ as a Field and an Ordered Ring (i), The Structure of ℝ as a Field and an Ordered Ring (ii), Arithmetic and Order Rules in the Real Numbers, Stated for ℝ (v), Arithmetic and Order Rules in the Real Numbers, Stated for ℝ (ix), Rules of Arithmetic and Order in an Ordered Field (iii) and The Quotient 1/x, Order Rules for Sums, and Nesting of Closed Intervals in ℝ (i); norms and distances in $\mathbb{R}^{n}$ through Properties of the Euclidean Norm on $\mathbb{R}^n$: Square, Distance, Vanishing, Coordinate Bounds, Homogeneity, Triangle Inequalities and Expansion of a Square (ii), Properties of the Euclidean Norm on $\mathbb{R}^n$: Square, Distance, Vanishing, Coordinate Bounds, Homogeneity, Triangle Inequalities and Expansion of a Square (v), Properties of the Euclidean Norm on $\mathbb{R}^n$: Square, Distance, Vanishing, Coordinate Bounds, Homogeneity, Triangle Inequalities and Expansion of a Square (vi) and Standard Basis Vectors of ℝ^q: Coordinates, Orthonormality and Unit Length (iii); and the metric spaces $(\mathbb{R}^{n},d_{E})$ and $(\mathbb{R},d_{\mathbb{R}})$ through $\mathbb{R}^n$ with the Euclidean Distance Is a Complete Metric Space with Coordinatewise Convergence (i) and The Absolute Value Metric on the Real Numbers (i). In particular $d_{E}(a,a+v)=\lVert v\rVert$ for every $v\in\mathbb{R}^{n}$, which is how membership of points in balls around $a$ is checked below.

We first prove (i). Let $u=\nabla f(a)$, so that $u_{i}=\partial_{i}f(a)$ for every $i\in[n]$ by Gradient of a Real-Valued Function on an Open Subset of $\mathbb{R}^n$ (i). Let $\varepsilon>0$. The choices are made in the following order: first a radius $r$, then the numbers $\delta_{1},\dots,\delta_{n}$, then $\delta$, and finally the increment $h$. Since $U$ is open (Open Subset of a Metric Space (i)), there is $r>0$ such that the open ball $B(a,r)$ is contained in $U$. For each $i\in[n]$, continuity of $\partial_{i}f$ at $a$ (Continuous Map Between Metric Spaces (i)) gives $\delta_{i}>0$ such that

$$\bigl|\partial_{i}f(w)-\partial_{i}f(a)\bigr|<\frac{\varepsilon}{n}\qquad\text{for all }w\in U\text{ with }d_{E}(a,w)<\delta_{i}.$$

Put $\delta=\min(r,\delta_{1},\dots,\delta_{n})>0$, and fix $h\in\mathbb{R}^{n}$ with $\lVert h\rVert<\delta$. We show that $a+h\in U$ and that $\bigl|f(a+h)-f(a)-u\cdot h\bigr|\le\varepsilon\lVert h\rVert$.

We move from $a$ to $a+h$ one coordinate at a time. Let $v_{0}=0$ and $v_{j}=v_{j-1}+h_{j}e_{j}$ for $j\in[n]$, so that $v_{j}=(h_{1},\dots,h_{j},0,\dots,0)$ and $v_{n}=h$. For $j\in[n]$ and $|s|\le|h_{j}|$, the point $v_{j-1}+se_{j}$ has coordinates $h_{1},\dots,h_{j-1},s,0,\dots,0$, each at most the corresponding $|h_{k}|$ in absolute value; hence $\lVert v_{j-1}+se_{j}\rVert\le\lVert h\rVert<\delta$ (Properties of the Euclidean Norm on $\mathbb{R}^n$: Square, Distance, Vanishing, Coordinate Bounds, Homogeneity, Triangle Inequalities and Expansion of a Square (i)), and so $a+v_{j-1}+se_{j}$ lies in $B(a,r)\subseteq U$ (Properties of the Euclidean Norm on $\mathbb{R}^n$: Square, Distance, Vanishing, Coordinate Bounds, Homogeneity, Triangle Inequalities and Expansion of a Square (ii)). Taking $j=n$ and $s=h_{n}$ shows in particular that $a+h\in U$. The increment of $f$ telescopes along these points:

$$f(a+h)-f(a)=\sum_{j=1}^{n}\bigl(f(a+v_{j})-f(a+v_{j-1})\bigr).$$

Each term of this sum is an increment of $f$ in a single coordinate direction, to which the one-variable mean value theorem applies. Fix $j\in[n]$. If $h_{j}=0$, the $j$th term is $0=h_{j}\,\partial_{j}f(x_{j})$ with $x_{j}=a+v_{j-1}$, and we put $\theta_{j}=0$. If $h_{j}\neq0$, let $J_{j}$ be the closed interval with endpoints $0$ and $h_{j}$, and define $\varphi_{j}:J_{j}\to\mathbb{R}$ by $\varphi_{j}(s)=f(a+v_{j-1}+se_{j})$, which makes sense by the previous paragraph. For $s,s'\in J_{j}$ with $s'\neq s$, writing $w=a+v_{j-1}+se_{j}$ and $\tau=s'-s$, we have

$$\frac{\varphi_{j}(s')-\varphi_{j}(s)}{s'-s}=\frac{f(w+\tau e_{j})-f(w)}{\tau},$$

so the existence of $\partial_{j}f(w)$ (Partial Derivative of a Real-Valued Function on an Open Subset of $\mathbb{R}^n$ (i)) shows that $\partial_{j}f(w)$ is a derivative of $\varphi_{j}$ at $s$ relative to $J_{j}$ (Derivative of a Real Function on an Interval (i)). Thus $\varphi_{j}$ is differentiable on $J_{j}$, hence continuous on $J_{j}$ (The Increment Form of the Derivative, and Differentiability Implies Continuity (ii)), and the mean value theorem (Mean Value Theorem on a Closed Real Interval and on Any Interval (i)) gives $\theta_{j}$ strictly between $0$ and $h_{j}$ with

$$f(a+v_{j})-f(a+v_{j-1})=h_{j}\,\partial_{j}f(x_{j}),\qquad x_{j}=a+v_{j-1}+\theta_{j}e_{j}$$

(for $h_{j}<0$ it is applied on $[h_{j},0]$; the identity is the same), and $\varphi_{j}'(\theta_{j})=\partial_{j}f(x_{j})$ by uniqueness of the derivative (Derivative of a Real Function on an Interval (ii)). In both cases $|\theta_{j}|\le|h_{j}|$, so the previous paragraph gives $x_{j}\in U$ and $d_{E}(a,x_{j})=\lVert v_{j-1}+\theta_{j}e_{j}\rVert<\delta\le\delta_{j}$ (Properties of the Euclidean Norm on $\mathbb{R}^n$: Square, Distance, Vanishing, Coordinate Bounds, Homogeneity, Triangle Inequalities and Expansion of a Square (ii)).

Now combine. Since $u\cdot h=\sum_{j=1}^{n}u_{j}h_{j}=\sum_{j=1}^{n}h_{j}\,\partial_{j}f(a)$, the telescoping identity and the mean value step give

$$\bigl|f(a+h)-f(a)-u\cdot h\bigr|=\Bigl|\sum_{j=1}^{n}h_{j}\bigl(\partial_{j}f(x_{j})-\partial_{j}f(a)\bigr)\Bigr|\le\sum_{j=1}^{n}|h_{j}|\,\frac{\varepsilon}{n}\le\varepsilon\lVert h\rVert,$$

where the first inequality uses the choice of $\delta_{j}$ and $d_{E}(a,x_{j})<\delta_{j}$, and the second uses $|h_{j}|\le\lVert h\rVert$ (Properties of the Euclidean Norm on $\mathbb{R}^n$: Square, Distance, Vanishing, Coordinate Bounds, Homogeneity, Triangle Inequalities and Expansion of a Square (iv)): each of the $n$ terms is at most $\lVert h\rVert\,\varepsilon/n$. As $\varepsilon>0$ was arbitrary and $a+h\in U$ for every $h$ with $\lVert h\rVert<\delta$, this is exactly the statement that $f$ is differentiable at $a$ with derivative vector $u=\nabla f(a)$ (Differentiability at a Point for Functions Between Euclidean Spaces (ii)).

Finally we prove (ii). By (i), each component $f_{k}$, $k\in[m]$, is differentiable at $a$ with derivative vector $\nabla f_{k}(a)=(\partial_{1}f_{k}(a),\dots,\partial_{n}f_{k}(a))$, which is the $k$th row of $Df(a)$ by Jacobian Matrix of a Function Between Euclidean Spaces (i). Since a map is differentiable with a given derivative matrix exactly when each component is differentiable with the corresponding row as derivative vector (The Derivative Is the Jacobian Matrix, and Is Unique (iii)), $f$ is differentiable at $a$ with derivative matrix $Df(a)$.