Proof of The Derivative Is the Jacobian Matrix, and Is Unique

Throughout, the arithmetic and order of $\mathbb{R}$ are used through The Structure of ℝ as a Field and an Ordered Ring (i), The Structure of ℝ as a Field and an Ordered Ring (ii), Arithmetic and Order Rules in the Real Numbers, Stated for ℝ (v) and Arithmetic and Order Rules in the Real Numbers, Stated for ℝ (ix), and norms in $\mathbb{R}^{n}$ through Properties of the Euclidean Norm on $\mathbb{R}^n$: Square, Distance, Vanishing, Coordinate Bounds, Homogeneity, Triangle Inequalities and Expansion of a Square (ii), Properties of the Euclidean Norm on $\mathbb{R}^n$: Square, Distance, Vanishing, Coordinate Bounds, Homogeneity, Triangle Inequalities and Expansion of a Square (v) and Standard Basis Vectors of ℝ^q: Coordinates, Orthonormality and Unit Length (iii). We prove (i) first, then (iii), and deduce (ii) from the two. For $A\in\mathcal{M}_{m\times n}(\mathbb{R})$ and $k\in[m]$ we write $A_{k}=(A_{k1},\dots,A_{kn})\in\mathbb{R}^{n}$ for the $k$-th row of $A$.

Proof of (i). Fix $i\in[n]$ and a real $\varepsilon>0$. By the differentiability condition Differentiability at a Point for Functions Between Euclidean Spaces (ii) with tolerance $\varepsilon/2$ there is a real $\delta>0$ such that every $h\in\mathbb{R}^{n}$ with $\lVert h\rVert<\delta$ satisfies $a+h\in U$ and $|f(a+h)-f(a)-w\cdot h|\le(\varepsilon/2)\lVert h\rVert$. Let $t$ be real with $0<|t|<\delta$ and take $h=te_{i}$. Since $\lVert te_{i}\rVert=|t|\,\lVert e_{i}\rVert=|t|<\delta$, the point $a+te_{i}$ lies in $U$ and

$$\bigl|f(a+te_{i})-f(a)-w\cdot(te_{i})\bigr|\le\frac{\varepsilon}{2}\,|t| .$$

The linear term is $w\cdot(te_{i})=t\,(w\cdot e_{i})=t\,w_{i}$, by linearity of the dot product in its second argument, The Dot Product on $\mathbb{R}^n$ Is Symmetric, Bilinear and Positive Definite (iii) (using The Dot Product on $\mathbb{R}^n$ Is Symmetric, Bilinear and Positive Definite (i) and Standard Basis Vectors of ℝ^q: Coordinates, Orthonormality and Unit Length (i), $w\cdot e_{i}=e_{i}\cdot w=w_{i}$). Dividing the last estimate by $|t|>0$ therefore gives

$$\Bigl|\frac{f(a+te_{i})-f(a)}{t}-w_{i}\Bigr|\le\frac{\varepsilon}{2}<\varepsilon .$$

As $\varepsilon>0$ was arbitrary, $w_{i}$ is a partial derivative of $f$ with respect to the $i$-th variable at $a$ in the sense of Partial Derivative of a Real-Valued Function on an Open Subset of $\mathbb{R}^n$ (i). So $\partial_{i}f(a)$ exists (Partial Derivative of a Real-Valued Function on an Open Subset of $\mathbb{R}^n$ (ii)), and $\partial_{i}f(a)$ and $w_{i}$ are both partial derivatives of $f$ with respect to the $i$-th variable at $a$, hence equal by Uniqueness of the Partial Derivative on an Open Subset of $\mathbb{R}^n$ (i): $\partial_{i}f(a)=w_{i}$.

This holds for every $i\in[n]$, so the gradient $\nabla f(a)$ is defined by Gradient of a Real-Valued Function on an Open Subset of $\mathbb{R}^n$ (i), and its $i$-th component is $w_{i}$ for every $i$. Points of $\mathbb{R}^{n}$ with the same components are equal (Euclidean Space $\mathbb{R}^n$: n-Tuples of Real Numbers with Their Componentwise Operations (iii)), hence $w=\nabla f(a)$. Finally, if $f$ is also differentiable at $a$ with derivative vector $v\in\mathbb{R}^{n}$, the same argument applied to $v$ gives $v=\nabla f(a)=w$.

Proof of (iii). For $h\in\mathbb{R}^{n}$ with $a+h\in U$ put $r(h)=f(a+h)-f(a)-Ah\in\mathbb{R}^{m}$. Computing componentwise (differences, component maps, the matrix-vector product and the dot product are as in Euclidean Space $\mathbb{R}^n$: n-Tuples of Real Numbers with Their Componentwise Operations (iv), Component Maps and Entry Maps (i), Matrix-Vector Product (i) and The Dot Product on $\mathbb{R}^n$ (i)), for every $k\in[m]$

$$r(h)_{k}=f_{k}(a+h)-f_{k}(a)-A_{k}\cdot h .$$

Suppose first that $f$ is differentiable at $a$ with derivative matrix $A$, and let $\varepsilon>0$. By Differentiability at a Point for Functions Between Euclidean Spaces (i) there is a real $\delta>0$ such that every $h$ with $\lVert h\rVert<\delta$ satisfies $a+h\in U$ and $\lVert r(h)\rVert\le\varepsilon\lVert h\rVert$. For such $h$ and every $k\in[m]$, a coordinate is bounded by the norm (Properties of the Euclidean Norm on $\mathbb{R}^n$: Square, Distance, Vanishing, Coordinate Bounds, Homogeneity, Triangle Inequalities and Expansion of a Square (iv)), so

$$\bigl|f_{k}(a+h)-f_{k}(a)-A_{k}\cdot h\bigr|=|r(h)_{k}|\le\lVert r(h)\rVert\le\varepsilon\lVert h\rVert .$$

This is Differentiability at a Point for Functions Between Euclidean Spaces (ii) for $f_{k}$ with derivative vector $A_{k}$.

Conversely, suppose that every $f_{k}$ is differentiable at $a$ with derivative vector $A_{k}$, and let $\varepsilon>0$. For each $k\in[m]$, Differentiability at a Point for Functions Between Euclidean Spaces (ii) with tolerance $\varepsilon/m$ gives a real $\delta_{k}>0$ such that every $h$ with $\lVert h\rVert<\delta_{k}$ satisfies $a+h\in U$ and $|r(h)_{k}|\le(\varepsilon/m)\lVert h\rVert$; having chosen these, put $\delta=\min_{k\in[m]}\delta_{k}>0$. Let $\lVert h\rVert<\delta$. Then $a+h\in U$ (already from $k=1$), and $|r(h)_{k}|\le(\varepsilon/m)\lVert h\rVert$ for every $k\in[m]$, so the second part of Properties of the Euclidean Norm on $\mathbb{R}^n$: Square, Distance, Vanishing, Coordinate Bounds, Homogeneity, Triangle Inequalities and Expansion of a Square (iv) in $\mathbb{R}^{m}$, with $s=(\varepsilon/m)\lVert h\rVert$, gives

$$\lVert f(a+h)-f(a)-Ah\rVert=\lVert r(h)\rVert\le m\cdot\frac{\varepsilon}{m}\,\lVert h\rVert=\varepsilon\lVert h\rVert .$$

This is Differentiability at a Point for Functions Between Euclidean Spaces (i).

Proof of (ii). By (iii), each component map $f_{k}$ is differentiable at $a$ with derivative vector $A_{k}$, and then (i), applied to $f_{k}$, shows that $\partial_{i}f_{k}(a)$ exists and equals $(A_{k})_{i}=A_{ki}$ for every $i\in[n]$. Hence the Jacobian matrix $Df(a)$ is defined by Jacobian Matrix of a Function Between Euclidean Spaces (i), with $(Df(a))_{ki}=A_{ki}$ for all $k\in[m]$ and $i\in[n]$; matrices with the same entries are equal (Real Matrices: the Set of Real m x n Matrices with Their Entrywise Operations (iii)), so $A=Df(a)$. If $f$ is also differentiable at $a$ with derivative matrix $B$, the same argument applied to $B$ gives $B=Df(a)=A$.