Proof of Rules for Derivatives of Functions on Open Subsets of ℝⁿ

Throughout, the arithmetic and order of $\mathbb{R}$ are used through The Structure of ℝ as a Field and an Ordered Ring (i), The Structure of ℝ as a Field and an Ordered Ring (ii), Arithmetic and Order Rules in the Real Numbers, Stated for ℝ (v), Arithmetic and Order Rules in the Real Numbers, Stated for ℝ (ix), Rules of Arithmetic and Order in an Ordered Field (iii) and The Quotient 1/x, Order Rules for Sums, and Nesting of Closed Intervals in ℝ (i); norms and distances in $\mathbb{R}^{n}$ through Properties of the Euclidean Norm on $\mathbb{R}^n$: Square, Distance, Vanishing, Coordinate Bounds, Homogeneity, Triangle Inequalities and Expansion of a Square (ii), Properties of the Euclidean Norm on $\mathbb{R}^n$: Square, Distance, Vanishing, Coordinate Bounds, Homogeneity, Triangle Inequalities and Expansion of a Square (v), Properties of the Euclidean Norm on $\mathbb{R}^n$: Square, Distance, Vanishing, Coordinate Bounds, Homogeneity, Triangle Inequalities and Expansion of a Square (vi) and Standard Basis Vectors of ℝ^q: Coordinates, Orthonormality and Unit Length (iii); and the metric spaces $(\mathbb{R}^{n},d_{E})$ and $(\mathbb{R},d_{\mathbb{R}})$ through $\mathbb{R}^n$ with the Euclidean Distance Is a Complete Metric Space with Coordinatewise Convergence (i) and The Absolute Value Metric on the Real Numbers (i).

In every clause we exhibit a derivative matrix in the sense of Differentiability at a Point for Functions Between Euclidean Spaces (i), or a derivative vector in the sense of Differentiability at a Point for Functions Between Euclidean Spaces (ii). That this matrix is $Df(a)$ is then The Derivative Is the Jacobian Matrix, and Is Unique (ii), and that this vector is $\nabla f(a)$ is The Derivative Is the Jacobian Matrix, and Is Unique (i); so it suffices in each case to bound the remainder of the candidate. Conversely, a function differentiable at $a$ has $Df(a)$, respectively $\nabla f(a)$, as a derivative matrix, respectively derivative vector, by The Derivative Is the Jacobian Matrix, and Is Unique (ii), respectively The Derivative Is the Jacobian Matrix, and Is Unique (i); this is how $A$, $B$, $u$ and $w$ are used below.

Constant and affine functions ((i), (ii)). Since $U$ is open (Open Subset of a Metric Space (i)), there is $\delta>0$ with $B(a,\delta)\subseteq U$, and every $h\in\mathbb{R}^{n}$ with $\lVert h\rVert<\delta$ has $d_{E}(a+h,a)=\lVert h\rVert<\delta$, hence $a+h\in U$. We show that for such $h$ the remainder vanishes identically, so that this one $\delta$ serves for every $\varepsilon>0$. For the constant function $f(x)=c$ and the candidate $0_{m\times n}$,

$$f(a+h)-f(a)-0_{m\times n}h=c-c-0_{m\times n}h=0,$$

because $0_{m\times n}h=(0\,0_{m\times n})h=0\,(0_{m\times n}h)=0$ by the rule $(sA)x=s(Ax)$ of Algebraic Rules for Matrix Products, Matrix-Vector Products and Transposes (v). For the constant function $\varphi(x)=c$ and the candidate $0\in\mathbb{R}^{n}$ the remainder is $-0\cdot h$, and $0\cdot h=(0\,0)\cdot h=0\,(0\cdot h)=0$ by The Dot Product on $\mathbb{R}^n$ Is Symmetric, Bilinear and Positive Definite (ii). For $f(x)=Bx+c$ and the candidate $B$, distributivity of the matrix-vector product (Algebraic Rules for Matrix Products, Matrix-Vector Products and Transposes (v)) gives

$$f(a+h)-f(a)-Bh=\bigl(B(a+h)+c\bigr)-(Ba+c)-Bh=0,$$

and for $\varphi(x)=v\cdot x+c$ and the candidate $v$, linearity of the dot product in its second argument (The Dot Product on $\mathbb{R}^n$ Is Symmetric, Bilinear and Positive Definite (iii)) gives

$$\varphi(a+h)-\varphi(a)-v\cdot h=\bigl(v\cdot(a+h)+c\bigr)-(v\cdot a+c)-v\cdot h=0 .$$

Coordinate functions ((iii)). For $x\in U$ we have $\pi_{i}(x)=x_{i}=e_{i}\cdot x$ (Standard Basis Vectors of ℝ^q: Coordinates, Orthonormality and Unit Length (i)), so $\pi_{i}$ is the affine function $x\mapsto v\cdot x+c$ with $v=e_{i}$ and $c=0$, and (ii) gives $\nabla\pi_{i}(a)=e_{i}$.

Sums and scalar multiples ((iv)). Let $A=Df(a)$ and $B=Dg(a)$ be the derivative matrices of $f$ and $g$ at $a$. Given $\varepsilon>0$, choose $\delta_{f}>0$ for $f$ and $\delta_{g}>0$ for $g$ by Differentiability at a Point for Functions Between Euclidean Spaces (i) with tolerance $\varepsilon/2$, and put $\delta=\min(\delta_{f},\delta_{g})$. For $\lVert h\rVert<\delta$ we have $a+h\in U$, and since $(A+B)h=Ah+Bh$ (Algebraic Rules for Matrix Products, Matrix-Vector Products and Transposes (v)) the remainder of $f+g$ for the candidate $A+B$ is the sum of the remainders of $f$ and $g$; the triangle inequality (Properties of the Euclidean Norm on $\mathbb{R}^n$: Square, Distance, Vanishing, Coordinate Bounds, Homogeneity, Triangle Inequalities and Expansion of a Square (vi)) gives

$$\bigl\lVert(f+g)(a+h)-(f+g)(a)-(A+B)h\bigr\rVert\le\lVert f(a+h)-f(a)-Ah\rVert+\lVert g(a+h)-g(a)-Bh\rVert\le\varepsilon\lVert h\rVert .$$

For $\lambda f$ and the candidate $\lambda A$, given $\varepsilon>0$ choose $\delta>0$ for $f$ with tolerance $\varepsilon/(|\lambda|+1)$. Since $(\lambda A)h=\lambda(Ah)$ (Algebraic Rules for Matrix Products, Matrix-Vector Products and Transposes (v)), the remainder of $\lambda f$ is $\lambda$ times that of $f$, and by homogeneity of the norm (Properties of the Euclidean Norm on $\mathbb{R}^n$: Square, Distance, Vanishing, Coordinate Bounds, Homogeneity, Triangle Inequalities and Expansion of a Square (v)), for $\lVert h\rVert<\delta$,

$$\bigl\lVert\lambda f(a+h)-\lambda f(a)-(\lambda A)h\bigr\rVert=|\lambda|\,\lVert f(a+h)-f(a)-Ah\rVert\le\frac{|\lambda|\,\varepsilon}{|\lambda|+1}\lVert h\rVert\le\varepsilon\lVert h\rVert .$$

Sums and scalar multiples, real-valued ((v)). The argument is the same, with $u=\nabla f(a)$ and $w=\nabla g(a)$, the candidates $u+w$ for $f+g$ and $\lambda u$ for $\lambda f$, the identities

$$(u+w)\cdot h=u\cdot h+w\cdot h,\qquad(\lambda u)\cdot h=\lambda(u\cdot h)$$

of The Dot Product on $\mathbb{R}^n$ Is Symmetric, Bilinear and Positive Definite (ii) in place of the matrix rules, and the triangle inequality $|\alpha+\beta|\le|\alpha|+|\beta|$ for real $\alpha,\beta$ (Rules of Arithmetic and Order in an Ordered Field (xi)) in place of that for the norm.

Products ((vi)). Let $u=\nabla f(a)$ and $w=\nabla g(a)$, and write

$$r_{f}(h)=f(a+h)-f(a)-u\cdot h,\qquad r_{g}(h)=g(a+h)-g(a)-w\cdot h$$

for the remainders of $f$ and $g$. Expanding, and using $\bigl(f(a)w+g(a)u\bigr)\cdot h=f(a)(w\cdot h)+g(a)(u\cdot h)$ (The Dot Product on $\mathbb{R}^n$ Is Symmetric, Bilinear and Positive Definite (ii)), the remainder of $fg$ for the candidate $f(a)w+g(a)u$ splits as

$$f(a+h)g(a+h)-f(a)g(a)-\bigl(f(a)w+g(a)u\bigr)\cdot h=r_{f}(h)\,g(a+h)+(u\cdot h)\bigl(g(a+h)-g(a)\bigr)+f(a)\,r_{g}(h).$$

The choices are made in this order. Given $\varepsilon>0$, first take $\delta_{0}>0$ from Differentiable Functions Are Continuous (iii) for $g$, so that

$$|g(a+h)-g(a)|\le\bigl(\lVert w\rVert+1\bigr)\lVert h\rVert\qquad(\lVert h\rVert<\delta_{0});$$

then take $\delta_{1}>0$ for $f$ by Differentiability at a Point for Functions Between Euclidean Spaces (ii) with tolerance $\varepsilon/(3(|g(a)|+1))$, and $\delta_{2}>0$ for $g$ with tolerance $\varepsilon/(3(|f(a)|+1))$; finally set

$$\delta=\min\Bigl(\delta_{0},\delta_{1},\delta_{2},\frac{1}{\lVert w\rVert+1},\frac{\varepsilon}{3(\lVert u\rVert+1)(\lVert w\rVert+1)}\Bigr).$$

Let $\lVert h\rVert<\delta$. Then $a+h\in U$, and $|g(a+h)-g(a)|<1$, so $|g(a+h)|\le|g(a)|+1$; moreover $|u\cdot h|\le\lVert u\rVert\,\lVert h\rVert$ by the Cauchy--Schwarz inequality (The Cauchy-Schwarz Inequality on $\mathbb{R}^n$ (i)). Hence the three terms are bounded by

$$|r_{f}(h)|\,|g(a+h)|\le\frac{\varepsilon}{3}\lVert h\rVert,\qquad|u\cdot h|\,|g(a+h)-g(a)|\le\lVert u\rVert\bigl(\lVert w\rVert+1\bigr)\lVert h\rVert\,\delta\le\frac{\varepsilon}{3}\lVert h\rVert,\qquad|f(a)|\,|r_{g}(h)|\le\frac{\varepsilon}{3}\lVert h\rVert,$$

and the remainder of $fg$ is at most $\varepsilon\lVert h\rVert$. So $fg$ is differentiable at $a$ with derivative vector $f(a)\nabla g(a)+g(a)\nabla f(a)$.

Reciprocals ((vii)). Let $w=\nabla g(a)$ and $b=g(a)$, so $b\neq0$; we show that $-w/b^{2}$ is a derivative vector of $1/g$ at $a$. With $r_{g}$ as above, and since $g(a+h)\neq0$ whenever $a+h\in U$, a direct computation gives

$$\frac{1}{g(a+h)}-\frac{1}{b}+\frac{w\cdot h}{b^{2}}=-\frac{r_{g}(h)}{b\,g(a+h)}+\frac{(w\cdot h)\bigl(g(a+h)-b\bigr)}{b^{2}\,g(a+h)} .$$

The choices are made in this order. Given $\varepsilon>0$, first take $\delta_{0}>0$ from Differentiable Functions Are Continuous (iii) for $g$ and put $\delta_{3}=\min\bigl(\delta_{0},|b|/(2(\lVert w\rVert+1))\bigr)$; for $\lVert h\rVert<\delta_{3}$ this gives $|g(a+h)-b|<|b|/2$, hence

$$|g(a+h)|\ge\frac{|b|}{2}.$$

Then take $\delta_{2}>0$ for $g$ by Differentiability at a Point for Functions Between Euclidean Spaces (ii) with tolerance $\varepsilon|b|^{2}/4$, and set

$$\delta=\min\Bigl(\delta_{2},\delta_{3},\frac{\varepsilon|b|^{3}}{4(\lVert w\rVert+1)^{2}}\Bigr).$$

For $\lVert h\rVert<\delta$, the first term is at most $(\varepsilon|b|^{2}/4)\lVert h\rVert/(|b|\cdot|b|/2)=\tfrac{\varepsilon}{2}\lVert h\rVert$, and, using $|w\cdot h|\le\lVert w\rVert\,\lVert h\rVert$ (The Cauchy-Schwarz Inequality on $\mathbb{R}^n$ (i)), the second is at most

$$\frac{\lVert w\rVert\,\lVert h\rVert\,\bigl(\lVert w\rVert+1\bigr)\lVert h\rVert}{|b|^{3}/2}\le\frac{2(\lVert w\rVert+1)^{2}\,\delta}{|b|^{3}}\lVert h\rVert\le\frac{\varepsilon}{2}\lVert h\rVert .$$

So the remainder is at most $\varepsilon\lVert h\rVert$, and $\nabla(1/g)(a)=-\nabla g(a)/g(a)^{2}$.

Quotients ((viii)). Pointwise $f/g=f\cdot(1/g)$. By (vii) the function $1/g$ is differentiable at $a$, and (vi) then gives

$$\nabla\Bigl(\frac{f}{g}\Bigr)(a)=f(a)\Bigl(-\frac{1}{g(a)^{2}}\nabla g(a)\Bigr)+\frac{1}{g(a)}\nabla f(a)=\frac{1}{g(a)^{2}}\bigl(g(a)\,\nabla f(a)-f(a)\,\nabla g(a)\bigr).$$