Proof of Rules for Derivatives of Vector- and Matrix-Valued Maps

For $x:I\to\mathbb{R}^{q}$ and $A:I\to\mathcal{M}_{p\times q}(\mathbb{R})$, the component maps $x_{k}$ and the entry maps $A_{ij}$ are those of Derivative of a Vector-Valued or Matrix-Valued Map on an Interval (i) and Derivative of a Vector-Valued or Matrix-Valued Map on an Interval (ii), which are the component and entry maps of Component Maps and Entry Maps (i) and Component Maps and Entry Maps (ii); the component and entry maps of the maps formed below are formed in the same way. Continuity of Vector- and Matrix-Valued Maps: the Componentwise Criterion, Norms and Algebraic Operations is applied with $(X,d)=(\mathbb{R},d_{\mathbb{R}})$ and $S=I$.

Step 1 (components). By Derivative of a Vector-Valued or Matrix-Valued Map on an Interval (i), $x$ is differentiable at $t$ if and only if every $x_{k}$ ($k\in[q]$) is, and then $x'(t)$ is the vector with components $x_{k}'(t)$; by Derivative of a Vector-Valued or Matrix-Valued Map on an Interval (ii) the same holds for $A$ with entries. Since vectors and matrices are equal when their components, respectively entries, agree (Euclidean Space $\mathbb{R}^n$: n-Tuples of Real Numbers with Their Componentwise Operations (iii), Real Matrices: the Set of Real m x n Matrices with Their Entrywise Operations (iii)), to show that a map $X$ from $I$ into $\mathbb{R}^{q}$ (or into a space of matrices) is differentiable at $t$ with $X'(t)=V$ it suffices to show that each component (entry) map of $X$ is differentiable at $t$ with derivative the corresponding component (entry) of $V$.

Step 2 (finite sums of products). Let $\ell\in\mathbb{N}$ and let $j\mapsto g_{j}$ and $j\mapsto h_{j}$ be maps from $[\ell]$ to the set $\mathbb{R}^{I}$ of maps $I\to\mathbb{R}$, with every $g_{j}$ and $h_{j}$ differentiable at $t$. By Derivatives of Constants, the Identity, Sums, Scalar Multiples, Products, Reciprocals, Quotients and Powers (vi), each product $g_{j}h_{j}$ ($j\in[\ell]$) is differentiable at $t$ with derivative $g_{j}'(t)h_{j}(t)+g_{j}(t)h_{j}'(t)$. By Finite Sums of Real-Valued Maps: Continuity, Integral and Derivative (iii), with $J=[\ell]$, finite by Finite Sets: the Pigeonhole Principle, Uniqueness of the Length, Subsets, Unions, Products, Images, Bounded Sets of Natural Numbers, Extreme Elements, Sets of Maps, Finite Unions and Finite Choice (vii), and the family $j\mapsto g_{j}h_{j}$ (Maps Formed Pointwise: Constant Maps, Pairs of Values, Maps Applied to Values, and Families of Maps (iii), with the pointwise product of Pointwise Operations on Real-Valued Maps Exist and Are Unique (i)), the map $G_{\ell}:I\to\mathbb{R}$, $s\mapsto\sum_{j=1}^{\ell}g_{j}(s)h_{j}(s)$, is differentiable at $t$, and by Iterated Operations over Finite Sets: Singletons, Disjoint Unions, Reindexing, Products of Sets, Termwise Combination, Homomorphisms and Intervals (v) and Natural Numbers as Indices: ℕ ⊆ ℕ₀, 1 ∈ ℕ₀, and the Two Readings of an Operation from 1 to n (v),

$$G_{\ell}'(t)=\sum_{j=1}^{\ell}\bigl(g_{j}'(t)h_{j}(t)+g_{j}(t)h_{j}'(t)\bigr)=\sum_{j=1}^{\ell}g_{j}'(t)h_{j}(t)+\sum_{j=1}^{\ell}g_{j}(t)h_{j}'(t).$$

Proof of (i). If $x$ is differentiable at $t$, every $x_{k}$ is differentiable at $t$ (Step 1), hence continuous at $t$ relative to $I$ by The Increment Form of the Derivative, and Differentiability Implies Continuity (ii), and $x$ is continuous at $t$ by Continuity of Vector- and Matrix-Valued Maps: the Componentwise Criterion, Norms and Algebraic Operations (i). For $A$, use the entries and Continuity of Vector- and Matrix-Valued Maps: the Componentwise Criterion, Norms and Algebraic Operations (ii).

Proof of (ii) and (iii). By Euclidean Space $\mathbb{R}^n$: n-Tuples of Real Numbers with Their Componentwise Operations (iv), the $k$-th component maps of $x+y$ and $\varphi x$ are $x_{k}+y_{k}$ and the pointwise product $\varphi x_{k}$. By Derivatives of Constants, the Identity, Sums, Scalar Multiples, Products, Reciprocals, Quotients and Powers (iv) and Derivatives of Constants, the Identity, Sums, Scalar Multiples, Products, Reciprocals, Quotients and Powers (vi) they are differentiable at $t$ with derivatives $x_{k}'(t)+y_{k}'(t)$ and $\varphi'(t)x_{k}(t)+\varphi(t)x_{k}'(t)$, which are the $k$-th components of $x'(t)+y'(t)$ and $\varphi'(t)x(t)+\varphi(t)x'(t)$ (Euclidean Space $\mathbb{R}^n$: n-Tuples of Real Numbers with Their Componentwise Operations (iv)). Step 1 gives both clauses for vectors; for matrices use the entries and Real Matrices: the Set of Real m x n Matrices with Their Entrywise Operations (iv).

Proof of (iv). Let $\varphi:I\to\mathbb{R}$ be the constant map with value $-1$; it is differentiable at $t$ with $\varphi'(t)=0$ by Derivatives of Constants, the Identity, Sums, Scalar Multiples, Products, Reciprocals, Quotients and Powers (i). For $s\in I$, $-x(s)=(-1)\,x(s)=\varphi(s)\,x(s)$ and $x(s)-y(s)=x(s)+\bigl(-y(s)\bigr)$, as their components agree (Euclidean Space $\mathbb{R}^n$: n-Tuples of Real Numbers with Their Componentwise Operations (iv), Rules of Arithmetic in a Commutative Ring: Zero, Signs and Squares, and No Zero Divisors in a Field (ii), Euclidean Space $\mathbb{R}^n$: n-Tuples of Real Numbers with Their Componentwise Operations (iii)); so $-x=\varphi x$ and $x-y=x+(-y)$. By (iii), $-x$ is differentiable at $t$ with $(-x)'(t)=0\,x(t)+(-1)\,x'(t)=-x'(t)$; applied to $y$, this shows that $-y$ is differentiable at $t$ with derivative $-y'(t)$, and (ii) gives that $x-y$ is differentiable at $t$ with $(x-y)'(t)=x'(t)+\bigl(-y'(t)\bigr)=x'(t)-y'(t)$. Here $0\,x(t)+(-1)\,x'(t)=-x'(t)$ and $x'(t)+\bigl(-y'(t)\bigr)=x'(t)-y'(t)$ because, for every $k\in[q]$, the $k$-th components of the two sides are $0\cdot x_{k}(t)+(-1)\,x_{k}'(t)=-x_{k}'(t)$, respectively $x_{k}'(t)+\bigl(-y_{k}'(t)\bigr)=x_{k}'(t)-y_{k}'(t)$, by Euclidean Space $\mathbb{R}^n$: n-Tuples of Real Numbers with Their Componentwise Operations (iv), Rules of Arithmetic in a Commutative Ring: Zero, Signs and Squares, and No Zero Divisors in a Field (i) and Rules of Arithmetic in a Commutative Ring: Zero, Signs and Squares, and No Zero Divisors in a Field (ii), and vectors with the same components are equal by Euclidean Space $\mathbb{R}^n$: n-Tuples of Real Numbers with Their Componentwise Operations (iii). For matrices use the entries, Real Matrices: the Set of Real m x n Matrices with Their Entrywise Operations (iv) and Real Matrices: the Set of Real m x n Matrices with Their Entrywise Operations (iii).

Proof of (v), (vi) and (vii). By Product of Real Matrices (i), Matrix-Vector Product (i) and The Dot Product on $\mathbb{R}^n$ (i), for $s\in I$,

$$(A(s)B(s))_{ij}=\sum_{l=1}^{r}A_{il}(s)B_{lj}(s),\qquad(A(s)x(s))_{i}=\sum_{j=1}^{q}A_{ij}(s)x_{j}(s),\qquad x(s)\cdot y(s)=\sum_{k=1}^{q}x_{k}(s)y_{k}(s),$$

with $A$ and $B$ of the sizes of the respective clause. By Step 2, applied to the component families $k\mapsto x_{k}$ and $k\mapsto y_{k}$ of Component Maps of a Map into $\mathbb{R}^{q}$, Entry Maps of a Map into Matrices, and Maps Assembled from Them (i) and to a row $l\mapsto A_{il}$ and a column $l\mapsto B_{lj}$ of the entry families of Component Maps of a Map into $\mathbb{R}^{q}$, Entry Maps of a Map into Matrices, and Maps Assembled from Them (ii) (a row or column of a map $G$ on $[a]\times[b]$ being its composite with $l\mapsto(i,l)$, respectively $l\mapsto(l,j)$: Maps Formed Pointwise: Constant Maps, Pairs of Values, Maps Applied to Values, and Families of Maps (ii), applied to an identity map, Basic Properties of Functions: Equality, Composition, Identity, Inverse and Restriction (iii), and a constant map, Maps Formed Pointwise: Constant Maps, Pairs of Values, Maps Applied to Values, and Families of Maps (i); then Maps Formed Pointwise: Constant Maps, Pairs of Values, Maps Applied to Values, and Families of Maps (iii)), these maps are differentiable at $t$, with derivatives

$$\sum_{l=1}^{r}A_{il}'(t)B_{lj}(t)+\sum_{l=1}^{r}A_{il}(t)B_{lj}'(t),\qquad\sum_{j=1}^{q}A_{ij}'(t)x_{j}(t)+\sum_{j=1}^{q}A_{ij}(t)x_{j}'(t),\qquad\sum_{k=1}^{q}x_{k}'(t)y_{k}(t)+\sum_{k=1}^{q}x_{k}(t)y_{k}'(t).$$

Since $A_{il}'(t)$, $B_{lj}'(t)$ and $x_{k}'(t)$ are the entries and components of $A'(t)$, $B'(t)$ and $x'(t)$ (Step 1), by the same three definitions and by Real Matrices: the Set of Real m x n Matrices with Their Entrywise Operations (iv) and Euclidean Space $\mathbb{R}^n$: n-Tuples of Real Numbers with Their Componentwise Operations (iv) these are the $(i,j)$ entry of $A'(t)B(t)+A(t)B'(t)$, the $i$-th component of $A'(t)x(t)+A(t)x'(t)$, and $x'(t)\cdot y(t)+x(t)\cdot y'(t)$. Step 1 gives (v) and (vi); (vii) concerns a map into $\mathbb{R}$ and is the last statement itself.

Proof of (viii). By Transpose of a Real Matrix (i), $(A(s)^{\top})_{ji}=A_{ij}(s)$ for $s\in I$, so the $(j,i)$ entry map of $A^{\top}$ is $A_{ij}$, differentiable at $t$ with derivative $A_{ij}'(t)=(A'(t))_{ij}=(A'(t)^{\top})_{ji}$. Step 1 gives the clause.

Proof of (ix). The component maps of $s\mapsto b$ are the constant maps with values $b_{k}$, differentiable at $t$ with derivative $0$ by Derivatives of Constants, the Identity, Sums, Scalar Multiples, Products, Reciprocals, Quotients and Powers (i), and $0$ is every component of the origin of $\mathbb{R}^{q}$ (Euclidean Space $\mathbb{R}^n$: n-Tuples of Real Numbers with Their Componentwise Operations (v)); by Step 1 the derivative is $0$. For $s\mapsto C$ use the entries and Real Matrices: the Set of Real m x n Matrices with Their Entrywise Operations (v).

Proof of (x). We use that a zero matrix or zero vector annihilates finite sums: for $\ell\in\mathbb{N}$ and $a:[\ell]\to\mathbb{R}$, $\sum_{j=1}^{\ell}0\cdot a_{j}=0\cdot\sum_{j=1}^{\ell}a_{j}=0$ by Finite Sums in a Commutative Ring and in an Ordered Field: Distributivity, Differences, Telescoping, Constant Terms, Comparison and the Triangle Inequality (i) and Rules of Arithmetic in a Commutative Ring: Zero, Signs and Squares, and No Zero Divisors in a Field (i). Hence, by Matrix-Vector Product (i), The Dot Product on $\mathbb{R}^n$ (i) and Product of Real Matrices (i), for every $s\in I$ the vectors $0_{m\times q}x(s)$ and $A(s)0$ are the origin, the dot product $0\cdot x(s)$ of the origin with $x(s)$ is the real number $0$, and the matrices $0_{m\times p}A(s)$ and $(MA(s))0_{q\times n}$ are zero matrices, each component or entry being such a sum.

Let $x$ be differentiable at $t$. By (vi), applied to the constant map $s\mapsto L$ (with $m$ rows) and $x$, the map $s\mapsto Lx(s)$ is differentiable at $t$ with derivative $0_{m\times q}x(t)+Lx'(t)=Lx'(t)$. By (vii), applied to the constant map $s\mapsto z$ and $x$, the map $s\mapsto z\cdot x(s)$ is differentiable at $t$ with derivative $0\cdot x(t)+z\cdot x'(t)=z\cdot x'(t)$.

Let $A$ be differentiable at $t$. By (vi), applied to $A$ and the constant map $s\mapsto z$, the map $s\mapsto A(s)z$ is differentiable at $t$ with derivative $A'(t)z+A(t)0=A'(t)z$. By (v), applied to the constant map $s\mapsto M$ and $A$, the map $s\mapsto MA(s)$ (Maps Formed Pointwise: Constant Maps, Pairs of Values, Maps Applied to Values, and Families of Maps (iii), with the matrix product as the map of The Matrix Product, Matrix-Vector Product, Transpose, Trace, Identity Matrix and Standard Basis Vectors Exist and Are Unique (i)) is differentiable at $t$ with derivative $0_{m\times p}A(t)+MA'(t)=MA'(t)$; by (v) again, applied to this map and the constant map $s\mapsto N$, the map $s\mapsto(MA(s))N$ is differentiable at $t$ with derivative $(MA'(t))N+(MA(t))0_{q\times n}=(MA'(t))N$. The constant maps used here (Maps Formed Pointwise: Constant Maps, Pairs of Values, Maps Applied to Values, and Families of Maps (i)) are differentiable at $t$ with zero derivatives by (ix).

Proof of (xi). Since $\psi(\sigma)\in I$ for every $\sigma\in J$, $\psi$ is also a map $J\to I$, so $x\circ\psi$ and each $x_{k}\circ\psi$ are maps on $J$. For $\sigma\in J$ and $k\in[q]$, the $k$-th component of $(x\circ\psi)(\sigma)=x(\psi(\sigma))$ is $x_{k}(\psi(\sigma))=(x_{k}\circ\psi)(\sigma)$, so the $k$-th component map of $x\circ\psi$ is $x_{k}\circ\psi$ (Basic Properties of Functions: Equality, Composition, Identity, Inverse and Restriction (ii), Component Maps and Entry Maps (i)). By Step 1, $x_{k}$ is differentiable at $\psi(s_{0})$ with derivative $x_{k}'(\psi(s_{0}))$, the $k$-th component of $x'(\psi(s_{0}))$. By Chain Rule for Real Functions on Intervals (i), with $J$, $I$, $\psi$ and $x_{k}$ in the roles of $I$, $J$, $f$ and $g$ there, $x_{k}\circ\psi$ is differentiable at $s_{0}$ relative to $J$ with derivative $x_{k}'(\psi(s_{0}))\,\psi'(s_{0})$, the $k$-th component of $\psi'(s_{0})\,x'(\psi(s_{0}))$ by Euclidean Space $\mathbb{R}^n$: n-Tuples of Real Numbers with Their Componentwise Operations (iv). Step 1, applied on $J$ (Derivative of a Vector-Valued or Matrix-Valued Map on an Interval (i) is stated for every interval containing two distinct points), gives the claim; for $A$ use the entries, Derivative of a Vector-Valued or Matrix-Valued Map on an Interval (ii) and Real Matrices: the Set of Real m x n Matrices with Their Entrywise Operations (iv).