Proof of The State-Transition Matrix of a Constant Coefficient Matrix Is a Matrix Exponential

Computations with real numbers use The Real Numbers as a Type. By Basic Facts about Intervals of the Real Line (v), $[a,b]$ is an interval containing the two distinct points $a$ and $b$; derivatives of maps on $[a,b]$ are taken relative to $[a,b]$, as in Derivative of a Real Function on an Interval (ii) and Derivative of a Vector-Valued or Matrix-Valued Map on an Interval (ii), and continuity on $[a,b]$ is as in Continuity of Vector- and Matrix-Valued Maps: the Componentwise Criterion, Norms and Algebraic Operations. $(\mathcal{M}_{n}(\mathbb{R}),d_{n})$ is a metric space by The Real Matrices with the Operator-Norm Distance Form a Complete Metric Space with Entrywise Convergence (i), with $d_{n}(X,Y)=\lVert X-Y\rVert$ (Euclidean Space and Real Matrices: Standing Notation (vi)).

Fix $s\in[a,b]$. Let $K=\lVert M\rVert$, so $K\ge0$ by Rules for the Operator Norm: Bound, Least Constant, Definiteness, Homogeneity, Triangle Inequality and Products (i), and let $\Phi:[a,b]\to\mathcal{M}_{n}(\mathbb{R})$ be the map $t\mapsto\Phi_{A}(t,s)$. By The State-Transition Matrix of a Linear System (ii) and The State-Transition Matrix of a Linear System (iii), $\Phi$ is differentiable, $\Phi'(t)=A(t)\Phi(t)=M\Phi(t)$ for every $t\in[a,b]$, and $\Phi(s)=I_{n}$.

Step 1: the maps $T_{N}$. For $N\in\mathbb{N}_{0}$ let $\psi_{N}:[a,b]\to\mathbb{R}$ be the map $r\mapsto(r-s)^{N}/N!$ of Shifted Power Maps, and Integrating a Power Bound on the Derivative of a Matrix-Valued Map. By Shifted Power Maps, and Integrating a Power Bound on the Derivative of a Matrix-Valued Map (i), $\psi_{0}(r)=1$ for every $r\in[a,b]$, $\psi_{N}(s)=0$ for every $N\in\mathbb{N}$, and every $\psi_{N}$ is differentiable on $[a,b]$, with $\psi_{N+1}'(r)=\psi_{N}(r)$ for every $r\in[a,b]$.

For $t\in[a,b]$ and $N\in\mathbb{N}$ let $T_{N}(t)=S_{N}\bigl((t-s)M\bigr)$ be the matrix of The Exponential Series of a Real Square Matrix: Convergence, a Norm Bound and the Exponential Map with $(t-s)M$ in place of $M$, and regard $T_{N}$ as the map $[a,b]\to\mathcal{M}_{n}(\mathbb{R})$, $t\mapsto T_{N}(t)$: the composite (Maps Formed Pointwise: Constant Maps, Pairs of Values, Maps Applied to Values, and Families of Maps (iii)) of the map $[a,b]\to\mathcal{M}_{n}(\mathbb{R})$, $t\mapsto(t-s)M$ (Values, Restriction and Equality of a Map, and the Map $u\mapsto t(u)$ as a Defined Set Symbol (v)), with the map $X\mapsto S_{N}(X)$ of The Exponential Series of a Real Square Matrix: Convergence, a Norm Bound and the Exponential Map (i). For every $t\in[a,b]$, $(T_{N}(t))_{N\in\mathbb{N}}$ is the sequence of The Exponential Series of a Real Square Matrix: Convergence, a Norm Bound and the Exponential Map (i) with $(t-s)M$ in place of $M$, and, by The Exponential of a Real Square Matrix (ii),

$$(T_{N}(t))_{N\in\mathbb{N}}\ \text{converges to}\ e^{(t-s)M}\ \text{in}\ (\mathcal{M}_{n}(\mathbb{R}),d_{n}).$$

For $N\in\mathbb{N}$, $N+1\in\mathbb{N}$ and $(N+1)+1\in\mathbb{N}$ by Natural Numbers Are the Successors in Omega: One Is Least and Not a Successor of a Natural Number, the Successor Is Injective, and N Is Closed under Addition and Multiplication (ii) and Natural Numbers Are the Successors in Omega: One Is Least and Not a Successor of a Natural Number, the Successor Is Injective, and N Is Closed under Addition and Multiplication (vi), so $T_{N+1}$ and $T_{(N+1)+1}$ are among these maps. By The Exponential Series of a Real Square Matrix: Convergence, a Norm Bound and the Exponential Map (iii) and Powers of a Square Matrix (iv), $T_{1}(t)=I_{n}$ and, for $N\in\mathbb{N}$,

$$T_{N+1}(t)=T_{N}(t)+\frac{1}{N!}\bigl((t-s)M\bigr)^{N}=T_{N}(t)+\frac{1}{N!}\bigl((t-s)^{N}M^{N}\bigr)=T_{N}(t)+\psi_{N}(t)\,M^{N},$$

the last equality entrywise by Real Matrices: the Set of Real m x n Matrices with Their Entrywise Operations (iv). As $\psi_{N}(s)=0$ for $N\in\mathbb{N}$ and $0\,X=0_{n}$ for $X\in\mathcal{M}_{n}(\mathbb{R})$, induction on $N$ (Arithmetic and Order of the Natural Numbers (xiii), and Additive Laws for Vectors in $\mathbb{R}^q$ and for Real $p\times q$ Matrices: Group Laws and Rules for Differences (i)) gives $T_{N}(s)=I_{n}$ for every $N\in\mathbb{N}$.

We show by induction on $N\in\mathbb{N}$ (Arithmetic and Order of the Natural Numbers (xiii)) that $T_{N}$ and $T_{N+1}$ are differentiable and $T_{N+1}'(t)=MT_{N}(t)$ for every $t\in[a,b]$. First, for every $N\in\mathbb{N}$ such that $T_{N}$ is differentiable at $t\in[a,b]$, the map $T_{N+1}$ is the pointwise sum of $T_{N}$ and the product of $\psi_{N}$ with the constant map with value $M^{N}$; by Rules for Derivatives of Vector- and Matrix-Valued Maps (ix), Rules for Derivatives of Vector- and Matrix-Valued Maps (iii) and Rules for Derivatives of Vector- and Matrix-Valued Maps (ii), with $[a,b]$ in place of $I$ (and Additive Laws for Vectors in $\mathbb{R}^q$ and for Real $p\times q$ Matrices: Group Laws and Rules for Differences (i)), $T_{N+1}$ is differentiable at $t$ and

$$T_{N+1}'(t)=T_{N}'(t)+\psi_{N}'(t)M^{N}+\psi_{N}(t)\,0_{n}=T_{N}'(t)+\psi_{N}'(t)M^{N}.\tag{$\ast$}$$

For $N=1$: $T_{1}$ is the constant map with value $I_{n}$, differentiable with $T_{1}'(t)=0_{n}$ by Rules for Derivatives of Vector- and Matrix-Valued Maps (ix). By Powers of a Square Matrix (i) with $k=0$, as $0+1=1$ by Arithmetic of Addition on Omega: Recursion Rules, Associativity, Commutativity, Cancellation and Compatibility with the Order (i), $M^{1}=M^{0}M=I_{n}M=M$, using Algebraic Rules for Matrix Products, Matrix-Vector Products and Transposes (iv). As $\psi_{1}=\psi_{0+1}$, $\psi_{1}'(t)=\psi_{0}(t)$ by Shifted Power Maps, and Integrating a Power Bound on the Derivative of a Matrix-Valued Map (i) with $N=0$. So, by $(\ast)$, $T_{1+1}'(t)=0_{n}+\psi_{0}(t)M^{1}=M=MI_{n}=MT_{1}(t)$. If the claim holds for $N$, then $T_{N+1}$ is differentiable, and by $(\ast)$ for $N+1$, Shifted Power Maps, and Integrating a Power Bound on the Derivative of a Matrix-Valued Map (i), Powers of a Square Matrix (i), Algebraic Rules for Matrix Products, Matrix-Vector Products and Transposes (iii) and Algebraic Rules for Matrix Products, Matrix-Vector Products and Transposes (ii),

$$T_{(N+1)+1}'(t)=MT_{N}(t)+\psi_{N}(t)\,(MM^{N})=MT_{N}(t)+M\bigl(\psi_{N}(t)M^{N}\bigr)=M\bigl(T_{N}(t)+\psi_{N}(t)M^{N}\bigr)=MT_{N+1}(t).$$

This proves the claim for $N+1$, and so for every $N\in\mathbb{N}$.

Step 2: the remainders. For $N\in\mathbb{N}$ let $D_{N}=\Phi-T_{N}:[a,b]\to\mathcal{M}_{n}(\mathbb{R})$, the pointwise difference. Then $D_{N}(s)=I_{n}-I_{n}=0_{n}$ (Additive Laws for Vectors in $\mathbb{R}^q$ and for Real $p\times q$ Matrices: Group Laws and Rules for Differences (ii)). By Step 1 and Rules for Derivatives of Vector- and Matrix-Valued Maps (iv), $D_{N}$ is differentiable, and, by Products of Real Matrices with Zero Matrices, Negatives and Differences (iii),

$$D_{N+1}'(t)=M\Phi(t)-MT_{N}(t)=M\bigl(\Phi(t)-T_{N}(t)\bigr)=MD_{N}(t)\qquad(t\in[a,b]).$$

By Rules for Derivatives of Vector- and Matrix-Valued Maps (i), $D_{N}$ is continuous at every point of $[a,b]$; so the map $t\mapsto MD_{N}(t)$ is continuous on $[a,b]$, by Continuity of Vector- and Matrix-Valued Maps: the Componentwise Criterion, Norms and Algebraic Operations (iv) (the constant map with value $M$ and a product) and Continuity of Vector- and Matrix-Valued Maps: the Componentwise Criterion, Norms and Algebraic Operations (v).

Step 3: the constant $C$. Let $C=\exp\bigl(K(b-a)\bigr)+1$; then $C>0$ by Basic Properties of the Exponential Function (iii). Since $\lVert A(r)\rVert=K$ for every $r\in[a,b]$, Basic Properties of the State-Transition Matrix (vi) gives $\lVert\Phi(r)\rVert\le\exp(K|r-s|)$ for every $r\in[a,b]$. For $r\in[a,b]$, $|r-s|\le b-a$, so $K|r-s|\le K(b-a)$, and $\exp(K|r-s|)\le\exp(K(b-a))$: by Basic Properties of the Exponential Function (ix) if $K|r-s|<K(b-a)$, with equality if $K|r-s|=K(b-a)$. By The Operator Norm of Entries, Transposes, Multiples of the Identity, Orthogonal Factors and Diagonal Matrices (vi) with $t=1$, $\lVert I_{n}\rVert=\lVert1\,I_{n}\rVert=1$. For $r\in[a,b]$, $D_{1}(r)=\Phi(r)-I_{n}=\Phi(r)+(-I_{n})$ by Step 1 and Additive Laws for Vectors in $\mathbb{R}^q$ and for Real $p\times q$ Matrices: Group Laws and Rules for Differences (ii), and $-I_{n}=(-1)I_{n}$ by Rules for Pointwise Operations on Real-Valued Maps: a Commutative Ring, Scalar Multiples and Evaluation (iv), with $[n]\times[n]$ for its $I$, as $\mathcal{M}_{n}(\mathbb{R})$ is the set of maps $[n]\times[n]\to\mathbb{R}$ with the pointwise operations (Real Matrices: the Set of Real m x n Matrices with Their Entrywise Operations (i), Real Matrices: the Set of Real m x n Matrices with Their Entrywise Operations (iv)). Hence, by Rules for the Operator Norm: Bound, Least Constant, Definiteness, Homogeneity, Triangle Inequality and Products (vi) and Rules for the Operator Norm: Bound, Least Constant, Definiteness, Homogeneity, Triangle Inequality and Products (v),

$$\lVert D_{1}(r)\rVert=\lVert\Phi(r)+(-1)I_{n}\rVert\le\lVert\Phi(r)\rVert+\lVert I_{n}\rVert\le C\qquad(r\in[a,b]).$$

Step 4: the bound. We show, for every $k\in\mathbb{N}$,

$$\lVert D_{k}(r)\rVert\le C\,\frac{K^{k-1}|r-s|^{k-1}}{(k-1)!}\qquad\text{for every }r\in[a,b],$$

by induction on $k$ (Arithmetic and Order of the Natural Numbers (xiii)). For $k=1$: $1-1=0$ by The Difference of Two Natural Numbers with Zero (i), as $1+0=1$ by Arithmetic of Addition on Omega: Recursion Rules, Associativity, Commutativity, Cancellation and Compatibility with the Order (i); so the right side is $C$, by Powers with Exponents in the Natural Numbers with Zero (ii) and $0!=1$ (The Factorial as a Map on the Natural Numbers with Zero (ii)), and the bound is Step 3.

Suppose the bound holds for $k$, and let $N=k-1\in\mathbb{N}_{0}$. Then $1+N=k$ by The Difference of Two Natural Numbers with Zero (i), so $N+1=k$ by Arithmetic of Addition on Omega: Recursion Rules, Associativity, Commutativity, Cancellation and Compatibility with the Order (iv); likewise, $k+1\in\mathbb{N}$ by Natural Numbers Are the Successors in Omega: One Is Least and Not a Successor of a Natural Number, the Successor Is Injective, and N Is Closed under Addition and Multiplication (ii) and Natural Numbers Are the Successors in Omega: One Is Least and Not a Successor of a Natural Number, the Successor Is Injective, and N Is Closed under Addition and Multiplication (vi), so $1\le k+1$ by Natural Numbers Are the Successors in Omega: One Is Least and Not a Successor of a Natural Number, the Successor Is Injective, and N Is Closed under Addition and Multiplication (ii) and the difference $(k+1)-1$ is defined, and $(k+1)-1=k$, as $1+k=k+1$. For $r\in[a,b]$, by Rules for the Operator Norm: Bound, Least Constant, Definiteness, Homogeneity, Triangle Inequality and Products (vii), the induction hypothesis and $K\ge0$,

$$\lVert MD_{k}(r)\rVert\le K\,\lVert D_{k}(r)\rVert\le C\,K^{k}\,\frac{|r-s|^{N}}{N!},$$

as $K\,K^{N}=K^{N}K^{1}=K^{N+1}=K^{k}$ by Powers in a Commutative Ring, a Field and an Ordered Field: Exponent Laws, Factorisation, Geometric Sums, Monotonicity and Bernoulli's Inequality (ii) and Powers in a Commutative Ring, a Field and an Ordered Field: Exponent Laws, Factorisation, Geometric Sums, Monotonicity and Bernoulli's Inequality (i). By Step 2, $D_{k+1}$ is differentiable, $D_{k+1}(s)=0_{n}$, and its derivative $r\mapsto MD_{k}(r)$ is continuous on $[a,b]$; and $C\,K^{k}\ge0$ by $C>0$ and Powers in a Commutative Ring, a Field and an Ordered Field: Exponent Laws, Factorisation, Geometric Sums, Monotonicity and Bernoulli's Inequality (vi). So Shifted Power Maps, and Integrating a Power Bound on the Derivative of a Matrix-Valued Map (ii), with $n$ and $n$ in place of $p$ and $q$, $C\,K^{k}$ in place of $c$ and $D_{k+1}$ in place of $E$, gives, for every $t\in[a,b]$,

$$\lVert D_{k+1}(t)\rVert\le C\,K^{k}\,\frac{|t-s|^{N+1}}{(N+1)!}=C\,\frac{K^{(k+1)-1}|t-s|^{(k+1)-1}}{((k+1)-1)!}.$$

This is the bound for $k+1$, and so the bound holds for every $k\in\mathbb{N}$.

Step 5: the limit. Fix $t\in[a,b]$ and let $w=K|t-s|$, so $w\ge0$. Let $(x_{k})_{k\in\mathbb{N}}$ be the sequence of The Exponential Series Converges Absolutely, and Its Sum Is a Map on ℝ (i) with $w$ in place of its parameter, so $x_{k}=w^{k-1}/(k-1)!$. Its series converges by The Exponential Series Converges Absolutely, and Its Sum Is a Map on ℝ (ii), so $(x_{k})$ is a null sequence by Elementary Properties of Series of Real Numbers: Linearity, Null Terms, the Cauchy Criterion, Index Shifts, Tails, Order and Telescoping (ii). For $k\in\mathbb{N}$, $K^{k-1}|t-s|^{k-1}=w^{k-1}$ by Powers in a Commutative Ring, a Field and an Ordered Field: Exponent Laws, Factorisation, Geometric Sums, Monotonicity and Bernoulli's Inequality (ii), so Step 4 gives $\lVert D_{k}(t)\rVert\le C\,x_{k}$.

Let $\varepsilon>0$. As $(x_{k})$ converges to $0$, Convergent Sequences of Real Numbers (i), applied to $\varepsilon/C>0$, gives $N_{0}\in\mathbb{N}$ with $|x_{k}|<\varepsilon/C$ for every $k\in\mathbb{N}$ with $k\ge N_{0}$. For such $k$, by Rules for the Operator Norm: Bound, Least Constant, Definiteness, Homogeneity, Triangle Inequality and Products (v),

$$d_{n}\bigl(T_{k}(t),\Phi(t)\bigr)=\lVert T_{k}(t)-\Phi(t)\rVert=\lVert(-1)D_{k}(t)\rVert=\lVert D_{k}(t)\rVert\le C\,x_{k}<\varepsilon .$$

So $(T_{k}(t))_{k\in\mathbb{N}}$ converges to $\Phi(t)$ in $(\mathcal{M}_{n}(\mathbb{R}),d_{n})$, by Convergent Sequence in a Metric Space (i). By Step 1 it also converges to $e^{(t-s)M}$, so $\Phi_{A}(t,s)=\Phi(t)=e^{(t-s)M}$ by Uniqueness of Limits in a Metric Space (i). As $s,t\in[a,b]$ were arbitrary, this proves (i).