Real Matrices, Symmetric Matrices and the Semidefinite Ordering: Standing Notation

This setting fixes the standing notation used by results of linear algebra on real matrices and by the analysis built on them.

1. (Numbers and Euclidean space)¶ The natural numbers $\mathbb{N}$, the real numbers $\mathbb{R}$ with the addition, multiplication and order $\le$ of their ordered field structure, the strict order $a<b$, the difference $a-b$, the multiplicative inverse $a^{-1}$ of a nonzero $a$, the absolute value $|\cdot|$, the words positive and nonnegative, the initial segments $[p]$, sequences, and convergence of sequences of real numbers are as fixed in clause 1 of the Euclidean setting. Clause 2 there fixes, for every natural number $q$ with $1\le q$, Euclidean space $\mathbb{R}^{q}$ as a real vector space with the sum $x+y$ of points, the scalar multiple $\mu x$, the difference $x-y$ and the dot product $x\cdot y$, the Euclidean norm $\lVert\,\cdot\,\rVert$, the Euclidean distance $d_{E}$ satisfying $d_{E}(x,y)=\lVert x-y\rVert$, the metric space $(\mathbb{R}^{q},d_{E})$, and the notions of openness, closedness, boundedness and compactness. We abbreviate $\lVert z\rVert^{2}=\lVert z\rVert\cdot\lVert z\rVert$, and $0_{\mathbb{R}^{q}}$ denotes the origin of $\mathbb{R}^{q}$.

Throughout clauses 2 to 7, $p$ and $q$ denote natural numbers with $1\le p$ and $1\le q$, and the notation is introduced for all such $p$ and $q$ simultaneously; a result adopting this setting uses it for whichever dimensions it names.

2. (Real matrices and their algebra)¶ $\mathcal{M}_{p\times q}(\mathbb{R})$ denotes the set of real $p\times q$ matrices and $\mathcal{M}_{p}(\mathbb{R})$ the set of square real $p\times p$ matrices, with $A_{ij}$ the entry of $A$ in row $i\in[p]$ and column $j\in[q]$; as recorded there, two real matrices of the same size are equal exactly when all their entries agree. We write $0_{p\times q}$ for the element of $\mathcal{M}_{p\times q}(\mathbb{R})$ all of whose entries are $0$, and abbreviate $0_{p}=0_{p\times p}$. For real matrices of sizes for which the operation is defined, $A+B$ is the sum, $A-B$ the difference, $\mu A$ the scalar multiple by $\mu\in\mathbb{R}$, $AB$ the product, $A^{\top}$ the transpose, and $Az$ the matrix-vector product of $A\in\mathcal{M}_{p\times q}(\mathbb{R})$ with $z\in\mathbb{R}^{q}$. Further, $I_{p}$ is the identity matrix of size $p$, and for $A\in\mathcal{M}_{p}(\mathbb{R})$ we write $A^{2}=AA$.

3. (Symmetric matrices)¶ A square real matrix $A$ is symmetric if $A=A^{\top}$, equivalently, by the entry formula for the transpose together with the criterion recorded in clause 2 that two real matrices of the same size are equal exactly when all their entries agree, if $A_{ij}=A_{ji}$ for all indices $i$ and $j$; and $\mathcal{S}(p)$ denotes the set of symmetric real $p\times p$ matrices. The matrices $I_{p}$ and $0_{p}$ lie in $\mathcal{S}(p)$, their entries being unchanged when the two indices are interchanged. By claim 1 of The Positive Semidefinite Ordering is a Partial Order Compatible with the Linear Structure the sum, the difference and every scalar multiple of elements of $\mathcal{S}(p)$ again lie in $\mathcal{S}(p)$; in particular $aI_{p}\in\mathcal{S}(p)$ for every $a\in\mathbb{R}$, and we write $-aI_{p}$ for $(-a)I_{p}$.

4. (The positive semidefinite ordering)¶ An $A\in\mathcal{S}(p)$ is positive semidefinite if $z\cdot(Az)\ge0$ for every $z\in\mathbb{R}^{p}$. On $\mathcal{S}(p)$ the relation $\preceq$ is the positive semidefinite ordering, so that $X\preceq Y$ means that $z\cdot(Xz)\le z\cdot(Yz)$ for every $z\in\mathbb{R}^{p}$. By The Positive Semidefinite Ordering is a Partial Order Compatible with the Linear Structure it is reflexive, transitive and antisymmetric, and is compatible with adding a fixed element of $\mathcal{S}(p)$ to both sides and with multiplying both sides by a nonnegative real number; by The Positive Semidefinite Ordering Compared by Differences, $X\preceq Y$ holds if and only if $Y-X$ is positive semidefinite.

5. (Norm and distance on symmetric matrices)¶ For $A\in\mathcal{S}(p)$, $\lVert A\rVert$ denotes the norm of the symmetric real matrix $A$, namely the least upper bound of the set $\{\,|\xi\cdot(A\xi)|:\xi\in\mathbb{R}^{p},\ \lVert\xi\rVert\le1\,\}$, which exists by the reference recorded in that definition. The symbol $\lVert\,\cdot\,\rVert$ is used both for the Euclidean norm of a point and for this norm; the argument determines which is meant. Next, $d_{\mathcal{S}(p)}(X,Y)=\lVert X-Y\rVert$ is the distance between symmetric real matrices, a metric by The Set of Symmetric Real Matrices is a Metric Space, so that $\bigl(\mathcal{S}(p),d_{\mathcal{S}(p)}\bigr)$ is a metric space. Convergence of a sequence in a metric space is as defined there; convergence in $\mathbb{R}^{q}$ always refers to $d_{E}$, convergence in $\mathcal{S}(p)$ to $d_{\mathcal{S}(p)}$, and convergence in $\mathbb{R}$ to the metric of The Absolute Value Metric on the Real Line.

6. (Standard basis vectors)¶ For $i\in[q]$, $e_{i}$ denotes the $i$th standard basis vector of $\mathbb{R}^{q}$, the point whose $i$th coordinate is $1$ and whose other coordinates are $0$; as recorded there the family $e_{1},\dots,e_{q}$ is orthonormal, and the $i$th coordinate of a point $z\in\mathbb{R}^{q}$ equals $z\cdot e_{i}$. Consequently $\lVert e_{i}\rVert=1$: by claim 1 of Elementary Properties of the Euclidean Norm on $\mathbb{R}^n$ we have $\lVert e_{i}\rVert^{2}=e_{i}\cdot e_{i}$, which is $1$ by orthonormality, and $\lVert e_{i}\rVert$ is nonnegative by that same claim, so claim 3 of Monotonicity of Squaring on the Nonnegative Elements of an Ordered Field, applied with $1=1\cdot1$, gives $\lVert e_{i}\rVert=1$. Which dimension is meant is determined by the space in which $e_{i}$ is used.

7. (Concatenation and block matrices)¶ For natural numbers $m$ and $n$ with $1\le m$ and $1\le n$, let

be the concatenation map, a bijection by claim 1 of that lemma; as recorded there, every $k\in[m+n]$ satisfies exactly one of $k\in[m]$ and $k=m+j$ for a unique $j\in[n]$. For real matrices $A$, $B$, $C$ and $D$ of sizes $m\times m$, $m\times n$, $n\times m$ and $n\times n$ respectively, the block matrix they determine, an element of $\mathcal{M}_{m+n}(\mathbb{R})$, is written