Euclidean Space and Real Matrices: Standing Notation

(i) This setting extends Routine Facts Used Without Citation: Arithmetic, Maps, Predicativity and Indices: its conventions and notation are in force.

Throughout, $p$ and $q$ denote natural numbers, and the notation below is introduced for all of them at once; a result adopting this setting uses it for whichever dimensions it names.

(ii) Numbers and Euclidean space. The natural numbers $\mathbb{N}$, the real numbers $\mathbb{R}$ with their field operations and order, the absolute value $|\cdot|$, sequences, and convergence in a metric space are those of the extended settings, and the segments $[p]$ are as in Routine Facts Used Without Citation: Arithmetic, Maps, Predicativity and Indices (v). A real number $a$ is positive if $0<a$ and nonnegative if $0\le a$, and $a<b$ is the strict order. Euclidean space $\mathbb{R}^{q}$, the dot product $x\cdot y$, the Euclidean norm $\lVert\cdot\rVert$ and the Euclidean distance $d_{E}$ are as in Routine Facts Used Without Citation: Arithmetic, Maps, Predicativity and Indices (iii), with $d_{E}(x,y)=\lVert x-y\rVert$ by The Euclidean Distance on $\mathbb{R}^n$ (i). The components $x_{k}$ of $x\in\mathbb{R}^{q}$ are those of Euclidean Space $\mathbb{R}^n$: n-Tuples of Real Numbers with Their Componentwise Operations (ii); the sum $x+y$, the scalar multiple $\mu x$, the negative $-x$ and the difference $x-y$ are those of Euclidean Space $\mathbb{R}^n$: n-Tuples of Real Numbers with Their Componentwise Operations (iv); and $0$, also written $0_{\mathbb{R}^{q}}$, is the origin of $\mathbb{R}^{q}$. The vector-space laws of $\mathbb{R}^{q}$ are those of Rules for Pointwise Operations on Real-Valued Maps: a Commutative Ring, Scalar Multiples and Evaluation, as carried by Routine Facts Used Without Citation: Arithmetic, Maps, Predicativity and Indices (iii). $(\mathbb{R}^{q},d_{E})$ is a metric space by $\mathbb{R}^n$ with the Euclidean Distance Is a Complete Metric Space with Coordinatewise Convergence (i). Convergence in $\mathbb{R}^{q}$ refers to $d_{E}$, and convergence in $\mathbb{R}$ to the absolute-value metric $d_{\mathbb{R}}$ of Metric Spaces: Balls, the Metric Topology, and the Real and Complex Numbers as Metric Spaces (iv). Open, closed, bounded and compact subsets of $\mathbb{R}^{q}$ likewise refer to $d_{E}$: for a subset of $\mathbb{R}^{q}$, open and bounded mean open and bounded in the metric space $(\mathbb{R}^{q},d_{E})$, and closed and compact mean closed and compact in the topological space $(\mathbb{R}^{q},\mathcal{T}_{d_{E}})$, with $\mathcal{T}_{d_{E}}$ the topology of $(\mathbb{R}^{q},d_{E})$ as in Metric Spaces: Balls, the Metric Topology, and the Real and Complex Numbers as Metric Spaces (iii).

(iii) Real matrices. $\mathcal{M}_{p\times q}(\mathbb{R})$ is the set of real $p\times q$ matrices and $\mathcal{M}_{p}(\mathbb{R})=\mathcal{M}_{p\times p}(\mathbb{R})$, as in Routine Facts Used Without Citation: Arithmetic, Maps, Predicativity and Indices (iii). $A_{ij}$ is the entry of $A$ in row $i\in[p]$ and column $j\in[q]$; two matrices of the same size are equal exactly when their entries agree, by Real Matrices: the Set of Real m x n Matrices with Their Entrywise Operations (iii); and $A+B$, $A-B$, $-A$ and $\mu A$ ($\mu\in\mathbb{R}$) are the sum, difference, negative and scalar multiple of Real Matrices: the Set of Real m x n Matrices with Their Entrywise Operations (iv). $0_{p\times q}$ is the zero matrix of $\mathcal{M}_{p\times q}(\mathbb{R})$, and $0_{p}=0_{p\times p}$. For matrices of matching sizes, $AB$ is the matrix product, $A^{\top}$ the transpose and, for $A\in\mathcal{M}_{p\times q}(\mathbb{R})$ and $z\in\mathbb{R}^{q}$, $Az$ the matrix-vector product; $I_{p}$ is the identity matrix of size $p$; all as in Routine Facts Used Without Citation: Arithmetic, Maps, Predicativity and Indices (iii). For $A\in\mathcal{M}_{p}(\mathbb{R})$, $A^{2}=AA$. Background: the algebraic rules of Algebraic Rules for Matrix Products, Matrix-Vector Products and Transposes.

(iv) Symmetric matrices. $\mathcal{S}(p)$ is the set of symmetric real $p\times p$ matrices, the $A\in\mathcal{M}_{p}(\mathbb{R})$ that are symmetric: $A$ is symmetric if $A=A^{\top}$ (Symmetric Real Matrices and the Set $\mathcal{S}(n)$ (i)). By Symmetric Real Matrices and the Set $\mathcal{S}(n)$ (iii), $A\in\mathcal{S}(p)$ exactly when $A_{ij}=A_{ji}$ for all $i,j\in[p]$. By Rules for Symmetric and Positive Semidefinite Matrices: Closure, Self-Adjointness, Polarization, Congruence and Cones (i), $0_{p}$ and $I_{p}$ lie in $\mathcal{S}(p)$, and sums, differences and scalar multiples of elements of $\mathcal{S}(p)$ lie in $\mathcal{S}(p)$; in particular $aI_{p}\in\mathcal{S}(p)$ for $a\in\mathbb{R}$, and $-aI_{p}$ denotes $(-a)I_{p}$.

(v) The semidefinite ordering. For $A\in\mathcal{S}(p)$, positive semidefinite and positive definite are as defined there, and $\preceq$ is the positive semidefinite ordering on $\mathcal{S}(p)$. Background: The Positive Semidefinite Ordering is a Partial Order Compatible with the Linear Structure (in particular The Positive Semidefinite Ordering is a Partial Order Compatible with the Linear Structure (vi): $X\preceq Y$ iff $Y-X$ is positive semidefinite) and Rules for Symmetric and Positive Semidefinite Matrices: Closure, Self-Adjointness, Polarization, Congruence and Cones.

(vi) Norm and distance. For $A\in\mathcal{M}_{p\times q}(\mathbb{R})$, $\lVert A\rVert$ is the operator norm of $A$; the symbol $\lVert\cdot\rVert$ is shared with the Euclidean norm, and the argument, a matrix or a point, decides which is meant. $d_{p\times q}$, with $d_{p}=d_{p\times p}$, is the distance $d_{p\times q}(A,B)=\lVert A-B\rVert$ on $\mathcal{M}_{p\times q}(\mathbb{R})$, a metric by The Real Matrices with the Operator-Norm Distance Form a Complete Metric Space with Entrywise Convergence (i), and $d_{\mathcal{S}(p)}$ is its restriction to $\mathcal{S}(p)\times\mathcal{S}(p)$, the restricted metric of Subsets of a Metric Space: the Restricted Metric, Sequences, Limits in Closed Subsets and Completeness of Closed Subsets (i), so a metric on $\mathcal{S}(p)$; $(\mathcal{S}(p),d_{\mathcal{S}(p)})$ is a complete metric space by Limits of Matrix Operations, and the Closed Complete Sets of Symmetric and of Positive Semidefinite Matrices (ix). Convergence in $\mathcal{M}_{p\times q}(\mathbb{R})$ refers to $d_{p\times q}$, and convergence in $\mathcal{S}(p)$ to $d_{\mathcal{S}(p)}$. For $A\in\mathcal{S}(p)$, $\lVert A\rVert$ is the supremum of $\{|\xi\cdot(A\xi)|:\xi\in\mathbb{R}^{p},\ \lVert\xi\rVert\le1\}$, by The Operator Norm of a Symmetric Matrix: Quadratic Forms, Eigenvalues and the Semidefinite Ordering (i). Background: The Real Matrices with the Operator-Norm Distance Form a Complete Metric Space with Entrywise Convergence, Rules for the Operator Norm: Bound, Least Constant, Definiteness, Homogeneity, Triangle Inequality and Products, The Operator Norm of Entries, Transposes, Multiples of the Identity, Orthogonal Factors and Diagonal Matrices, The Operator Norm of a Symmetric Matrix: Quadratic Forms, Eigenvalues and the Semidefinite Ordering, Limits of Matrix Operations, and the Closed Complete Sets of Symmetric and of Positive Semidefinite Matrices and Subsets of a Metric Space: the Restricted Metric, Sequences, Limits in Closed Subsets and Completeness of Closed Subsets.

(vii) Standard basis vectors. For $i\in[q]$, $e_{i}\in\mathbb{R}^{q}$ is the $i$th standard basis vector, as in Routine Facts Used Without Citation: Arithmetic, Maps, Predicativity and Indices (iii): its $i$th coordinate is $1$ and its other coordinates are $0$. Background: Standard Basis Vectors of ℝ^q: Coordinates, Orthonormality and Unit Length; the $i$th coordinate of $z\in\mathbb{R}^{q}$ equals $z\cdot e_{i}$ (Standard Basis Vectors of ℝ^q: Coordinates, Orthonormality and Unit Length (i) and The Dot Product on $\mathbb{R}^n$ Is Symmetric, Bilinear and Positive Definite (i)). Which dimension is meant is determined by the space in which $e_{i}$ is used.

(viii) Concatenation and block matrices. For $m,n\in\mathbb{N}$,

$$\iota:\mathbb{R}^{m}\times\mathbb{R}^{n}\to\mathbb{R}^{m+n}$$

is the concatenation map of Concatenation Identifies a Product of Euclidean Spaces with a Euclidean Space (ii), a bijection by Concatenation Identifies a Product of Euclidean Spaces with a Euclidean Space (iii); indices of $[m+n]$ split as in Concatenation Identifies a Product of Euclidean Spaces with a Euclidean Space (i). For real matrices $A$, $B$, $C$ and $D$ of sizes $m\times m$, $m\times n$, $n\times m$ and $n\times n$, the block matrix they determine, an element of $\mathcal{M}_{m+n}(\mathbb{R})$, is written

$$\begin{pmatrix}A&B\\C&D\end{pmatrix}.$$

Background: Action and Quadratic Form of a Block Matrix.

(ix) Trace. For $A\in\mathcal{M}_{p}(\mathbb{R})$, $\operatorname{tr}(A)$ is the trace of $A$, as in Routine Facts Used Without Citation: Arithmetic, Maps, Predicativity and Indices (iii).

(x) Inverses. Invertible matrices in $\mathcal{M}_{p}(\mathbb{R})$ and the inverse $A^{-1}$ of an invertible $A$ are as defined there, and $\mathrm{GL}_{p}(\mathbb{R})$ is the set of invertible matrices in $\mathcal{M}_{p}(\mathbb{R})$, as in Perturbations of an Invertible Matrix: the Invertible Matrices Form an Open Set and Inversion Is Continuous. Background: Criteria for Invertibility of a Square Real Matrix, Adjugate Formula for the Matrix Inverse and Perturbations of an Invertible Matrix: the Invertible Matrices Form an Open Set and Inversion Is Continuous.

(xi) Determinant. For $A\in\mathcal{M}_{p}(\mathbb{R})$, $\det A$ is the determinant of $A$, as in Routine Facts Used Without Citation: Arithmetic, Maps, Predicativity and Indices (iii). Background: The Determinant is Multiplicative, A Square Real Matrix Is Invertible if and only if Its Determinant Is Nonzero and Cofactor Expansion of the Determinant along a Row or a Column, and the Adjugate Identity.

(xii) Orthogonal matrices. $\mathrm{O}(p)$ is the set of orthogonal $p\times p$ matrices, the $Q\in\mathcal{M}_{p}(\mathbb{R})$ that are orthogonal, with membership as in Orthogonal Real Matrices (iii). Background: Rules for Orthogonal Matrices: Inverse, Products, Isometry, Columns, Determinant, Orthonormal Extension and Expansion.

(xiii) Spectral notation. For $d\in\mathbb{R}^{p}$, $\operatorname{diag}(d)\in\mathcal{M}_{p}(\mathbb{R})$ is the diagonal matrix with diagonal $d$. Eigenvalues and eigenvectors of $A\in\mathcal{M}_{p}(\mathbb{R})$ are as defined there. For a positive semidefinite $A\in\mathcal{S}(p)$, $A^{1/2}$ is its positive semidefinite square root. Background: The Spectral Theorem for Real Symmetric Matrices, Eigenvalues: the Determinant Criterion, Diagonal Matrices, Orthogonal Similarity and Orthogonality of Eigenvectors of Symmetric Matrices, Positive Semidefinite and Positive Definite Matrices by Their Eigenvalues and Every Positive Semidefinite Matrix Has Exactly One Positive Semidefinite Square Root.