Proof of Eigenvalues: the Determinant Criterion, Diagonal Matrices, Orthogonal Similarity and Orthogonality of Eigenvectors of Symmetric Matrices

Each result cited is universally quantified over the data in its own statement. We work in Routine Facts Used Without Citation: Arithmetic, Maps, Predicativity and Indices.

Two identities. For $a\in\mathbb{R}^{n}$ and $i\in[n]$, by Matrix-Vector Product §product, Identity Matrix and Standard Basis Vectors §identity and the rule $I_{n}x=x$ of Algebraic Rules for Matrix Products, Matrix-Vector Products and Transposes §matrix-vector,

$$\sum_{j=1}^{n}\delta_{ij}a_{j}=(I_{n}a)_{i}=a_{i}.\tag{1}$$

For $y\in\mathbb{R}^{n}$ and $i\in[n]$, by Matrix-Vector Product §product, Real Matrices: the Set of Real m x n Matrices with Their Entrywise Operations §operations and (1),

$$\bigl((A-\lambda I_{n})y\bigr)_{i}=\sum_{j=1}^{n}(A_{ij}-\lambda\delta_{ij})y_{j}=(Ay)_{i}-\lambda y_{i};$$

hence, by Euclidean Space $\mathbb{R}^n$: n-Tuples of Real Numbers with Their Componentwise Operations §equality, Euclidean Space $\mathbb{R}^n$: n-Tuples of Real Numbers with Their Componentwise Operations §operations and Euclidean Space $\mathbb{R}^n$: n-Tuples of Real Numbers with Their Componentwise Operations §origin, we have the following statement (2): $(A-\lambda I_{n})y=0$ if and only if $Ay=\lambda y$.

Clause determinant. By Real Eigenvalues and Eigenvectors of a Real Square Matrix §eigenvalue and Real Eigenvalues and Eigenvectors of a Real Square Matrix §eigenvector, $\lambda$ is an eigenvalue of $A$ if and only if there is $y\in\mathbb{R}^{n}$ with $y\neq0$ and $Ay=\lambda y$, that is, by (2), with $y\neq0$ and $(A-\lambda I_{n})y=0$. Since $(A-\lambda I_{n})0=0$ by Algebraic Rules for Matrix Products, Matrix-Vector Products and Transposes §matrix-vector, such a $y$ exists if and only if condition Criteria for Invertibility of a Square Real Matrix §trivial-kernel fails for $A-\lambda I_{n}$ in place of $A$. That condition is equivalent to condition Criteria for Invertibility of a Square Real Matrix §invertible by that theorem; so $\lambda$ is an eigenvalue of $A$ if and only if $A-\lambda I_{n}$ is not invertible. By A Square Real Matrix Is Invertible if and only if Its Determinant Is Nonzero §invertible, $A-\lambda I_{n}$ is not invertible if and only if $\det(A-\lambda I_{n})=0$.

Clause diagonal-algebra. Write $D=\operatorname{diag}(\mu)$ and $E=\operatorname{diag}(\nu)$, so that $D_{ij}=\mu_{i}\delta_{ij}$ and $E_{ij}=\nu_{i}\delta_{ij}$ for $i,j\in[n]$ by Diagonal Matrix with Prescribed Diagonal §diagonal. By Identity Matrix and Standard Basis Vectors §identity, $\delta_{ij}=\delta_{ji}$ and $\mu_{i}\delta_{ij}=\mu_{j}\delta_{ij}$, both sides being $\mu_{i}$ if $i=j$ and $0$ if $i\neq j$. All matrix equalities below follow from equality of entries by Real Matrices: the Set of Real m x n Matrices with Their Entrywise Operations §equality.

By Transpose of a Real Matrix §transpose, $(D^{\top})_{ij}=D_{ji}=\mu_{j}\delta_{ji}=\mu_{i}\delta_{ij}=D_{ij}$, so $D^{\top}=D$.

For $i,k\in[n]$, by Product of Real Matrices §product and by (1) applied to the point $j\mapsto\nu_{j}\delta_{jk}$ of $\mathbb{R}^{n}$,

$$(DE)_{ik}=\sum_{j=1}^{n}\mu_{i}\delta_{ij}\,\nu_{j}\delta_{jk}=\mu_{i}\sum_{j=1}^{n}\delta_{ij}\,\nu_{j}\delta_{jk}=\mu_{i}\nu_{i}\delta_{ik}=(\mu\nu)_{i}\delta_{ik}=\operatorname{diag}(\mu\nu)_{ik},$$

using Pointwise Operations on Real-Valued Maps §operations and Diagonal Matrix with Prescribed Diagonal §diagonal; so $DE=\operatorname{diag}(\mu\nu)$.

For $k\in[n]$, by Matrix-Vector Product §product and (1), $(Dx)_{k}=\sum_{j=1}^{n}\mu_{k}\delta_{kj}x_{j}=\mu_{k}x_{k}$. Hence, by The Dot Product on $\mathbb{R}^n$ §dot-product, $x\cdot(Dx)=\sum_{k=1}^{n}x_{k}\,\mu_{k}x_{k}=\sum_{k=1}^{n}\mu_{k}x_{k}^{2}$.

Finally, by Pointwise Operations on Real-Valued Maps §constants and Diagonal Matrix with Prescribed Diagonal §diagonal, $\operatorname{diag}(1)_{ij}=1\cdot\delta_{ij}=\delta_{ij}=(I_{n})_{ij}$ for $i,j\in[n]$, so $\operatorname{diag}(1)=I_{n}$.

Clause diagonal-eigenvalues. Let $D=\operatorname{diag}(\mu)$ and $k\in[n]$. By Identity Matrix and Standard Basis Vectors §basis, $(e_{k})_{i}=\delta_{ik}$ for $i\in[n]$; so by clause diagonal-algebra and the identity $\mu_{i}\delta_{ik}=\mu_{k}\delta_{ik}$ noted there, $(De_{k})_{i}=\mu_{i}\delta_{ik}=\mu_{k}\delta_{ik}=(\mu_{k}e_{k})_{i}$, using Euclidean Space $\mathbb{R}^n$: n-Tuples of Real Numbers with Their Componentwise Operations §operations. Hence $De_{k}=\mu_{k}e_{k}$ by Euclidean Space $\mathbb{R}^n$: n-Tuples of Real Numbers with Their Componentwise Operations §equality. Moreover $(e_{k})_{k}=\delta_{kk}=1\neq0$ by Identity Matrix and Standard Basis Vectors §basis and Identity Matrix and Standard Basis Vectors §identity, whereas every component of the origin is $0$ by Euclidean Space $\mathbb{R}^n$: n-Tuples of Real Numbers with Their Componentwise Operations §origin, so $e_{k}\neq0$. By Real Eigenvalues and Eigenvectors of a Real Square Matrix §eigenvector, $e_{k}$ is an eigenvector of $D$ with eigenvalue $\mu_{k}$.

Now let $\kappa\in\mathbb{R}$ be an eigenvalue of $D$. By Real Eigenvalues and Eigenvectors of a Real Square Matrix §eigenvalue and Real Eigenvalues and Eigenvectors of a Real Square Matrix §eigenvector, there is $y\in\mathbb{R}^{n}$ with $y\neq0$ and $Dy=\kappa y$. Since $y\neq0$, Euclidean Space $\mathbb{R}^n$: n-Tuples of Real Numbers with Their Componentwise Operations §equality and Euclidean Space $\mathbb{R}^n$: n-Tuples of Real Numbers with Their Componentwise Operations §origin give $k\in[n]$ with $y_{k}\neq0$. By clause diagonal-algebra and Euclidean Space $\mathbb{R}^n$: n-Tuples of Real Numbers with Their Componentwise Operations §operations, $\mu_{k}y_{k}=(Dy)_{k}=\kappa y_{k}$, and dividing by $y_{k}$ gives $\kappa=\mu_{k}$.

Clause similarity. Let $Q\in\mathrm{O}(n)$, so $Q^{\top}Q=I_{n}$ by Orthogonal Real Matrices §membership. For every $y\in\mathbb{R}^{n}$, by the rules $(AB)z=A(Bz)$ and $I_{n}x=x$ of Algebraic Rules for Matrix Products, Matrix-Vector Products and Transposes §matrix-vector,

$$Q^{\top}(Qy)=(Q^{\top}Q)y=I_{n}y=y,\tag{3}$$ $$(QAQ^{\top})(Qy)=(QA)\bigl(Q^{\top}(Qy)\bigr)=(QA)y=Q(Ay).\tag{4}$$

Suppose that $x$ is an eigenvector of $A$ with eigenvalue $\lambda$, so $x\neq0$ and $Ax=\lambda x$ by Real Eigenvalues and Eigenvectors of a Real Square Matrix §eigenvector. If $Qx=0$, then $x=Q^{\top}0=0$ by (3) and the rule $A0=0$ of Algebraic Rules for Matrix Products, Matrix-Vector Products and Transposes §matrix-vector, which is false; so $Qx\neq0$. By (4) and the rule $A(sx)=s(Ax)$ of Algebraic Rules for Matrix Products, Matrix-Vector Products and Transposes §matrix-vector, $(QAQ^{\top})(Qx)=Q(\lambda x)=\lambda(Qx)$. So $Qx$ is an eigenvector of $QAQ^{\top}$ with eigenvalue $\lambda$.

Conversely, suppose that $Qx\neq0$ and $(QAQ^{\top})(Qx)=\lambda(Qx)$. Then $x\neq0$, since $Q0=0$ by Algebraic Rules for Matrix Products, Matrix-Vector Products and Transposes §matrix-vector. By (4) and the same rules, $Q(Ax)=\lambda(Qx)=Q(\lambda x)$; applying $Q^{\top}$ and using (3) gives $Ax=\lambda x$. So $x$ is an eigenvector of $A$ with eigenvalue $\lambda$.

For the eigenvalues: if $\lambda$ is an eigenvalue of $A$, with an eigenvector $x'$, then $Qx'$ is an eigenvector of $QAQ^{\top}$ with eigenvalue $\lambda$ by the first part (with $x'$ in place of $x$), so $\lambda$ is an eigenvalue of $QAQ^{\top}$ by Real Eigenvalues and Eigenvectors of a Real Square Matrix §eigenvalue. Conversely, let $y$ be an eigenvector of $QAQ^{\top}$ with eigenvalue $\lambda$ and let $x'=Q^{\top}y$. By Rules for Orthogonal Matrices: Inverse, Products, Isometry, Columns, Determinant, Orthonormal Extension and Expansion §inverse and Algebraic Rules for Matrix Products, Matrix-Vector Products and Transposes §matrix-vector, $Qx'=(QQ^{\top})y=I_{n}y=y$; so $Qx'$ is an eigenvector of $QAQ^{\top}$ with eigenvalue $\lambda$, hence $x'$ is an eigenvector of $A$ with eigenvalue $\lambda$ by the first part, and $\lambda$ is an eigenvalue of $A$.

Clause symmetric-orthogonal. Let $A\in\mathcal{S}(n)$; then $A$ is symmetric by Symmetric Real Matrices and the Set $\mathcal{S}(n)$ §set, that is, $A^{\top}=A$ by Symmetric Real Matrices and the Set $\mathcal{S}(n)$ §symmetric. By Real Eigenvalues and Eigenvectors of a Real Square Matrix §eigenvector, $Ax=\lambda x$ and $Ay=\lambda'y$. By The Dot Product on $\mathbb{R}^n$ Is Symmetric, Bilinear and Positive Definite §first, by Algebraic Rules for Matrix Products, Matrix-Vector Products and Transposes §adjoint with $m=n$ and $w=y$, and by The Dot Product on $\mathbb{R}^n$ Is Symmetric, Bilinear and Positive Definite §second,

$$\lambda\,(x\cdot y)=(\lambda x)\cdot y=(Ax)\cdot y=x\cdot(A^{\top}y)=x\cdot(Ay)=x\cdot(\lambda'y)=\lambda'\,(x\cdot y).$$

Hence $(\lambda-\lambda')(x\cdot y)=0$, and since $\lambda-\lambda'\neq0$, $x\cdot y=0$.