Proof of Eigenvalue, Quadratic-Form and Order Bounds for a Symmetric Matrix

Routine facts about the semidefinite ordering and scalars in the dot product are used without comment (Toolkit for AL3).

Write $\lambda_{k}=\lambda_{k}(A)$ and $\lambda=(\lambda_{1},\dots,\lambda_{n})$. By The Spectral Theorem for Real Symmetric Matrices (i) there are an orthogonal matrix $Q\in\mathrm{O}(n)$ and $\mu\in\mathbb{R}^{n}$ with $\mu_{1}\ge\dots\ge\mu_{n}$ and $A=(Q\operatorname{diag}(\mu))Q^{\top}$, and $\mu=\lambda$ by The Ordered Eigenvalues of a Symmetric Matrix (i); so

$$A=(Q\operatorname{diag}(\lambda))Q^{\top}.$$

Put $q_{k}=Qe_{k}$ for $k\in[n]$; then $q_{k}\cdot q_{k}=1$ by Rules for Orthogonal Matrices: Inverse, Products, Isometry, Columns, Determinant, Orthonormal Extension and Expansion (iv).

Order and quadratic form. For $t\in\mathbb{R}$ and $\xi\in\mathbb{R}^{n}$, The Operator Norm of a Symmetric Matrix: Quadratic Forms, Eigenvalues and the Semidefinite Ordering (ix) gives $\xi\cdot((tI_{n})\xi)=t\lVert\xi\rVert^{2}$. Hence, by the definition of the ordering, $aI_{n}\preceq A$ says exactly that $a\lVert\xi\rVert^{2}\le\xi\cdot(A\xi)$ for every $\xi$, and $A\preceq bI_{n}$ says exactly that $\xi\cdot(A\xi)\le b\lVert\xi\rVert^{2}$ for every $\xi$. This is (i).

Quadratic form and eigenvalues. Suppose first that the quadratic-form bounds hold. By The Spectral Theorem for Real Symmetric Matrices (ii), $q_{k}$ is an eigenvector of $A$ with eigenvalue $\lambda_{k}$, so by The Dot Product on $\mathbb{R}^n$ Is Symmetric, Bilinear and Positive Definite (iii)

$$q_{k}\cdot(Aq_{k})=\lambda_{k}\,q_{k}\cdot q_{k}=\lambda_{k},\qquad\lVert q_{k}\rVert^{2}=1.$$

Taking $\xi=q_{n}$ in the lower bound gives $a\le\lambda_{n}$, and taking $\xi=q_{1}$ in the upper bound gives $\lambda_{1}\le b$. Conversely, since $\lambda$ is non-increasing, The Spectral Theorem for Real Symmetric Matrices (iii) gives

$$\lambda_{n}\lVert\xi\rVert^{2}\le\xi\cdot(A\xi)\le\lambda_{1}\lVert\xi\rVert^{2}\qquad(\xi\in\mathbb{R}^{n}).$$

If $a\le\lambda_{n}$ and $\lambda_{1}\le b$, then, as $\lVert\xi\rVert^{2}\ge0$, the quadratic-form bounds follow. The two directions together are (ii).

Extreme eigenvalues. The last display is the quadratic-form bound with $a=\lambda_{n}$ and $b=\lambda_{1}$, so (i) gives $\lambda_{n}I_{n}\preceq A\preceq\lambda_{1}I_{n}$, which is (iii).

Operator norm. By The Operator Norm of a Symmetric Matrix: Quadratic Forms, Eigenvalues and the Semidefinite Ordering (iv), applied with $d=\lambda$, $\lVert A\rVert=\max_{k\in[n]}|\lambda_{k}|$. For each $k$ we have $\lambda_{n}\le\lambda_{k}\le\lambda_{1}$: if $\lambda_{k}\ge0$, then $|\lambda_{k}|=\lambda_{k}\le\lambda_{1}\le|\lambda_{1}|$, and if $\lambda_{k}<0$, then $|\lambda_{k}|=-\lambda_{k}\le-\lambda_{n}\le|\lambda_{n}|$. As $|\lambda_{1}|$ and $|\lambda_{n}|$ are themselves among the $|\lambda_{k}|$, the maximum is

$$\lVert A\rVert=\max\bigl(|\lambda_{1}|,|\lambda_{n}|\bigr),$$

which is (iv).