The Trace as a Sum of Quadratic Forms, its Monotonicity and a Norm Bound

Throughout we work in the setting of Euclidean Space and Real Matrices: Standing Notation, whose notation is in force in the dimensions $m$ and $p$, natural numbers with $1\le m$ and $1\le p$. In addition $\operatorname{tr}$ denotes the trace of a square real matrix.

Let $A\in\mathcal{M}_{m\times p}(\mathbb{R})$, and for $k\in[m]$ let $a_{k}\in\mathbb{R}^{p}$ be the $k$th row of $A$, the point whose $j$th coordinate is $A_{kj}$ for $j\in[p]$. For $X\in\mathcal{M}_{p}(\mathbb{R})$ the matrices $A^{\top}A$ and $AX$ lie in $\mathcal{M}_{p}(\mathbb{R})$ and $\mathcal{M}_{m\times p}(\mathbb{R})$ respectively, and $(A^{\top}A)X=A^{\top}(AX)$ by Associativity of the Matrix Product, so that the notation $A^{\top}AX$ is unambiguous and denotes an element of $\mathcal{M}_{p}(\mathbb{R})$. Finally put

$$\beta_{p}=\sum_{k=1}^{p}1 ,$$

a finite sum in $\mathbb{R}$; since $0<1$ by claim 6 of Elementary Order Arithmetic in an Ordered Field, claim 6 of Properties of Finite Sums gives $1\le\beta_{p}$.

Then the following hold.

1. (Quadratic forms)(i) For every $X\in\mathcal{S}(p)$,

$$\operatorname{tr}\bigl(A^{\top}AX\bigr)=\sum_{k=1}^{m}a_{k}\cdot(Xa_{k}).$$

2. (Squared rows)(ii)

$$\operatorname{tr}\bigl(A^{\top}A\bigr)=\sum_{k=1}^{m}\lVert a_{k}\rVert^{2}.$$

3. (Trace of a symmetric matrix)(iii) For every $X\in\mathcal{S}(p)$,

$$\operatorname{tr}(X)=\sum_{k=1}^{p}e_{k}\cdot(Xe_{k}).$$

4. (Monotonicity)(iv) If $X,Y\in\mathcal{S}(p)$ satisfy $X\preceq Y$, then

$$\operatorname{tr}\bigl(A^{\top}AX\bigr)\le\operatorname{tr}\bigl(A^{\top}AY\bigr);$$

in particular $\operatorname{tr}(X)\le\operatorname{tr}(Y)$.

5. (Norm bound)(v) For every $X\in\mathcal{S}(p)$,

$$\bigl|\operatorname{tr}(X)\bigr|\le\beta_{p}\lVert X\rVert .$$