Proof of Invertibility of Symmetric Positive Definite Matrices

Since $M$ is symmetric positive definite, claims 1-2 of Triangular Orthonormalization of a Positive Definite Gram Matrix provide a lower triangular real $p\times p$ matrix $T$ with positive diagonal entries such that

with the identity matrix $I_p$, the matrix product, and the transpose, and such that $T$ is invertible. All products below are associative by Associativity of the Matrix Product, $I_p$ is a two-sided multiplicative identity by The Identity Matrix is a Two-Sided Multiplicative Identity, and we use the involutivity $(U^{\top})^{\top}=U$, immediate from the definition of the transpose.

The transpose of the inverse. Transposing the identities $TT^{-1}=I_p$ and $T^{-1}T=I_p$ and using $(UV)^{\top}=V^{\top}U^{\top}$ (claim 3 of Componentwise Estimates, Transpose Identities, and Indefinite Riemann Integrals) together with $I_p^{\top}=I_p$ gives $(T^{-1})^{\top}T^{\top}=I_p$ and $T^{\top}(T^{-1})^{\top}=I_p$. Hence $T^{\top}$ is invertible with $(T^{\top})^{-1}=(T^{-1})^{\top}$, the inverse being unique by Uniqueness of the Matrix Inverse.

Invertibility of $M$. Put $N:=T^{\top}T$. From $TMT^{\top}=I_p$, multiplying on the left by $T^{-1}$ gives $MT^{\top}=T^{-1}$, and multiplying this on the right by $T$ gives

Similarly, multiplying $TMT^{\top}=I_p$ on the right by $(T^{\top})^{-1}$ gives $TM=(T^{\top})^{-1}$, and multiplying this on the left by $T^{\top}$ gives

Hence $M$ is invertible with $M^{-1}=N=T^{\top}T$, again unique by Uniqueness of the Matrix Inverse.

Symmetry and positive definiteness of $M^{-1}$. By claim 3 of Componentwise Estimates, Transpose Identities, and Indefinite Riemann Integrals and involutivity, $N^{\top}=(T^{\top}T)^{\top}=T^{\top}(T^{\top})^{\top}=T^{\top}T=N$, so $M^{-1}$ is symmetric. For $x\in\mathbb{R}^{p}$, the identity $y\cdot(Uz)=(U^{\top}y)\cdot z$ of the same claim, applied with $U=T^{\top}$, $y=x$, $z=Tx$, together with involutivity, gives, with the dot product,

If $x\ne0$, then $Tx\ne0$: otherwise $x=T^{-1}(Tx)=T^{-1}0=0$, a contradiction. For $Tx\ne0$ some component $(Tx)^{i}$ is nonzero, so the sum of squares above is strictly positive. Hence $x\cdot(M^{-1}x)>0$ for every nonzero $x$, and $M^{-1}$ is positive definite. $\square$