Proof of Theorem on Sums for Two Upper Semicontinuous Functions

Throughout, the notation is that of the statement. Convergence of a sequence in $\mathbb{R}^{q}$ refers to $d_{E}$ and convergence in $\mathcal{S}(q)$ to $d_{\mathcal{S}(q)}$, as fixed in Real Matrices, Symmetric Matrices and the Semidefinite Ordering: Standing Notation §norm. A constant sequence in a metric space converges to its value, directly from Convergent Sequence in a Metric Space, since the distance from that value to itself is $0$.

Step 0 (two elementary observations).

(0a) Inverses reverse the order. If $a,b\in\mathbb{R}$ satisfy $0<a\le b$, then $b^{-1}\le a^{-1}$. Indeed $a^{-1}$ and $b^{-1}$ exist and are positive by claim 7 of Elementary Order Arithmetic in an Ordered Field, so $c=a^{-1}b^{-1}$ is positive by claim 5 there; if $a<b$ then $ca<cb$ by claim 10 there, and $ca=b^{-1}(a^{-1}a)=b^{-1}$ while $cb=a^{-1}(b^{-1}b)=a^{-1}$, so $b^{-1}<a^{-1}$; while if $a=b$ then $a^{-1}=b^{-1}$.

(0b) A null sequence. Let $\iota_{\mathbb{R}}:\mathbb{N}\to\mathbb{R}$ be the canonical map of $\mathbb{R}$ and put $\theta_{k}=\iota_{\mathbb{R}}(k)^{-1}$ for $k\in\mathbb{N}$. By claim 3 of Properties of the Canonical Map from the Natural Numbers to an Ordered Field each $\theta_{k}$ exists and is positive, and by claim 2 there $1\le\iota_{\mathbb{R}}(k)$, so $\theta_{k}\le 1^{-1}=1$ by (0a). Moreover $(\theta_{k})_{k\in\mathbb{N}}$ converges to $0$ in $\mathbb{R}$: given a positive $\varepsilon'\in\mathbb{R}$, claim 3 of The Archimedean Property of the Real Numbers provides $p\in\mathbb{N}$ with $0<\iota_{\mathbb{R}}(p)^{-1}<\varepsilon'$, and for $k\in\mathbb{N}$ with $p\le k$ we have $\iota_{\mathbb{R}}(p)\le\iota_{\mathbb{R}}(k)$ by claim 6 of Properties of the Canonical Map from the Natural Numbers to an Ordered Field when $p<k$ and trivially when $p=k$, whence $\theta_{k}\le\iota_{\mathbb{R}}(p)^{-1}<\varepsilon'$ by (0a).

Step 1 (the reduction and the global theorem, for each $k$). Fix $k\in\mathbb{N}$ and put $A_{k}=A+\theta_{k}I_{N}$, which lies in $\mathcal{S}(N)$ by Real Matrices, Symmetric Matrices and the Semidefinite Ordering: Standing Notation §symmetric. The hypotheses of Reduction of the Theorem on Sums to a Global Quadratic Bound hold with the present data and with $\theta=\theta_{k}$, so it provides a positive $r_{k}$, which may depend on $k$ and is not used below, and functions $v^{(k)}_{1}:\mathbb{R}^{n_{1}}\to\mathbb{R}$ and $v^{(k)}_{2}:\mathbb{R}^{n_{2}}\to\mathbb{R}$ satisfying its three claims. By Reduction of the Theorem on Sums to a Global Quadratic Bound §localised each $v^{(k)}_{i}$ is upper semicontinuous on $\mathbb{R}^{n_{i}}$, its set of values has an upper bound in $\mathbb{R}$, and $v^{(k)}_{i}(0_{\mathbb{R}^{n_{i}}})=0$; and by Reduction of the Theorem on Sums to a Global Quadratic Bound §bound,

These are exactly the hypotheses of The Theorem on Sums for a Global Quadratic Bound, in Two Groups of Variables, applied in the dimensions $n_{1}$ and $n_{2}$ with the functions $v^{(k)}_{1}$ and $v^{(k)}_{2}$, the matrix $A_{k}$ and the given positive $\varepsilon$. It provides $Y^{(k)}_{1}\in\mathcal{S}(n_{1})$ and $Y^{(k)}_{2}\in\mathcal{S}(n_{2})$ such that, by The Theorem on Sums for a Global Quadratic Bound, in Two Groups of Variables §test-data, the quadruple $\bigl(0_{\mathbb{R}^{n_{i}}},v^{(k)}_{i}(0_{\mathbb{R}^{n_{i}}}),0_{\mathbb{R}^{n_{i}}},Y^{(k)}_{i}\bigr)$ is approximable by test data from above for $v^{(k)}_{i}$ for $i\in\{1,2\}$, and, by The Theorem on Sums for a Global Quadratic Bound, in Two Groups of Variables §bounds,

Applying Reduction of the Theorem on Sums to a Global Quadratic Bound §transfer to $Y^{(k)}_{i}$ we obtain, for every $k\in\mathbb{N}$ and $i\in\{1,2\}$, that

Step 2 (the two sides of (1) converge). Write $E_{k}=-\bigl(\varepsilon^{-1}+\lVert A_{k}\rVert\bigr)I_{N}$, $E=-\bigl(\varepsilon^{-1}+\lVert A\rVert\bigr)I_{N}$, $C_{k}=A_{k}+\varepsilon A_{k}^{2}$ and $C=A+\varepsilon A^{2}$; the matrices $C_{k}$ and $C$ lie in $\mathcal{S}(N)$ by A Weighted Young Inequality and the Splitting of a Quadratic Form §matrix. Note that $A_{k}-A=\theta_{k}I_{N}$, so $\lVert A_{k}-A\rVert=\theta_{k}$ by Vector, Entry and Comparison Bounds for the Norm of a Symmetric Real Matrix §identity, and hence, by claim 5 of Properties of the Norm of a Symmetric Real Matrix,

so that $\bigl|\lVert A_{k}\rVert-\lVert A\rVert\bigr|\le\theta_{k}$ by claim 6 of Properties of the Absolute Value in an Ordered Field.

The lower bounds converge. We have $E_{k}-E=\bigl(\lVert A\rVert-\lVert A_{k}\rVert\bigr)I_{N}$, so by Vector, Entry and Comparison Bounds for the Norm of a Symmetric Real Matrix §identity

and since $(\theta_{k})$ converges to $0$, the sequence $(E_{k})$ converges to $E$ in $\mathcal{S}(N)$.

The upper bounds converge. Let $z\in\mathbb{R}^{N}$ with $\lVert z\rVert\le1$. By Vector, Entry and Comparison Bounds for the Norm of a Symmetric Real Matrix §square we have $z\cdot(A_{k}^{2}z)=\lVert A_{k}z\rVert^{2}$ and $z\cdot(A^{2}z)=\lVert Az\rVert^{2}$. Applying Vector, Entry and Comparison Bounds for the Norm of a Symmetric Real Matrix §quadratic-comparison with the matrix $I_{N}$ to the points $A_{k}z$ and $Az$, and using Vector, Entry and Comparison Bounds for the Norm of a Symmetric Real Matrix §identity in the case $a=1$, which gives $y\cdot(I_{N}y)=\lVert y\rVert^{2}$ and $\lVert I_{N}\rVert=1$,

By claims 1 and 2 of Linearity of the Matrix-Vector Product and the Quadratic Form as a Double Sum, $A_{k}z-Az=(\theta_{k}I_{N})z=\theta_{k}z$, so $\lVert A_{k}z-Az\rVert=\theta_{k}\lVert z\rVert\le\theta_{k}$ by claim 5 of Elementary Properties of the Euclidean Norm on $\mathbb{R}^n$; and by claim 6 of Elementary Properties of the Euclidean Norm on $\mathbb{R}^n$ together with Vector, Entry and Comparison Bounds for the Norm of a Symmetric Real Matrix §vector-bound,

Since $A_{k}^{2}-A^{2}\in\mathcal{S}(N)$ and, by claim 1 of Linearity of the Matrix-Vector Product and the Quadratic Form as a Double Sum and claim 5 of Bilinearity and Symmetry of the Dot Product on $\mathbb{R}^n$, $z\cdot\bigl((A_{k}^{2}-A^{2})z\bigr)=z\cdot(A_{k}^{2}z)-z\cdot(A^{2}z)$, the three displays give

As $\lVert A_{k}^{2}-A^{2}\rVert$ is the least upper bound of the numbers on the left, by Real Matrices, Symmetric Matrices and the Semidefinite Ordering: Standing Notation §norm, we get $\lVert A_{k}^{2}-A^{2}\rVert\le\theta_{k}(2\lVert A\rVert+1)$. Hence, by claim 5 of Properties of the Norm of a Symmetric Real Matrix,

where $L=1+\varepsilon(2\lVert A\rVert+1)$ is positive. Since $(\theta_{k})$ converges to $0$, the sequence $(C_{k})$ converges to $C$ in $\mathcal{S}(N)$.

Step 3 (a convergent subsequence of matrices). By Limits and Bounded Sequences of Symmetric Real Matrices §order-bound, applied to (1) with the nonnegative number $\varepsilon^{-1}+\lVert A_{k}\rVert$ and the matrix $C_{k}$,

using $\lVert C_{k}\rVert\le\lVert C\rVert+\lVert C_{k}-C\rVert\le\lVert C\rVert+L\theta_{k}\le\lVert C\rVert+L$ from claim 5 of Properties of the Norm of a Symmetric Real Matrix and $\theta_{k}\le1$. By Block Diagonal Symmetric Matrices and the Blocks of a Symmetric Matrix §norm the number $\lVert Y^{(k)}_{1}\oplus Y^{(k)}_{2}\rVert$ is the larger of $\lVert Y^{(k)}_{1}\rVert$ and $\lVert Y^{(k)}_{2}\rVert$, so both of these are at most $R$, for every $k$.

By Limits and Bounded Sequences of Symmetric Real Matrices §compactness, applied to the sequence $(Y^{(k)}_{1})_{k\in\mathbb{N}}$ in $\mathcal{S}(n_{1})$ with the bound $R$, there are a strictly increasing $\kappa_{1}:\mathbb{N}\to\mathbb{N}$ and $X_{1}\in\mathcal{S}(n_{1})$ such that $\bigl(Y^{(\kappa_{1}(j))}_{1}\bigr)_{j\in\mathbb{N}}$ converges to $X_{1}$. Applying it again to the sequence $\bigl(Y^{(\kappa_{1}(j))}_{2}\bigr)_{j\in\mathbb{N}}$ in $\mathcal{S}(n_{2})$, which is also bounded by $R$, there are a strictly increasing $\kappa_{2}:\mathbb{N}\to\mathbb{N}$ and $X_{2}\in\mathcal{S}(n_{2})$ such that $\bigl(Y^{(\kappa_{1}(\kappa_{2}(l)))}_{2}\bigr)_{l\in\mathbb{N}}$ converges to $X_{2}$.

Put $k_{l}=\kappa_{1}(\kappa_{2}(l))$ for $l\in\mathbb{N}$; the sequence $(k_{l})_{l\in\mathbb{N}}$ is strictly increasing by claim 2 of A Subsequence of a Subsequence is a Subsequence. By A Subsequence of a Convergent Sequence Has the Same Limit, the sequence $\bigl(Y^{(k_{l})}_{1}\bigr)_{l\in\mathbb{N}}$, being a subsequence of $\bigl(Y^{(\kappa_{1}(j))}_{1}\bigr)_{j\in\mathbb{N}}$, converges to $X_{1}$; and $\bigl(Y^{(k_{l})}_{2}\bigr)_{l\in\mathbb{N}}$ converges to $X_{2}$. By the same lemma the sequences $(E_{k_{l}})_{l\in\mathbb{N}}$ and $(C_{k_{l}})_{l\in\mathbb{N}}$ converge to $E$ and to $C$ respectively.

Step 4 (proof of claim 1). Fix $i\in\{1,2\}$. For every $l\in\mathbb{N}$ the quadruple $\bigl(\hat{x}_{i},u_{i}(\hat{x}_{i}),p_{i},Y^{(k_{l})}_{i}\bigr)$ is approximable by test data from above for $u_{i}$, by (2). Consider the constant sequences with values $\hat{x}_{i}$, $u_{i}(\hat{x}_{i})$ and $p_{i}$, which converge to $\hat{x}_{i}$ in $\mathbb{R}^{n_{i}}$, to $u_{i}(\hat{x}_{i})$ in $\mathbb{R}$ and to $p_{i}$ in $\mathbb{R}^{n_{i}}$, together with the sequence $\bigl(Y^{(k_{l})}_{i}\bigr)_{l\in\mathbb{N}}$, which converges to $X_{i}$ in $\mathcal{S}(n_{i})$ by Step 3. By Quadratic Test Functions, Limits, Translation and Locality for Approximability by Test Data §limits, applied with $U=\Omega_{i}$, $u=u_{i}$, $x_{0}=\hat{x}_{i}$, $p=p_{i}$ and the matrix $X_{i}$, the quadruple

is approximable by test data from above for $u_{i}$, the domain being $\Omega_{i}$. This is claim 1.

Step 5 (proof of claim 2). By Block Diagonal Symmetric Matrices and the Blocks of a Symmetric Matrix §norm the sequence $\bigl(Y^{(k_{l})}_{1}\oplus Y^{(k_{l})}_{2}\bigr)_{l\in\mathbb{N}}$ converges to $X_{1}\oplus X_{2}$ in $\mathcal{S}(N)$, since its two block sequences converge to $X_{1}$ and to $X_{2}$. By (1) we have $E_{k_{l}}\preceq Y^{(k_{l})}_{1}\oplus Y^{(k_{l})}_{2}$ and $Y^{(k_{l})}_{1}\oplus Y^{(k_{l})}_{2}\preceq C_{k_{l}}$ for every $l$, while $(E_{k_{l}})$ converges to $E$ and $(C_{k_{l}})$ converges to $C$ by Step 3. Two applications of Limits and Bounded Sequences of Symmetric Real Matrices §closed therefore give

which is claim 2.