Proof of Structure of the Optimal Control Set and Separation of Near-Optimal Controls

Throughout we use the order arithmetic of Elementary Order Arithmetic in an Ordered Field. Its clauses 1 and 10 are stated for strict inequalities; the corresponding statements for $\le$, and the transitivity of $\le$, follow by treating the equality case (and, for multiplication, the case of a zero multiplier) separately, and we use them under this convention without further comment. We write $\iota:\mathbb{N}\to\mathbb{R}$ for the canonical map of $\mathbb{R}$, and $F_{\sigma}:\mathcal{U}_{\mathcal{A}}\to\mathbb{R}$ for the function $\xi\mapsto F(\sigma,\xi)$.

Call a map of the form $S(y_{0},\zeta)$, with $y_{0}\in\Delta^{l}$ and $\zeta\in\mathcal{U}_{\mathcal{A}}$, a flow. For flows $P$ and $Q$ put $E(P,Q)=\{|P_{t}-Q_{t}|:t\in[0,T]\}$ and, when the supremum exists, $\delta(P,Q)=\sup E(P,Q)$. Thus $\Psi(x_{0},\xi,\zeta)=\delta\bigl(S(x_{0},\xi),S(\sigma,\zeta)\bigr)$ and, in claim 3, $\Theta=\delta\bigl(S(x_{0},\xi),S(x_{0}',\xi')\bigr)$.

Step 1 (the suprema exist). Let $P=S(y_{0},\zeta)$ and $Q=S(y_{0}',\zeta')$ be flows. The set $E(P,Q)$ is nonempty because $0\in[0,T]$. By claim 2 of the flow stability lemma, $P_{0}=y_{0}$, $Q_{0}=y_{0}'$, and $|P_{t}-P_{0}|\le K_{b}|t-0|\le K_{b}T$ and $|Q_{t}-Q_{0}|\le K_{b}T$ for every $t\in[0,T]$. Using claim 6 of Elementary Properties of the Euclidean Norm on $\mathbb{R}^n$ twice, and claim 5 of that lemma with the scalar $-1$ to write $|Q_{0}-Q_{t}|=|Q_{t}-Q_{0}|$,

for every $t\in[0,T]$. So $E(P,Q)$ is nonempty and bounded above by the real number $2K_{b}T+|y_{0}-y_{0}'|$, and Least Upper Bound Property of the Real Numbers gives that $\delta(P,Q)=\sup E(P,Q)$ exists; it is unique by Uniqueness of the Supremum and of the Infimum. This proves the existence assertions of claims 2 and 3.

Step 2 (elementary properties of $\delta$). Let $P$, $Q$ and $R$ be flows.

(a) $0\le\delta(P,Q)$: the Euclidean norm is a nonnegative square root by its definition, so $0\le|P_{0}-Q_{0}|$, and $|P_{0}-Q_{0}|\in E(P,Q)$ while $\delta(P,Q)$ is an upper bound of $E(P,Q)$.

(b) $\delta(P,Q)=\delta(Q,P)$: by claim 5 of Elementary Properties of the Euclidean Norm on $\mathbb{R}^n$ with the scalar $-1$ we have $|P_{t}-Q_{t}|=|Q_{t}-P_{t}|$ for every $t$, so $E(P,Q)=E(Q,P)$.

(c) $\delta(P,R)\le\delta(P,Q)+\delta(Q,R)$: for every $t\in[0,T]$, claim 6 of Elementary Properties of the Euclidean Norm on $\mathbb{R}^n$ gives $|P_{t}-R_{t}|\le|P_{t}-Q_{t}|+|Q_{t}-R_{t}|\le\delta(P,Q)+\delta(Q,R)$, since each $\delta$ is an upper bound of the corresponding set. So $\delta(P,Q)+\delta(Q,R)$ is an upper bound of $E(P,R)$, and $\delta(P,R)$ is the least such.

(d) $\bigl|\delta(P,Q)-\delta(P,R)\bigr|\le\delta(Q,R)$: by (c) and (b), $\delta(P,Q)\le\delta(P,R)+\delta(R,Q)=\delta(P,R)+\delta(Q,R)$, hence $\delta(P,Q)-\delta(P,R)\le\delta(Q,R)$; exchanging $Q$ and $R$ gives $\delta(P,R)-\delta(P,Q)\le\delta(Q,R)$. By the definition of the absolute value, $|\delta(P,Q)-\delta(P,R)|$ is one of these two numbers, so it is at most $\delta(Q,R)$.

(e) If $\eta$ is a real number with $|P_{t}-Q_{t}|\le\eta$ for every $t\in[0,T]$, then $\delta(P,Q)\le\eta$, because $\eta$ is then an upper bound of $E(P,Q)$ and $\delta(P,Q)$ is the least upper bound.

Step 3 (small reciprocals). The real sequence $\bigl(\iota(n)^{-1}\bigr)_{n\in\mathbb{N}}$ has limit $0$. Indeed, let $\tau$ be a real number with $0<\tau$. By claim 3 of The Archimedean Property of the Real Numbers there is $N\in\mathbb{N}$ with $0<\iota(N)^{-1}<\tau$. Let $n\ge N$. If $n=N$ then $\iota(n)^{-1}=\iota(N)^{-1}$; if $N<n$ then $\iota(N)<\iota(n)$ by claim 6 of Properties of the Canonical Map from the Natural Numbers to an Ordered Field. In either case $\iota(N)\le\iota(n)$, and both are positive by claim 3 of that lemma. Multiplying by the positive number $\iota(n)^{-1}$ gives $\iota(N)\,\iota(n)^{-1}\le1$, and multiplying that by the positive number $\iota(N)^{-1}$ gives $\iota(n)^{-1}\le\iota(N)^{-1}$. Since $\iota(n)^{-1}$ is positive it equals its own absolute value, so $|\iota(n)^{-1}-0|=\iota(n)^{-1}\le\iota(N)^{-1}<\tau$ for every $n\ge N$, as required.

Claim 1. Since $J^{*}_{\sigma}$ is a lower bound of $V_{\sigma}$ and $F(\sigma,\xi)\in V_{\sigma}$ for every $\xi\in\mathcal{U}_{\mathcal{A}}$, we have $J^{*}_{\sigma}\le F(\sigma,\xi)$ for every such $\xi$. Hence for $\xi\in\mathcal{U}_{\mathcal{A}}$ the equality $F(\sigma,\xi)=J^{*}_{\sigma}$ holds if and only if $F(\sigma,\xi)\le J^{*}_{\sigma}$, which is the stated description of $\mathcal{M}^{*}_{\sigma}$.

By claim 5 of the boundedness, lower semicontinuity and attainment theorem there is $\xi^{*}\in\mathcal{U}_{\mathcal{A}}$ with $F(\sigma,\xi^{*})\le F(\sigma,\xi)$ for every $\xi\in\mathcal{U}_{\mathcal{A}}$. Then $F(\sigma,\xi^{*})$ is a lower bound of $V_{\sigma}$, and since $J^{*}_{\sigma}$ is the greatest lower bound, $F(\sigma,\xi^{*})\le J^{*}_{\sigma}$. By the description just proved, $\xi^{*}\in\mathcal{M}^{*}_{\sigma}$, so $\mathcal{M}^{*}_{\sigma}$ is nonempty.

By claim 4 of the same theorem, $F_{\sigma}$ is lower semicontinuous on $\mathcal{U}_{\mathcal{A}}$ for $\rho$. Applying claim 3 of Semicontinuity via Sublevel and Superlevel Sets with the metric space $(\mathcal{U}_{\mathcal{A}},\rho)$, the subset $A=\mathcal{U}_{\mathcal{A}}$ and the value $c=J^{*}_{\sigma}$ shows that $\{\xi\in\mathcal{U}_{\mathcal{A}}:F_{\sigma}(\xi)\le J^{*}_{\sigma}\}=\mathcal{M}^{*}_{\sigma}$ is closed in that topological space.

By claim 3 of the weak metrizability and compactness theorem, $\mathcal{U}_{\mathcal{A}}$ is a compact subset of $(\mathcal{U}_{\mathcal{A}},\rho)$. Applying claim 3 of Compactness of Intersections with Closed Sets and of Level Sets of Semicontinuous Functions with $(X,d)=(\mathcal{U}_{\mathcal{A}},\rho)$, $K=\mathcal{U}_{\mathcal{A}}$, $w=F_{\sigma}$ and $c=J^{*}_{\sigma}$ shows that $\mathcal{M}^{*}_{\sigma}$ is compact in $(\mathcal{U}_{\mathcal{A}},\rho)$, and Compactness and Sequential Compactness Agree for Subsets of a Metric Space then gives that it is sequentially compact. This proves claim 1.

Claim 2. The existence of every $\Psi(x_{0},\xi,\zeta)$ is Step 1. Fix $x_{0}\in\Delta^{l}$ and $\xi\in\mathcal{U}_{\mathcal{A}}$, write $P=S(x_{0},\xi)$, and define $\psi:\mathcal{M}^{*}_{\sigma}\to\mathbb{R}$ by $\psi(\zeta)=\Psi(x_{0},\xi,\zeta)=\delta\bigl(P,S(\sigma,\zeta)\bigr)$.

We show that $\psi$ is lower semicontinuous on $\mathcal{M}^{*}_{\sigma}$, viewed as a subset of the metric space $(\mathcal{U}_{\mathcal{A}},\rho)$, by verifying at each point the sequential condition of claim 1 of the sequential characterization of lower semicontinuity and then invoking claim 3 of that lemma.

Let $\zeta\in\mathcal{M}^{*}_{\sigma}$, let $(\zeta_{j})_{j\in\mathbb{N}}$ be a sequence in $\mathcal{M}^{*}_{\sigma}$ converging to $\zeta$ in the restriction of $\rho$ to $\mathcal{M}^{*}_{\sigma}$, and let $\varepsilon$ be a real number with $0<\varepsilon$. Convergence in the restricted metric is convergence in $\rho$, so claim 2 of the weak metrizability and compactness theorem gives $\zeta_{j}\rightharpoonup\zeta$ in the sense of weak convergence. The constant sequence all of whose terms are $\sigma$ satisfies $|\sigma-\sigma|=0$, so the real sequence of these distances has limit $0$. By claim 8 of Elementary Order Arithmetic in an Ordered Field pick a real $\varepsilon_{1}$ with $0<\varepsilon_{1}$ and $\varepsilon_{1}+\varepsilon_{1}=\varepsilon$; then $\varepsilon_{1}<\varepsilon$. Claim 6 of the flow stability lemma applies to the constant sequence of initial states and to $(\zeta_{j})$, and yields $N\in\mathbb{N}$ with $|S_{t}(\sigma,\zeta_{j})-S_{t}(\sigma,\zeta)|\le\varepsilon_{1}$ for every $t\in[0,T]$ and every $j\ge N$; by Step 2(e), $\delta\bigl(S(\sigma,\zeta_{j}),S(\sigma,\zeta)\bigr)\le\varepsilon_{1}$ for such $j$.

By Step 2(d) with $Q=S(\sigma,\zeta_{j})$ and $R=S(\sigma,\zeta)$, and by claim 3 of Properties of the Absolute Value in an Ordered Field,

for every $j\ge N$, whence $\psi(\zeta)-\varepsilon<\psi(\zeta_{j})$ for every $j\ge N$. This is the sequential condition at $\zeta$; since $\zeta$ was arbitrary, $\psi$ is lower semicontinuous on $\mathcal{M}^{*}_{\sigma}$.

By claim 1, $\mathcal{M}^{*}_{\sigma}$ is a nonempty compact subset of $(\mathcal{U}_{\mathcal{A}},\rho)$. Claim 2 of Semicontinuous Functions Attain Their Extrema on a Compact Set, applied with the metric space $(\mathcal{U}_{\mathcal{A}},\rho)$, the set $K=\mathcal{M}^{*}_{\sigma}$ and the function $\psi$, gives $\zeta^{\dagger}\in\mathcal{M}^{*}_{\sigma}$ with $\psi(\zeta^{\dagger})\le\psi(\zeta)$ for every $\zeta\in\mathcal{M}^{*}_{\sigma}$. If $\zeta^{\dagger}_{1}$ and $\zeta^{\dagger}_{2}$ both have this property, then $\psi(\zeta^{\dagger}_{1})\le\psi(\zeta^{\dagger}_{2})$ and $\psi(\zeta^{\dagger}_{2})\le\psi(\zeta^{\dagger}_{1})$, so the two values are equal by antisymmetry of the order. Hence $D(x_{0},\xi)$ is well defined, and $0\le D(x_{0},\xi)$ by Step 2(a).

Finally let $\xi^{*}\in\mathcal{M}^{*}_{\sigma}$ and consider $D(\sigma,\xi^{*})$. Every element of $E\bigl(S(\sigma,\xi^{*}),S(\sigma,\xi^{*})\bigr)$ equals $|S_{t}(\sigma,\xi^{*})-S_{t}(\sigma,\xi^{*})|=0$, so $\Psi(\sigma,\xi^{*},\xi^{*})=0$. Since $\xi^{*}\in\mathcal{M}^{*}_{\sigma}$ is one of the competitors in the minimum defining $D(\sigma,\xi^{*})$, we get $D(\sigma,\xi^{*})\le0$, and with $0\le D(\sigma,\xi^{*})$ this gives $D(\sigma,\xi^{*})=0$. This proves claim 2.

Claim 3. The existence of $\Theta$ is Step 1. Write $P=S(x_{0},\xi)$ and $P'=S(x_{0}',\xi')$, so $\Theta=\delta(P,P')$. By claim 2 there is $\zeta^{\dagger}\in\mathcal{M}^{*}_{\sigma}$ with $D(x_{0}',\xi')=\delta\bigl(P',S(\sigma,\zeta^{\dagger})\bigr)$. Since $D(x_{0},\xi)$ is a minimum over $\mathcal{M}^{*}_{\sigma}$ and $\zeta^{\dagger}\in\mathcal{M}^{*}_{\sigma}$, and using Step 2(c),

so $D(x_{0},\xi)-D(x_{0}',\xi')\le\Theta$. Exchanging the roles of $(x_{0},\xi)$ and $(x_{0}',\xi')$ and using $\delta(P',P)=\delta(P,P')=\Theta$ from Step 2(b) gives $D(x_{0}',\xi')-D(x_{0},\xi)\le\Theta$. By the definition of the absolute value, $|D(x_{0},\xi)-D(x_{0}',\xi')|$ is one of these two numbers, hence at most $\Theta$. This proves claim 3.

Claim 4. By claim 1 of Coordinatewise Convergence, Sequential Compactness and Density in a Product Metric Space the sequence $(x^{j}_{0})_{j\in\mathbb{N}}$ converges to $x_{0}$ in $(\Delta^{l},d_{\Delta})$ and $(\xi_{j})_{j\in\mathbb{N}}$ converges to $\xi$ in $(\mathcal{U}_{\mathcal{A}},\rho)$. As in the proof of claim 3 of the boundedness, lower semicontinuity and attainment theorem, the first statement says that the real sequence $(|x^{j}_{0}-x_{0}|)_{j\in\mathbb{N}}$ has limit $0$, and by claim 2 of the weak metrizability and compactness theorem the second gives $\xi_{j}\rightharpoonup\xi$.

Let $\tau$ be a real number with $0<\tau$, and by claim 8 of Elementary Order Arithmetic in an Ordered Field pick a real $\tau_{1}$ with $0<\tau_{1}$ and $\tau_{1}+\tau_{1}=\tau$, so $\tau_{1}<\tau$. By claim 6 of the flow stability lemma there is $N\in\mathbb{N}$ with $|S_{t}(x^{j}_{0},\xi_{j})-S_{t}(x_{0},\xi)|\le\tau_{1}$ for every $t\in[0,T]$ and every $j\ge N$. By Step 2(e), $\delta\bigl(S(x^{j}_{0},\xi_{j}),S(x_{0},\xi)\bigr)\le\tau_{1}$ for such $j$, and claim 3 then gives

Hence the real sequence $\bigl(D(x^{j}_{0},\xi_{j})\bigr)_{j\in\mathbb{N}}$ has limit $D(x_{0},\xi)$, which is claim 4.

Claim 5. Suppose, for contradiction, that the conclusion fails for some real $\varepsilon>0$; that is, for every real $\eta>0$ there is $\xi\in\mathcal{U}_{\mathcal{A}}$ with $F(\sigma,\xi)\le J^{*}_{\sigma}+\eta$ for which $D(\sigma,\xi)<\varepsilon$ fails. Since $\le$ is a total order, the failure means $\varepsilon\le D(\sigma,\xi)$.

For $n\in\mathbb{N}$ put

By claim 3 of Properties of the Canonical Map from the Natural Numbers to an Ordered Field we have $0<\iota(n)^{-1}$, so applying the previous paragraph with $\eta=\iota(n)^{-1}$ shows that each $B_{n}$ is nonempty; and every $B_{n}$ is a subset of the one set $\mathcal{U}_{\mathcal{A}}$. Hence Axiom of Countable Choice furnishes a sequence $(\xi_{n})_{n\in\mathbb{N}}$ with $\xi_{n}\in B_{n}$ for every $n\in\mathbb{N}$.

By claim 1 we have $J^{*}_{\sigma}\le F(\sigma,\xi_{n})$, and by the definition of $B_{n}$ we have $F(\sigma,\xi_{n})\le J^{*}_{\sigma}+\iota(n)^{-1}$. Hence $0\le F(\sigma,\xi_{n})-J^{*}_{\sigma}\le\iota(n)^{-1}$, so $|F(\sigma,\xi_{n})-J^{*}_{\sigma}|\le\iota(n)^{-1}$ by the definition of the absolute value. By Step 3 and claim 3 of Order Properties of Limits of Real Sequences the real sequence $\bigl(F(\sigma,\xi_{n})-J^{*}_{\sigma}\bigr)_{n\in\mathbb{N}}$ has limit $0$; adding the constant sequence with terms $J^{*}_{\sigma}$ and using claim 1 of Arithmetic of Limits of Real Sequences, the real sequence $\bigl(F(\sigma,\xi_{n})\bigr)_{n\in\mathbb{N}}$ has limit $J^{*}_{\sigma}$.

By claim 3 of the weak metrizability and compactness theorem, $\mathcal{U}_{\mathcal{A}}$ is sequentially compact in $(\mathcal{U}_{\mathcal{A}},\rho)$, so there are a subsequence $(\xi_{n_{k}})_{k\in\mathbb{N}}$ and $\xi_{\infty}\in\mathcal{U}_{\mathcal{A}}$ such that $(\xi_{n_{k}})_{k\in\mathbb{N}}$ converges to $\xi_{\infty}$ in $(\mathcal{U}_{\mathcal{A}},\rho)$.

By Convergence and the Cauchy Condition for Real Sequences Agree with Those in the Real Line as a Metric Space the sequence $\bigl(F(\sigma,\xi_{n})\bigr)_{n\in\mathbb{N}}$ converges to $J^{*}_{\sigma}$ in the metric space $(\mathbb{R},d_{\mathbb{R}})$ determined by the absolute-value metric; by A Subsequence of a Convergent Sequence Has the Same Limit the subsequence $\bigl(F(\sigma,\xi_{n_{k}})\bigr)_{k\in\mathbb{N}}$, formed with the same index map, converges to $J^{*}_{\sigma}$ there; and by Convergence and the Cauchy Condition for Real Sequences Agree with Those in the Real Line as a Metric Space again it has limit $J^{*}_{\sigma}$ as a real sequence. By claim 1 of the boundedness, lower semicontinuity and attainment theorem we have $|F(\sigma,\xi_{n_{k}})|\le C_{F}<C_{F}+1$ for every $k$, so this subsequence is a bounded sequence and claim 5 of Basic Properties of the Limit Inferior and Limit Superior of a Bounded Real Sequence gives $\liminf_{k}F(\sigma,\xi_{n_{k}})=J^{*}_{\sigma}$.

By claim 4 of the same theorem, $F_{\sigma}$ is lower semicontinuous on $\mathcal{U}_{\mathcal{A}}$ for $\rho$, in particular at $\xi_{\infty}$. Claim 2 of the sequential characterization of lower semicontinuity, applied at $\xi_{\infty}$ to the sequence $(\xi_{n_{k}})_{k\in\mathbb{N}}$, therefore gives

By the description of $\mathcal{M}^{*}_{\sigma}$ in claim 1 this means $\xi_{\infty}\in\mathcal{M}^{*}_{\sigma}$, and then $D(\sigma,\xi_{\infty})=0$ by claim 2.

The constant sequence all of whose terms are $\sigma$ converges to $\sigma$ in $(\Delta^{l},d_{\Delta})$, because $d_{\Delta}(\sigma,\sigma)=0$. So by claim 1 of Coordinatewise Convergence, Sequential Compactness and Density in a Product Metric Space the sequence $\bigl((\sigma,\xi_{n_{k}})\bigr)_{k\in\mathbb{N}}$ converges to $(\sigma,\xi_{\infty})$ in $(X,d_{X})$, and claim 4 shows that the real sequence $\bigl(D(\sigma,\xi_{n_{k}})\bigr)_{k\in\mathbb{N}}$ has limit $D(\sigma,\xi_{\infty})=0$. Hence there is $k\in\mathbb{N}$ with $|D(\sigma,\xi_{n_{k}})-0|<\varepsilon$; since $0\le D(\sigma,\xi_{n_{k}})$ by claim 2, this number equals its own absolute value by the definition of the absolute value, and we get $D(\sigma,\xi_{n_{k}})<\varepsilon$. But $\xi_{n_{k}}\in B_{n_{k}}$ gives $\varepsilon\le D(\sigma,\xi_{n_{k}})$, so $\varepsilon\le D(\sigma,\xi_{n_{k}})<\varepsilon$ and therefore $\varepsilon<\varepsilon$, which is impossible.

This contradiction proves claim 5.