Proof of Convergence of Corrections and Costs along Approximating Sequences of an Extended Admissible Control

Throughout, a real-valued function on a subinterval $I$ of the real numbers $\mathbb{R}$ is called continuous on $I$ when it is continuous relative to $I$, both $I$ and the codomain $\mathbb{R}$ carrying the metric of the real line.

Also throughout, $\lVert\cdot\rVert_{2}$ is the mean-square norm of Square-Integrable Random Variables and the Mean-Square Inner Product, $d$ and $g_{\beta,\gamma}(t):=\sum_{\kappa}\lVert\beta^{\kappa}_t-\gamma^{\kappa}_t\rVert_{2}^{2}$ are as in Almost-Everywhere Mean-Square Limits of Cauchy Sequences of Admissible Controls, and claims 3, 4, 6 of the interval toolkit on $[0,T]$ are used freely to pass between Riemann and Lebesgue integrals of continuous integrands and to apply the integral Cauchy-Schwarz inequality; $\lambda:=\lambda_{[0,T]}$. We record for repeated use: (i) for square-integrable $U,V$, $\bigl|\lVert U\rVert_{2}-\lVert V\rVert_{2}\bigr|\le\lVert U-V\rVert_{2}$, from claim 2 of Cauchy-Schwarz and Triangle Inequalities for the Mean-Square Norm; (ii) $\lVert U\rVert_{2}=0$ implies $U=0$ almost surely, by Markov's and Chebyshev's Inequalities applied to $U^{2}$ as in the proof of Almost-Everywhere Mean-Square Limits of Cauchy Sequences of Admissible Controls; (iii) for continuous $\varphi\ge0$ on $[0,T]$ and $t\in[0,T]$, $\int_0^t\varphi\,dr\le\int_0^T\varphi\,dr$: by claims 2 and 3 of Restricted Lebesgue Measure and Integral Toolkit on a Compact Interval both Riemann integrals equal Lebesgue integrals over $[0,T]$ of $\mathbf{1}_{[0,t]}\varphi$ and of $\varphi$ respectively (the zero extensions to the real line of $\varphi$ restricted to $[0,t]$ and of $\mathbf{1}_{[0,t]}\varphi$ coincide), and monotonicity is claim 1 of Linearity and Monotonicity of the Lebesgue Integral.

Claim 1. Let $\beta$ be admissible. Each $\beta_t$ is a tuple of square-integrable random variables satisfying condition (i) of Extended Admissible Control for the Linear-Gaussian State-Observation Model by condition (ii) of Admissible Control for the Linear-Gaussian State-Observation Model. For the constant sequence $\beta^{(n)}:=\beta$ and $D:=[0,T]$: $d(\beta^{(n)},\beta^{(m)})=0$ for all $n,m$, so the sequence is Cauchy for $d$; $D$ is co-null; and $\lVert\beta^{(n),\kappa}_t-\beta^{\kappa}_t\rVert_{2}=0\to0$ for every $t,\kappa$. Hence $\beta$ is extended admissible with this approximating sequence.

Step 1 (uniform correction estimate). Let $\beta,\gamma$ be admissible controls with correction processes $c^{\beta},c^{\gamma}$ as in Superposition Decomposition of the Controlled State and Observations, built from $\Phi$ and $\Psi=\Phi^{-1}$ of Fundamental Solution and Variation of Constants for Linear Ordinary Differential Equations. We show there is a constant $C_0$, depending only on $l,k,T$ and the entries of $\Phi,\Psi,B$, with

Write $h^{\beta},h^{\gamma}$ for the inner integrals in Superposition Decomposition of the Controlled State and Observations. By linearity of the mean-square Riemann integral (claim 1 of Basic Properties of the Mean-Square Riemann Integral), componentwise and almost surely $h^{\beta,j}_t-h^{\gamma,j}_t=\int_0^t\bigl(\Psi(r)B(r)(\beta_r-\gamma_r)\bigr)^{j}\,dr$, the integrand family being mean-square continuous by claims 1-2 there. By the norm bound over $[0,t]$ (claim 6 of Basic Properties of the Mean-Square Riemann Integral), the entrywise expansion of the matrix-vector product, and fact (iii) of the preamble applied to the continuous nonnegative integrand,

where $M$ bounds the entries of $\Psi B$ on $[0,T]$; the entries are continuous as finite sums of products of continuous functions (claim 5 of Continuity of Sums and Products of Real-Valued Functions on a Metric Space), so $M$ exists by Extreme Value Theorem on a Closed Real Interval. The integrand $r\mapsto\sum_{\kappa}\lVert\beta^{\kappa}_r-\gamma^{\kappa}_r\rVert_{2}$ is continuous (fact (i) of the preamble and mean-square continuity), and expanding the square of the sum with $2xy\le x^{2}+y^{2}$ gives $\bigl(\sum_{\kappa}\lVert\cdot\rVert_{2}\bigr)^{2}\le k\,g_{\beta,\gamma}$; hence by claims 3 and 4 of Restricted Lebesgue Measure and Integral Toolkit on a Compact Interval,

Since $c^{\beta}_t-c^{\gamma}_t=\Phi(t)(h^{\beta}_t-h^{\gamma}_t)$ componentwise and almost surely, bounding the entries of $\Phi$ by some $M'$ (same argument) and summing over $i$ yields the estimate with $C_0:=l\,M'\,l\,M\,(Tk)^{1/2}$ — one factor $l$ because each component of $\Phi(t)(h^{\beta}_t-h^{\gamma}_t)$ is a sum of $l$ terms, and one because we then sum over the $l$ components $i$.

Taking $\gamma$ the zero control (each $\gamma_t$ the zero tuple, which is admissible: mean-square continuous and $\mathcal{G}_t$-measurable), whose correction vanishes ($h^{\gamma}=0$ almost surely by claim 2 of Basic Properties of the Mean-Square Riemann Integral with $Z=0$), gives the a priori bound $\sum_i\lVert c^{\beta,i}_t\rVert_{2}\le C_0N_{\beta}$ for every $t$, where $N_{\beta}:=\bigl(\int_0^Tg_{\beta,0}\,dt\bigr)^{1/2}=d(\beta,0)$.

Claim 2. Let $\bigl((\alpha^{(n)}),D\bigr)$ be an approximating sequence for $\alpha$, with corrections $c^{(n)}$. (Existence.) Fix $t\in[0,T]$ and $i$. By Step 1, $\lVert c^{(n),i}_t-c^{(m),i}_t\rVert_{2}\le C_0\,d(\alpha^{(n)},\alpha^{(m)})$, so $(c^{(n),i}_t)_n$ is Cauchy in mean square, uniformly in $t$. By claim 3 of Superposition Decomposition of the Controlled State and Observations each $c^{(n),i}_t$ is almost surely equal to a $\mathcal{G}_t$-measurable square-integrable random variable; almost sure equality preserves mean-square distances, so Mean-Square Completeness of Square-Integrable Random Variables (Riesz-Fischer), applied with the sub-$\sigma$-algebra $\mathcal{G}_t$, yields a $\mathcal{G}_t$-measurable square-integrable $c^{\alpha,i}_t$ with $\lVert c^{(n),i}_t-c^{\alpha,i}_t\rVert_{2}\to0$. Given $\varepsilon>0$ choose $N$ with $C_0\,d(\alpha^{(n)},\alpha^{(m)})<\varepsilon$ for $n,m\ge N$; letting $m\to\infty$ in $\lVert c^{(n),i}_t-c^{(m),i}_t\rVert_{2}<\varepsilon$ (the norms converge by fact (i) of the preamble) gives

(a) Mean-square continuity: given $\varepsilon>0$ pick $n\ge N$ as in $(*)$; for $s,t\in[0,T]$, $\lVert c^{\alpha,i}_t-c^{\alpha,i}_s\rVert_{2}\le2\varepsilon+\lVert c^{(n),i}_t-c^{(n),i}_s\rVert_{2}$, and the last term tends to $0$ as $s\to t$ by claim 1 of Superposition Decomposition of the Controlled State and Observations; hence the limit superior of $\lVert c^{\alpha,i}_t-c^{\alpha,i}_s\rVert_{2}$ as $s\to t$ is at most $2\varepsilon$ for every $\varepsilon$, which is mean-square continuity. At $t=0$: $\lVert c^{\alpha,i}_0\rVert_{2}=\lim_n\lVert c^{(n),i}_0\rVert_{2}=0$ since $c^{(n),i}_0=0$ almost surely, so $c^{\alpha,i}_0=0$ almost surely by fact (ii). Adaptedness was built in via Mean-Square Completeness of Square-Integrable Random Variables (Riesz-Fischer). (b) For each $n$, both families in the difference are componentwise mean-square continuous, so $t\mapsto\sum_i\lVert c^{(n),i}_t-c^{\alpha,i}_t\rVert_{2}$ is continuous by fact (i); its maximum exists by Extreme Value Theorem on a Closed Real Interval and is at most $l\varepsilon$ for $n\ge N$ by $(*)$, so the maxima tend to $0$.

Cross-distance. Before (c), we record: if $\bigl((\beta^{(m)}),D'\bigr)$ is any approximating sequence for $\alpha$ (possibly the same one), then $d(\alpha^{(n)},\beta^{(m)})\to0$ as $n,m\to\infty$. Indeed $g_{\alpha^{(n)},\beta^{(m)}}$ is continuous, and on $D\cap D'$ pointwise $g_{\alpha^{(n)},\beta^{(m)}}\le2\,\mathbf{1}_D\sum_{\kappa}\lVert\alpha^{(n),\kappa}-\alpha^{\kappa}\rVert_{2}^{2}+2\,\mathbf{1}_{D'}\sum_{\kappa}\lVert\alpha^{\kappa}-\beta^{(m),\kappa}\rVert_{2}^{2}$ by the triangle inequality and $(x+y)^{2}\le2x^{2}+2y^{2}$; both right-hand functions are measurable with integrals tending to $0$ by claim 2 of Almost-Everywhere Mean-Square Limits of Cauchy Sequences of Admissible Controls. Using claim 6 of Restricted Lebesgue Measure and Integral Toolkit on a Compact Interval with the co-null set $D\cap D'$ (its complement is a union of two null sets) to evaluate $\int g_{\alpha^{(n)},\beta^{(m)}}\,d\lambda$ over $D\cap D'$, and monotonicity of the integral (Linearity and Monotonicity of the Lebesgue Integral), $d(\alpha^{(n)},\beta^{(m)})^{2}\to0$.

(c) Let $\tilde c$ satisfy (a) and (b) for $\bigl((\beta^{(m)}),D'\bigr)$ with corrections $b^{(m)}$. Fix $t,i$. Then

The first and third terms tend to $0$ by (b) for the respective sequences; the middle term is at most $C_0\,d(\alpha^{(n)},\beta^{(m)})\to0$ by Step 1 and the cross-distance paragraph. Hence $\lVert c^{\alpha,i}_t-\tilde c^{\,i}_t\rVert_{2}=0$, so $\tilde c^{\,i}_t=c^{\alpha,i}_t$ almost surely by fact (ii). If $\alpha$ is admissible, then by claim 1 the constant sequence is an approximating sequence, and the correction process $c$ of Superposition Decomposition of the Controlled State and Observations for $\alpha$ itself satisfies (a) by claims 1 and 3 there and (b) trivially (the difference vanishes); by the uniqueness just proven, $c^{\alpha,i}_t=c^{i}_t$ almost surely for every $t,i$.

Claim 3. (General cost estimate.) Let $\beta,\gamma$ be admissible controls. We claim

where $K_X:=\max_t\sum_i\lVert X^{i}_t\rVert_{2}$ (finite by mean-square continuity of the state of the model, continuity of the norm by fact (i), and Extreme Value Theorem on a Closed Real Interval) and $C_1$ depends only on $l,k,T,C_0$ and entry bounds for $Q,V,R,F$ (finite by Extreme Value Theorem on a Closed Real Interval). By claim 2 of Superposition Decomposition of the Controlled State and Observations, $X^{\beta}_t=X_t+c^{\beta}_t$ almost surely componentwise, and almost sure equality does not change any expectation below, by the uniqueness convention of Expectation, Variance, and Moments. Write the cost as in its definition. For tuples of square-integrable random variables and a matrix assignment $M$ with entries bounded by $\bar M$, claim 1 of Expected Bilinear Forms: Trace Formula and Mean-Square Continuity and the Cauchy-Schwarz inequality (claim 1 of Cauchy-Schwarz and Triangle Inequalities for the Mean-Square Norm) give, for each $t$,

by bilinearity of the componentwise sums. Apply this with: $(Y,Z)=(X^{\beta},X^{\beta})$, $(Y',Z')=(X^{\gamma},X^{\gamma})$ and $M=Q(t)$; with $(X^{\beta},\beta)$, $(X^{\gamma},\gamma)$ and $M=2V(t)$; with $(\beta,\beta)$, $(\gamma,\gamma)$ and $M=R(t)$; and at $t=T$ with $F$. All the resulting integrands are continuous, so their Riemann integrals equal Lebesgue integrals by claim 3 of Restricted Lebesgue Measure and Integral Toolkit on a Compact Interval, and we may use monotonicity and linearity from claim 1 of Linearity and Monotonicity of the Lebesgue Integral. Using $X^{\beta}-X^{\gamma}=c^{\beta}-c^{\gamma}$, Step 1, and the a priori bounds $\sum_i\lVert X^{\beta,i}_t\rVert_{2}\le K_X+C_0N_{\beta}$, the $Q$-, $F$- and the first half of the $V$-contributions integrate to at most a constant times $\bigl(1+K_X+N_{\beta}+N_{\gamma}\bigr)d(\beta,\gamma)$, where for the terms involving $\int\sum_{\kappa}\lVert\beta^{\kappa}_t\rVert_{2}\,dt\le(Tk)^{1/2}N_{\beta}$ and $\int\sum_{\kappa}\lVert\beta^{\kappa}_t-\gamma^{\kappa}_t\rVert_{2}\,dt\le(Tk)^{1/2}d(\beta,\gamma)$ we used claims 3 and 4 of Restricted Lebesgue Measure and Integral Toolkit on a Compact Interval as in Step 1; for the $R$-term, the integral Cauchy-Schwarz inequality (claim 4 of Restricted Lebesgue Measure and Integral Toolkit on a Compact Interval) bounds $\int\bigl(\sum_{\kappa}\lVert\beta^{\kappa}-\gamma^{\kappa}\rVert_{2}\bigr)\bigl(\sum_{\kappa'}\lVert\beta^{\kappa'}\rVert_{2}\bigr)dt\le k\,d(\beta,\gamma)\,N_{\beta}$, and similarly with $\gamma$. Collecting constants proves $(**)$.

(Conclusion.) Let $\bigl((\alpha^{(n)}),D\bigr)$ approximate $\alpha$. The numbers $N_{\alpha^{(n)}}^{2}=\int g_{\alpha^{(n)},0}\,dt$ are uniformly bounded: fixing $N$ with $d(\alpha^{(n)},\alpha^{(N)})\le1$ for $n\ge N$, pointwise $g_{\alpha^{(n)},0}\le2g_{\alpha^{(n)},\alpha^{(N)}}+2g_{\alpha^{(N)},0}$, so $N_{\alpha^{(n)}}^{2}\le2+2\max_{m\le N}N_{\alpha^{(m)}}^{2}$ for all $n$. By $(**)$, $\bigl|J[\alpha^{(n)}]-J[\alpha^{(m)}]\bigr|\le C_2\,d(\alpha^{(n)},\alpha^{(m)})\to0$, so $\bigl(J[\alpha^{(n)}]\bigr)_n$ is a Cauchy sequence and converges by Every Cauchy Sequence of Real Numbers Converges. If $\bigl((\beta^{(m)}),D'\bigr)$ is another approximating sequence, then the numbers $N_{\beta^{(m)}}^{2}$ are uniformly bounded by rerunning the same two-line argument for that sequence, and the cross-distance paragraph together with $(**)$ gives $\bigl|J[\alpha^{(n)}]-J[\beta^{(m)}]\bigr|\to0$ as $n,m\to\infty$, so the two limits coincide. If $\alpha$ is admissible, the constant approximating sequence of claim 1 has constant costs $J[\alpha]$, so the common limit is $J[\alpha]$. $\square$