Proof of The Separation Theorem over Extended Admissible Controls

Throughout, a real-valued function on a subinterval $I$ of the real numbers $\mathbb{R}$ is called continuous on $I$ when it is continuous relative to $I$, both $I$ and the codomain $\mathbb{R}$ carrying the metric of the real line.

Also throughout, $\lVert\cdot\rVert_{2}$ is the mean-square norm of Square-Integrable Random Variables and the Mean-Square Inner Product, $\lambda:=\lambda_{[0,T]}$, and $d$ and $g_{\beta,\gamma}$ are as in Almost-Everywhere Mean-Square Limits of Cauchy Sequences of Admissible Controls. Claims 3, 4, 5, 6 of the interval toolkit are used to pass between Riemann and Lebesgue integrals, for the integral Cauchy-Schwarz inequality, for null integrands, and for measurability of limits on co-null sets. Entry bounds $M_R$ for $R$ and $M_{\Gamma}$ for $\Gamma$ on $[0,T]$ exist by Extreme Value Theorem on a Closed Real Interval, the entries being continuous.

Fix an extended admissible $\alpha$ and an approximating sequence $\bigl((\alpha^{(n)}),D\bigr)$. Write $\widehat X^{(n)}:=\widehat X(\alpha^{(n)})$ for the controlled estimator of $\alpha^{(n)}$ and $\widehat X:=\widehat X(\alpha)$ for the extended controlled estimator, and set, componentwise,

so that $\varphi_{\alpha}(t)=\mathbb{E}[v_t\cdot(R(t)v_t)]$; all tuples here are tuples of square-integrable random variables, so these expectations are defined and finite by claim 1 of Expected Bilinear Forms: Trace Formula and Mean-Square Continuity. Since every $R(t)$ is positive definite, $x\cdot(R(t)x)\ge0$ for every $x\in\mathbb{R}^{k}$ with equality only at $x=0$; hence the random variables $v_t\cdot(R(t)v_t)$ and $v^{(n)}_t\cdot(R(t)v^{(n)}_t)$ are nonnegative pointwise, and $\varphi_{\alpha},\varphi_n\ge0$ by monotonicity of the expectation (Linearity and Monotonicity of the Lebesgue Integral). By claim 1 of The Separation Theorem for Partial-Information Linear-Quadratic-Gaussian Control,

each $\varphi_n$ being continuous, and $p_n=\int\varphi_n\,d\lambda=\int\mathbf{1}_D\varphi_n\,d\lambda$ by claims 3 and 6 of Restricted Lebesgue Measure and Integral Toolkit on a Compact Interval. By The Linear-Quadratic-Gaussian Cost of an Extended Admissible Control, $J[\alpha^{(n)}]\to J[\alpha]$, so $p_n\to J[\alpha]-V^{*}$.

Step 1 (pointwise convergence on $D$ and measurability). By claim 4 of Conditional Expectation and Estimation Error of the Extended Controlled State, $\max_t\sum_i\lVert\widehat X^{(n),i}_t-\widehat X^{i}_t\rVert_{2}\to0$. For $t\in D$ and each $\kappa$, using the triangle inequality (claim 2 of Cauchy-Schwarz and Triangle Inequalities for the Mean-Square Norm) and the entry bound $M_{\Gamma}$,

the first summand tending to $0$ along the full sequence by condition (ii) of Extended Admissible Control for the Linear-Gaussian State-Observation Model. As in the proof of Convergence of Corrections and Costs along Approximating Sequences of an Extended Admissible Control, bilinearity, claim 1 of Cauchy-Schwarz and Triangle Inequalities for the Mean-Square Norm, and claim 1 of Expected Bilinear Forms: Trace Formula and Mean-Square Continuity give, for every $t$ and $n$,

where $A_n(t):=\sum_{\kappa}\lVert v^{(n),\kappa}_t-v^{\kappa}_t\rVert_{2}$ and $B_n(t):=\sum_{\kappa}\bigl(\lVert v^{(n),\kappa}_t\rVert_{2}+\lVert v^{\kappa}_t\rVert_{2}\bigr)$. By (1), $A_n(t)\to0$ for $t\in D$ while $B_n(t)$ converges, so $\varphi_n(t)\to\varphi_{\alpha}(t)$ for every $t\in D$. Since each $\varphi_n$ is continuous, hence measurable, claim 6 of Restricted Lebesgue Measure and Integral Toolkit on a Compact Interval shows $\mathbf{1}_D\varphi_{\alpha}$ is $\mathcal{B}_{[0,T]}$-measurable.

The functions $\mathbf{1}_DA_n$ and $\mathbf{1}_DB_n$ are also measurable, by the same claim 6: for fixed $n$ and each $j\in\mathbb{N}$, the function $t\mapsto\sum_{\kappa}\lVert v^{(n),\kappa}_t-w^{(j),\kappa}_t\rVert_{2}$ with $w^{(j)}_t:=\alpha^{(j)}_t-\Gamma(t)\widehat X_t$ is continuous (every ingredient is componentwise mean-square continuous), and for $t\in D$ it converges to $A_n(t)$ as $j\to\infty$ because $\lVert\alpha^{(j),\kappa}_t-\alpha^{\kappa}_t\rVert_{2}\to0$ along the full sequence; similarly for $B_n$ using $\lVert v^{\kappa}_t\rVert_{2}=\lim_j\lVert w^{(j),\kappa}_t\rVert_{2}$ on $D$.

Step 2 (uniform bounds). Let $K_1$ bound $\max_t\sum_i\lVert\widehat X^{(n),i}_t\rVert_{2}$ uniformly in $n$: indeed $\sum_i\lVert\widehat X^{(n),i}_t\rVert_{2}\le\sum_i\lVert\widehat X^{(n),i}_t-\widehat X^{i}_t\rVert_{2}+\sum_i\lVert\widehat X^{i}_t\rVert_{2}$, where the first summand is bounded uniformly in $n$ and $t$ by claim 4 of Conditional Expectation and Estimation Error of the Extended Controlled State (a convergent sequence of maxima is bounded) and the second is a continuous function of $t$ (claim 1 there and the triangle inequality), bounded by Extreme Value Theorem on a Closed Real Interval; write $\widehat K:=\max_t\sum_i\lVert\widehat X^{i}_t\rVert_{2}$ for the latter bound. Next, $N_n^{2}:=\int_0^Tg_{\alpha^{(n)},0}\,dt$, with $0$ the zero control, is bounded uniformly in $n$: choosing $N_0$ with $d(\alpha^{(n)},\alpha^{(N_0)})\le1$ for $n\ge N_0$, the pointwise bound $g_{\alpha^{(n)},0}\le2g_{\alpha^{(n)},\alpha^{(N_0)}}+2g_{\alpha^{(N_0)},0}$ (triangle inequality and $(x+y)^{2}\le2x^{2}+2y^{2}$) gives $N_n^{2}\le2+2\max_{m\le N_0}N_m^{2}$.

Since $\Gamma(t)$ is a $k\times l$ matrix with entries bounded by $M_{\Gamma}$, for any tuple $Y=(Y^{1},\dots,Y^{l})$ of square-integrable random variables one has $\sum_{\kappa}\lVert(\Gamma(t)Y)^{\kappa}\rVert_{2}\le k\,M_{\Gamma}\sum_{i}\lVert Y^{i}\rVert_{2}$; combined with $(x+y)^{2}\le2x^{2}+2y^{2}$ and $\bigl(\sum_{\kappa}a_{\kappa}\bigr)^{2}\le k\sum_{\kappa}a_{\kappa}^{2}$ this yields the pointwise bounds

Integrating with claims 3 and 6 of Restricted Lebesgue Measure and Integral Toolkit on a Compact Interval gives $\int\mathbf{1}_D\bigl(\sum_{\kappa}\lVert v^{(n),\kappa}\rVert_{2}\bigr)^{2}d\lambda\le2kN_n^{2}+2k^{2}M_{\Gamma}^{2}TK_1^{2}$ and $\int\mathbf{1}_D\bigl(\sum_{\kappa}\lVert w^{(j),\kappa}\rVert_{2}\bigr)^{2}d\lambda\le2k\sup_mN_m^{2}+2k^{2}M_{\Gamma}^{2}T\widehat K^{2}$ for every $j$. On $D$, $\bigl(\sum_{\kappa}\lVert v^{\kappa}_t\rVert_{2}\bigr)^{2}=\lim_j\bigl(\sum_{\kappa}\lVert w^{(j),\kappa}_t\rVert_{2}\bigr)^{2}$, so by Fatou's Lemma,

Consequently, using $B_n^{2}\le2\bigl(\sum_{\kappa}\lVert v^{(n),\kappa}\rVert_{2}\bigr)^{2}+2\bigl(\sum_{\kappa}\lVert v^{\kappa}\rVert_{2}\bigr)^{2}$,

Finally, by the same $\Gamma$-bound applied to $A_n$,

by claim 2 of Almost-Everywhere Mean-Square Limits of Cauchy Sequences of Admissible Controls (the definitional approximating data satisfies its hypothesis) and claim 4 of Conditional Expectation and Estimation Error of the Extended Controlled State.

Step 3 (claim 1). First, finiteness: from (2) with $n=1$, pointwise $\mathbf{1}_D\varphi_{\alpha}\le\mathbf{1}_D\varphi_1+M_R\mathbf{1}_DA_1B_1$, so by monotonicity and linearity of the Lebesgue integral (Linearity and Monotonicity of the Lebesgue Integral) and the integral Cauchy-Schwarz inequality (claim 4 of Restricted Lebesgue Measure and Integral Toolkit on a Compact Interval),

Now, for every $n$, the two-sided pointwise bounds $\mathbf{1}_D\varphi_n\le\mathbf{1}_D\varphi_{\alpha}+M_R\mathbf{1}_DA_nB_n$ and $\mathbf{1}_D\varphi_{\alpha}\le\mathbf{1}_D\varphi_n+M_R\mathbf{1}_DA_nB_n$, integrated with the same tools, give (all integrals now being finite real numbers)

by (3) and (4). Hence $\int\mathbf{1}_D\varphi_{\alpha}\,d\lambda=\lim_np_n=J[\alpha]-V^{*}$, which is the displayed representation. If $\bigl((\beta^{(m)}),D'\bigr)$ is any other approximating sequence for $\alpha$, the entire argument applies verbatim to that sequence and yields $\int\mathbf{1}_{D'}\varphi_{\alpha}\,d\lambda=J[\alpha]-V^{*}$ as well, since $J[\alpha]$ does not depend on the approximating sequence by The Linear-Quadratic-Gaussian Cost of an Extended Admissible Control; so the integral is independent of the choice of approximating sequence.

Step 4 (claim 2). Since $\mathbf{1}_D\varphi_{\alpha}\ge0$, claim 1 gives $J[\alpha]\ge V^{*}$. Suppose $J[\alpha]=V^{*}$, so $\int\mathbf{1}_D\varphi_{\alpha}\,d\lambda=0$. By claim 5 of Restricted Lebesgue Measure and Integral Toolkit on a Compact Interval, $\lambda(\{\mathbf{1}_D\varphi_{\alpha}>0\})=0$. Set $D_0:=D\setminus\{\mathbf{1}_D\varphi_{\alpha}>0\}$; its complement is the union of two null sets, hence null by the subadditivity argument of claim 5 of Restricted Lebesgue Measure and Integral Toolkit on a Compact Interval. For $t\in D_0$: $\mathbb{E}[v_t\cdot(R(t)v_t)]=0$ with the integrand nonnegative pointwise, so by Markov's inequality (Markov's and Chebyshev's Inequalities) $v_t\cdot(R(t)v_t)=0$ almost surely; on that event, positive definiteness of $R(t)$ forces $v_t=0$, that is, $\alpha_t=\Gamma(t)\widehat X_t$ componentwise almost surely. Conversely, suppose such a co-null $D_0$ exists. For $t\in D_0$, $v_t=0$ almost surely componentwise, so $\varphi_{\alpha}(t)=0$ (the expectation of a random variable almost surely equal to $0$). Applying claim 6 of Restricted Lebesgue Measure and Integral Toolkit on a Compact Interval with the co-null set $D_0$ and $g:=\mathbf{1}_D\varphi_{\alpha}$ gives $\int\mathbf{1}_D\varphi_{\alpha}\,d\lambda=\int\mathbf{1}_D\varphi_{\alpha}\mathbf{1}_{D_0}\,d\lambda=0$, since the integrand vanishes identically. By claim 1, $J[\alpha]=V^{*}$.

Step 5 (claim 3). By claim 3 of The Separation Theorem for Partial-Information Linear-Quadratic-Gaussian Control, the closed-loop control $\alpha^{*}$ of Existence and Self-Consistency of the Closed-Loop Feedback Control is admissible with $J[\alpha^{*}]=V^{*}$; by claim 1 of Convergence of Corrections and Costs along Approximating Sequences of an Extended Admissible Control it is extended admissible, and by the consistency recorded in The Linear-Quadratic-Gaussian Cost of an Extended Admissible Control its extended cost is the same number $V^{*}$. Combined with claim 2, $J[\alpha]\ge V^{*}=J[\alpha^{*}]$ for every extended admissible $\alpha$, so $V^{*}$ is the minimum of the extended problem, and the optimal value over the extended class coincides with the optimal value over the admissible class. $\square$