Proof of Cost Limit Along the Approximate Kalman Policy

Throughout, fix the fluctuation LQG data and, for each $N$, the driving system and projected solution of the statement; superscripts $N$ are suppressed when no confusion can arise. Write $\mathbb{E}$ for the expectation of the $N$-th probability space, $z_t=(\mathfrak{s}_t,\mathfrak{a}_t)\in\mathbb{R}^{l+m}$, $\rho_t=N^{-1/2}|z_t|$, and $\kappa=1+\sup_N\mathbb{E}[|\mathfrak{s}^N_0|^4]$, finite by (I2). Fix, by conclusion 1 of the policy lemma and boundedness of continuous functions, entry bounds $C_Z$ for $Z_t$ and $c_W$ for $W_tR_t^{-1}W_t^{\top}$ on $[0,T]$; the entries of $W_tR_t^{-1}W_t^{\top}$ are continuous, as finite sums of products of the continuous entries of $Z$, $\mathcal{B}$, $V$, $R^{-1}$ (conclusion 1 of the policy lemma and (H2)). By conclusion 1 of the policy lemma, its matrices $Q_t$, $V_t$, $R_t$, $W_t$, and $\hat{F}$ are given by the same defining formulas as those of the completion-of-squares theorem, and the family $Z$ of (H2) satisfies the Riccati equation of hypothesis (H2) of that theorem; so both that theorem and the cost expansion theorem are available for each fixed $N$, their common hypothesis $\mathcal{A}=\int_{[0,T]}\mathbb{E}[|\mathfrak{a}_t|^2]dt<\infty$ holding by part (a) of the filter error covariance lemma. By part (d) of that lemma there is a real $C_1$, independent of $N$, with

for every $N$ (the second bound obtained from the first, together with $\mathbb{E}[|\cdot|^2]\le1+\mathbb{E}[|\cdot|^4]$ applied to the four summands of the first display and $\kappa\ge1$, as in part (d) of that lemma); in particular $\mathbb{E}[|z_t|^2]\le2(4+C_1)\kappa$ and $\mathbb{E}[|z_t|^4]\le8(\mathbb{E}[|\mathfrak{s}_t|^4]+\mathbb{E}[|\mathfrak{a}_t|^4])\le16C_1\kappa$ for all $t$ and $N$. Finally, the limit integrand $t\mapsto Z_t\cdot\Theta^\star_t+(W_tR_t^{-1}W_t^{\top})\cdot\Pi_t$ is continuous, being a finite sum of products of continuous functions (conclusions 1 and 2 of the policy lemma and (H2)), which proves the first assertion.

Step 1: expansion and completion of squares. Fix $N$. By parts (b) and (c) of the cost expansion theorem, $J^N[h^N]$ is finite, $LQG[(\mathfrak{s}),(\mathfrak{a})]$ is a well-defined real number, and

with $R_N$ obeying the modulus bound of its part (c). By part (c) of the completion-of-squares theorem, with $u_t=\mathfrak{a}_t+R_t^{-1}W_t^{\top}\mathfrak{s}_t$ and $e_s$ its linearization residual,

all terms finite. In particular each $N(J^N[h^N]-J^{MF})+\sum_\gamma P^\gamma_0\zeta^\gamma_N$ is a well-defined real number. We treat the four pieces in Steps 2--5.

Step 2: initial term. $\mathbb{E}[\mathfrak{s}_0\cdot Z_0\mathfrak{s}_0]=\sum_{\gamma,\delta}Z^{\gamma\delta}_0\,\mathbb{E}[\mathfrak{s}^{N,\gamma}_0\mathfrak{s}^{N,\delta}_0]\to\sum_{\gamma,\delta}Z^{\gamma\delta}_0\Pi^{\gamma\delta}_0=Z_0\cdot\Pi_0$ as $N\to\infty$, by (I1) and linearity (a finite sum of convergent sequences).

Step 3: control term. By conclusion 4(a) of the policy lemma, $\mathfrak{a}_t=N^{1/2}(\alpha_t-A_t)=-R_t^{-1}W_t^{\top}\hat{\mathfrak{s}}^N_t$ on the regular event, hence almost surely $u_t=R_t^{-1}W_t^{\top}(\mathfrak{s}_t-\hat{\mathfrak{s}}^N_t)=\mathcal{G}_t\,\varepsilon^N_t$. Therefore, for every $s$,

where the last equality uses $\mathcal{G}_s=R_s^{-1}W_s^{\top}$, the symmetry of $R_s$ and of $R_s^{-1}$ (invertibility of symmetric positive definite matrices), and $R_sR_s^{-1}=I$: $\mathcal{G}^{\top}R\mathcal{G}=WR^{-1}RR^{-1}W^{\top}=WR^{-1}W^{\top}$, with the reversal rule for the transpose. Consequently, by linearity and monotonicity of the integral (both integrands bounded and measurable in $s$: part (a) of the filter error covariance lemma for $\Pi^N$, continuity for $\Pi$),

By part (e) of the filter error covariance lemma, the supremum is at most $C(\max_{\gamma,\delta}|\mathbb{E}[\mathfrak{s}^{N,\gamma}_0\mathfrak{s}^{N,\delta}_0]-\Pi^{\gamma\delta}_0|+N^{-1/2}(1+\mathbb{E}[|\mathfrak{s}^N_0|^4]))\le C(\max_{\gamma,\delta}|\mathbb{E}[\mathfrak{s}^{N,\gamma}_0\mathfrak{s}^{N,\delta}_0]-\Pi^{\gamma\delta}_0|+N^{-1/2}\kappa)$, which tends to $0$ by (I1) and (I2). Hence $\int\mathbb{E}[u_s\cdot R_su_s]ds\to\int_{[0,T]}(W_sR_s^{-1}W_s^{\top})\cdot\Pi_s\,ds$.

Step 4: covariance term. By clause 6 of the fluctuation LQG data, $(\Theta^\star_s)^{\gamma\delta}=\Theta^{\gamma\delta}(S_s,A_s)$. By part (c) of the covariance deviation lemma together with (0.1) and monotonicity of the integral,

with the constant $c_\Theta$ of that lemma and $\mathcal{S}+\mathcal{A}\le2T(4+C_1)\kappa$ by (0.1). This tends to $0$, and $\int_{[0,T]}\sum_{\gamma,\delta}Z^{\gamma\delta}_s\Theta^{\gamma\delta}(S_s,A_s)ds=\int_{[0,T]}Z_s\cdot\Theta^\star_s\,ds$ by the definition of the pairing.

Step 5: residual term. By part (b) of the completion-of-squares theorem, $|e_s|\le c_eN^{-1/2}(|\mathfrak{s}_s|^2+|\mathfrak{a}_s|^2)=c_eN^{-1/2}|z_s|^2$ with $c_e=\tfrac32l^{3/2}(l+m)K$. Since $|x\cdot Z_sy|\le C_Z(\sum_\gamma|x^\gamma|)(\sum_\delta|y^\delta|)\le lC_Z|x||y|$, and pointwise $|\mathfrak{s}_s|\,|z_s|^2\le|z_s|^3\le1+|z_s|^4$,

Step 6: the remainder vanishes. Let $\theta>0$. By part (a) of the cost expansion theorem there is $\delta^\circ>0$ with $\omega_L(u)\le\theta$, $\omega_b(u)\le\theta$, and $\omega_G(u)\le\theta$ for all $u\in[0,\delta^\circ]$, and globally $\omega_L\le2K_c$, $\omega_b\le6lK$, $\omega_G\le2K_c$. Let $C_P$ be the co-state bound of that theorem and write $\bar{\omega}=2K_c+C_P\,6lK$. Splitting the time integral of its part (c) on the events $\{\rho_t\le\delta^\circ\}$ and $\{\rho_t>\delta^\circ\}$ (the moduli being nondecreasing) and using monotonicity,

where $\mathbf{1}_E$ denotes the indicator of the event $E$ and we used $d(\Sigma_T,S_T)=N^{-1/2}|\mathfrak{s}_T|$. By the Cauchy-Schwarz inequality and Markov's inequality applied to the nonnegative variable $|z_t|^2$ at level $N(\delta^\circ)^2$,

using $\sqrt{C_1(4+C_1)}\le4+C_1$ (since $C_1\le4+C_1$) and $4\sqrt{2}\le6$; using (0.1), and the same bound holds for the terminal term with $|\mathfrak{s}_T|$ in place of $|z_t|$. Hence, with (0.1) again,

and since $\theta>0$ was arbitrary, $R_N\to0$.

Step 7: conclusion. Combining Steps 1--6, the sequence $N(J^N[h^N]-J^{MF})+\sum_\gamma P^\gamma_0\zeta^\gamma_N$ is, for each $N$, the sum of four terms converging respectively to $Z_0\cdot\Pi_0$, $\int_{[0,T]}(W_sR_s^{-1}W_s^{\top})\cdot\Pi_s\,ds$, $\int_{[0,T]}Z_s\cdot\Theta^\star_s\,ds$, and $0$, plus the remainder $R_N\to0$. The limit identity follows, the two integrals combining by linearity of the integral into the integral of the continuous integrand recorded in the statement. $\blacksquare$