Proof of Filtering Lower-Bound Reduction of the Recentred N-Agent Cost

Throughout, fix the common data, $Z$, and $W$ as in the statement, write $M_2=\tfrac12(1+M)$, and abbreviate $\mathfrak{s}_t=\mathfrak{s}^{(N)}_t$, $\mathfrak{a}_t=\mathfrak{a}^{(N)}_t$, $u_t=u^{(N)}_t$, $\Sigma_t=\Sigma^{(N)}_t$, $\alpha_t=\alpha^{(N)}_t$, and $\varepsilon^\gamma_t=\varepsilon^{(N),\gamma}_t$ when $N$ is fixed. All pointwise inequalities between random variables below hold at every point of $\Omega$, and expectations of nonnegative random variables are taken in $[0,\infty]$ with the additivity and monotonicity of the linearity and monotonicity theorem.

Step 1 (second moments and applicability). For a real $a\ge0$, $(1-a)^2\ge0$ gives $a\le\tfrac12(1+a^2)$. Applying this at each $\omega$ with $a=|\mathfrak{a}_t|^2$ and $a=|\mathfrak{s}_t|^2$ and taking expectations, hypothesis (M) gives, for every $N$ and $t$,

The function $t\mapsto\mathbb{E}[|\mathfrak{a}_t|^2]$ is measurable with well-defined Lebesgue integral over the compact interval $[0,T]$, by part (a) of the a priori second-moment bound, and is bounded by the constant $M_2$, so $\mathcal{A}_N\le TM_2<\infty$ by the monotonicity of the interval integral. Hence the hypothesis of the second-order expansion holds for every $N$, and with it all conclusions of the expansion and of the completion-of-squares theorem.

Step 2 (the identity defining $r_N$). Conclusion (c) of the expansion gives $\mathcal{J}_N=LQG[(\mathfrak{s}),(\mathfrak{a})]+R_N$, and conclusion (c) of the completion-of-squares theorem gives

with all integrals finite. By clause (b) of the covariance deviation lemma, each $s\mapsto\mathbb{E}[\Theta^{\gamma\delta}(\Sigma_s,\alpha_s)]$ is bounded and measurable, and by clause (c) there each $s\mapsto\Theta^{\star\gamma\delta}_s$ is continuous and bounded; the entries of $s\mapsto Z_s$ are continuous and bounded by $C_Z$ (conclusion (a) of the completion-of-squares theorem and hypothesis (H2)). Hence $s\mapsto\sum_{\gamma\delta}Z^{\gamma\delta}_s\mathbb{E}[\Theta^{\gamma\delta}(\Sigma_s,\alpha_s)]$ and $s\mapsto\sum_{\gamma\delta}Z^{\gamma\delta}_s\Theta^{\star\gamma\delta}_s$ are bounded measurable functions with finite interval integrals, so the third integral above splits by the linearity of the interval integral, with $s\mapsto2\mathbb{E}[\mathfrak{s}_s\cdot Z_se_s]$, the difference of integrable functions, integrable. Subtracting $\int_{[0,T]}\sum_{\gamma\delta}Z^{\gamma\delta}_s\Theta^{\star\gamma\delta}_s\,ds$ from both sides of the identity of the statement, we conclude that $r_N$ is well defined and

Step 3 ($R_N$ vanishes). Let $\omega_L,\omega_b,\omega_G$ and $\rho_t$ be as in the expansion theorem, and set $\omega_\infty=2K_c+6\,l\,K\,C_P$, so that $\omega_L(u)+C_P\,\omega_b(u)\le\omega_\infty$ and $\omega_G(u)\le2K_c$ for every $u\ge0$ by conclusion (a) there. Fix $\delta>0$. At every point of $[0,T]\times\Omega$, either $\rho_t\le\delta$, and then $\big(\omega_L(\rho_t)+C_P\omega_b(\rho_t)\big)\big(|\mathfrak{s}_t|^2+|\mathfrak{a}_t|^2\big)\le\big(\omega_L(\delta)+C_P\omega_b(\delta)\big)\big(|\mathfrak{s}_t|^2+|\mathfrak{a}_t|^2\big)$ since the moduli are nondecreasing, or $\rho_t>\delta$, and then, because $|\mathfrak{s}_t|^2+|\mathfrak{a}_t|^2=N\rho_t^2>N\delta^2$ and $(x+y)^2\le2x^2+2y^2$,

Adding the two bounds and taking expectations, using Step 1 and (M),

the integral over $[0,T]$ of the left-hand side is well defined and finite by conclusion (b) of the expansion, and is at most $T$ times the constant right-hand side by monotonicity. Similarly, since $d(\Sigma_T,S_T)=N^{-1/2}|\mathfrak{s}_T|$,

By conclusion (c) of the expansion,

Given $\varepsilon''>0$, conclusion (a) of the expansion provides $\delta>0$ with $\omega_L(\delta)+C_P\omega_b(\delta)$ and $\omega_G(\delta)$ so small that the $\delta$-terms sum to at most $\varepsilon''/2$, and then $N_1$ with the $1/N$-terms at most $\varepsilon''/2$ for $N\ge N_1$; hence $|R_N|\le\varepsilon''$ for $N\ge N_1$, and $(R_N)$ has limit $0$.

Step 4 ($\mathrm{II}_N$ and $\mathrm{III}_N$ vanish). By the componentwise estimate $|x^\gamma|\le|x|$ of the componentwise calculus toolkit and $|Z^{\gamma\delta}_s|\le C_Z$, at every point $|\mathfrak{s}_s\cdot Z_se_s|\le\sum_{\gamma,\delta}C_Z|\mathfrak{s}^\gamma_s||e^\delta_s|\le C_Z\,l^2\,|\mathfrak{s}_s||e_s|$, so by conclusion (b) of the completion-of-squares theorem, $|\mathfrak{s}_s\cdot Z_se_s|\le C_Zl^2c_e\,N^{-1/2}|\mathfrak{s}_s|\big(|\mathfrak{s}_s|^2+|\mathfrak{a}_s|^2\big)$. Pointwise, $(|\mathfrak{s}|-|\mathfrak{s}|^2)^2\ge0$ gives $|\mathfrak{s}|^3\le\tfrac12(|\mathfrak{s}|^2+|\mathfrak{s}|^4)$, and $(|\mathfrak{s}|-|\mathfrak{a}|^2)^2\ge0$ gives $|\mathfrak{s}||\mathfrak{a}|^2\le\tfrac12(|\mathfrak{s}|^2+|\mathfrak{a}|^4)$; hence with Step 1 and (M), $\mathbb{E}\big[|\mathfrak{s}_s|(|\mathfrak{s}_s|^2+|\mathfrak{a}_s|^2)\big]\le M_2+M=:M_3$ for every $s$. Therefore $|2\,\mathbb{E}[\mathfrak{s}_s\cdot Z_se_s]|\le2\,\mathbb{E}\big[|\mathfrak{s}_s\cdot Z_se_s|\big]\le2C_Zl^2c_eM_3N^{-1/2}$ for every $s$ (monotonicity, and $\pm X\le|X|$), so by monotonicity of the interval integral $|\mathrm{II}_N|\le2TC_Zl^2c_eM_3\,N^{-1/2}$, and $(\mathrm{II}_N)$ has limit $0$. For $\mathrm{III}_N$: with $c_\Theta$ as in the covariance deviation lemma, its clause (b) and Step 1 give, for every $s$,

so $|\mathrm{III}_N|\le TC_Zl^2c_\Theta(2M_2)^{1/2}N^{-1/2}$, and $(\mathrm{III}_N)$ has limit $0$. Combining Steps 3--4 with the triangle inequality, $(r_N)$ has limit $0$. Together with the finiteness assertions of Step 2 this proves conclusion (a).

Step 5 (filtering bound). Fix $N$ and $t$, and write $\mathcal{K}=R_t^{-1}W_t^T$, a real matrix with $m$ rows and $l$ columns with entries bounded by $C_K$ (conclusion (a) of the completion-of-squares theorem). Each component of $\mathfrak{s}_t$ is bounded: every coordinate of the empirical state measure and of $S_t$ lies in $[0,1]$, both lying in the probability simplex at every $\omega$, so $|\mathfrak{s}^\gamma_t|\le\sqrt{N}$ at every $\omega$, and bounded random variables are square-integrable by monotonicity. Each component of $\mathfrak{a}_t$ satisfies $\mathbb{E}[(\mathfrak{a}^j_t)^2]\le\mathbb{E}[|\mathfrak{a}_t|^2]\le M_2$ (componentwise estimate and Step 1), hence is square-integrable, and is almost surely equal to a $\mathcal{G}^{(N)}_t$-measurable square-integrable random variable by conclusions 2--3 of the observation-adaptedness lemma. Set $X=-\mathcal{K}\mathfrak{s}_t$ componentwise, so $X^i=-\sum_{\gamma=1}^{l}\mathcal{K}^{i\gamma}\mathfrak{s}^\gamma_t$ is square-integrable by the closure properties of the square-integrability definition, and $Y=\mathfrak{a}_t$; then $Y-X=u_t$ componentwise, and $\mathbb{E}[u_t\cdot R_tu_t]=\mathbb{E}[(Y-X)\cdot(R_t(Y-X))]$, the entry pairing of the completion-of-squares theorem and the index formula of the conditional mean-square optimality lemma being the identical double sum. Each $R_t$ is symmetric positive definite (conclusion (a) of the completion-of-squares theorem), hence positive semidefinite, so that lemma applies with $k=m$, $\mathcal{G}=\mathcal{G}^{(N)}_t$, and weight $R_t$: fixing conditional expectations $\mu^\gamma$ of $\mathfrak{s}^\gamma_t$ given $\mathcal{G}^{(N)}_t$, so that $\varepsilon^\gamma_t=\mathfrak{s}^\gamma_t-\mu^\gamma$, and conditional expectations $\mu_X^i$ of $X^i$,

By the linearity of conditional expectation (part 1 of the basic properties lemma, applied finitely many times), $-\sum_\gamma\mathcal{K}^{i\gamma}\mu^\gamma$ is a conditional expectation of $X^i$, so $\mu_X^i=-\sum_\gamma\mathcal{K}^{i\gamma}\mu^\gamma$ almost surely by the uniqueness assertion of the existence and uniqueness theorem; hence $\varepsilon_X^i=-\sum_\gamma\mathcal{K}^{i\gamma}\varepsilon^\gamma_t$ almost surely. For square-integrable $U,\tilde{U},V'$ with $U$ almost surely equal to $\tilde{U}$ one has $\mathbb{E}[UV']=\mathbb{E}[\tilde{U}V']$, since $\lVert U-\tilde{U}\rVert_2=0$ by the null-equivalence statement of the square-integrability definition and $|\mathbb{E}[(U-\tilde{U})V']|\le\lVert U-\tilde{U}\rVert_2\lVert V'\rVert_2=0$ by the Cauchy--Schwarz inequality; a product both of whose factors are replaced by almost-sure equals requires two applications, one factor at a time. Applying this and the linearity of the integral entrywise,

by the entry formulas for matrix products and the transpose. Since $R_t$ is symmetric positive definite, $R_t^{-1}$ is symmetric positive definite by the invertibility lemma, and

using the identity $(UV)^T=V^TU^T$ of claim 3 of the componentwise calculus toolkit, the involutivity $(M^T)^T=M$, immediate from the definition of the transpose, and the symmetry of $R_t^{-1}$. The matrix $\Xi_t$ is symmetric, $\Xi_t^T=W_t(R_t^{-1})^TW_t^T=\Xi_t$, and positive semidefinite: for $x\in\mathbb{R}^l$, $x\cdot(\Xi_tx)=(W_t^Tx)\cdot\big(R_t^{-1}(W_t^Tx)\big)\ge0$, using the identity $y\cdot(Mz)=(M^Ty)\cdot z$ of claim 3 of the toolkit and the positive definiteness of $R_t^{-1}$. Finally $\sum_{\gamma\delta}\Xi^{\gamma\delta}_t\mathbb{E}[\varepsilon^\gamma_t\varepsilon^\delta_t]=\mathbb{E}[\varepsilon_t\cdot(\Xi_t\varepsilon_t)]\ge0$ by claim 1 of the expected quadratic form lemma and the pointwise nonnegativity of $\varepsilon_t(\omega)\cdot(\Xi_t\varepsilon_t(\omega))$ with monotonicity of the integral. If $\bar{\mu}^\gamma$ is another choice of conditional expectations, each $\bar{\mu}^\gamma$ is almost surely equal to $\mu^\gamma$ by the uniqueness assertion, so each product expectation $\mathbb{E}[\varepsilon^\gamma_t\varepsilon^\delta_t]$ is unchanged, again by two applications of the almost-sure substitution above; the middle quantity is therefore independent of the choice. This proves conclusion (b).

Step 6 (lower bound). By claim 1 of the expected quadratic form lemma, $\mathbb{E}[\mathfrak{s}_0\cdot Z_0\mathfrak{s}_0]=\sum_{\gamma\delta}Z_0^{\gamma\delta}\mathbb{E}[\mathfrak{s}_0^\gamma\mathfrak{s}_0^\delta]$. Let $\varepsilon'>0$. By hypothesis (I) there is $N_1$ such that $|\mathbb{E}[\mathfrak{s}^{(N),\gamma}_0\mathfrak{s}^{(N),\delta}_0]-\Pi_0^{\gamma\delta}|\le\varepsilon'/(2l^2C_Z+2)$ for all $\gamma,\delta$ and $N\ge N_1$, whence $\big|\mathbb{E}[\mathfrak{s}_0\cdot Z_0\mathfrak{s}_0]-\sum_{\gamma\delta}Z_0^{\gamma\delta}\Pi_0^{\gamma\delta}\big|\le l^2C_Z\cdot\varepsilon'/(2l^2C_Z+2)\le\varepsilon'/2$; and by conclusion (a) there is $N_2$ with $|r_N|\le\varepsilon'/2$ for $N\ge N_2$. For $N\ge N_0=\max(N_1,N_2)$, the identity of conclusion (a) gives

which is conclusion (c). $\square$