Proof of Filtering Lower-Bound Reduction of the Recentred N-Agent Cost

Throughout, fix the common data, $Z$, and $W$ as in the statement, write $M_2=\tfrac12(1+M)$, and abbreviate $\mathfrak{s}_t=\mathfrak{s}^{(N)}_t$, $\mathfrak{a}_t=\mathfrak{a}^{(N)}_t$, $u_t=u^{(N)}_t$, $\Sigma_t=\Sigma^{(N)}_t$, $\alpha_t=\alpha^{(N)}_t$, and $\varepsilon^\gamma_t=\varepsilon^{(N),\gamma}_t$ when $N$ is fixed. All pointwise inequalities between random variables below hold at every point of $\Omega$, and expectations of nonnegative random variables are taken in $[0,\infty]$ with the additivity and monotonicity of the linearity and monotonicity theorem.

Step 1 (second moments and applicability). For a real $a\ge0$, $(1-a)^2\ge0$ gives $a\le\tfrac12(1+a^2)$. Applying this at each $\omega$ with $a=|\mathfrak{a}_t|^2$ and $a=|\mathfrak{s}_t|^2$ and taking expectations, hypothesis (M) gives, for every $N$ and $t$,

$$\mathbb{E}\big[|\mathfrak{a}_t|^2\big]\le\tfrac12\big(1+\mathbb{E}[|\mathfrak{a}_t|^4]\big)\le M_2,\qquad \mathbb{E}\big[|\mathfrak{s}_t|^2\big]\le M_2 .$$

The function $t\mapsto\mathbb{E}[|\mathfrak{a}_t|^2]$ is measurable with well-defined Lebesgue integral over the compact interval $[0,T]$, by part (a) of the a priori second-moment bound, and is bounded by the constant $M_2$, so $\mathcal{A}^{(N)}_2\le TM_2<\infty$ by the monotonicity of the interval integral. Hence the integrability hypothesis $\mathcal{A}_2<\infty$ of the second-order expansion holds for every $N$ — its remaining hypotheses (the common data, the per-$N$ driving system, $\mathcal{A}$-valued policy, and solution, the extensions, and the convexity of the control set $\mathcal{A}$), and hypotheses (H1)--(H2) of the completion-of-squares theorem, being assumed in the statement — and with it all conclusions of the expansion and of the completion-of-squares theorem.

Step 2 (the identity defining $r_N$). Conclusion (c) of the expansion gives $\mathcal{J}_N=LQG[(\mathfrak{s}),(\mathfrak{a})]+R_N$, and conclusion (c) of the completion-of-squares theorem gives

$$LQG[(\mathfrak{s}),(\mathfrak{a})]=\mathbb{E}\big[\mathfrak{s}_0\cdot Z_0\mathfrak{s}_0\big]+\int_{[0,T]}\mathbb{E}\big[u_s\cdot R_su_s\big]ds+\int_{[0,T]}\Big(2\,\mathbb{E}\big[\mathfrak{s}_s\cdot Z_se_s\big]+\sum_{\gamma,\delta=1}^{l}Z^{\gamma\delta}_s\,\mathbb{E}\big[\Theta^{\gamma\delta}(\Sigma_s,\alpha_s)\big]\Big)ds,$$

with all integrals finite. By clause (b) of the covariance deviation lemma, each $s\mapsto\mathbb{E}[\Theta^{\gamma\delta}(\Sigma_s,\alpha_s)]$ is bounded and measurable, and by clause (c) there each $s\mapsto\Theta^{\star\gamma\delta}_s$ is continuous and bounded; the entries of $s\mapsto Z_s$ are continuous and bounded by $C_Z$ (conclusion (a) of the completion-of-squares theorem and hypothesis (H2)). Hence $s\mapsto\sum_{\gamma\delta}Z^{\gamma\delta}_s\mathbb{E}[\Theta^{\gamma\delta}(\Sigma_s,\alpha_s)]$ and $s\mapsto\sum_{\gamma\delta}Z^{\gamma\delta}_s\Theta^{\star\gamma\delta}_s$ are bounded measurable functions with finite interval integrals — each entry $s\mapsto Z^{\gamma\delta}_s$, being continuous, is measurable by clause 3 of the interval toolkit, as is each $s\mapsto\Theta^{\star\gamma\delta}_s$, and finite sums and products of measurable real functions are measurable by the arithmetic of measurable functions —, so the third integral above splits by the linearity of the interval integral, with $s\mapsto2\mathbb{E}[\mathfrak{s}_s\cdot Z_se_s]$, the difference of integrable functions, integrable. Subtracting $\int_{[0,T]}\sum_{\gamma\delta}Z^{\gamma\delta}_s\Theta^{\star\gamma\delta}_s\,ds$ from both sides of the identity of the statement, we conclude that $r_N$ is well defined and

$$r_N=R_N+\int_{[0,T]}2\,\mathbb{E}\big[\mathfrak{s}_s\cdot Z_se_s\big]\,ds+\int_{[0,T]}\sum_{\gamma,\delta=1}^{l}Z^{\gamma\delta}_s\Big(\mathbb{E}\big[\Theta^{\gamma\delta}(\Sigma_s,\alpha_s)\big]-\Theta^{\star\gamma\delta}_s\Big)ds\;=:\;R_N+\mathrm{II}_N+\mathrm{III}_N .$$

Step 3 ($R_N$ vanishes). Let $\omega_L,\omega_b,\omega_G$ and $\rho_t$ be as in the expansion theorem, and set $\omega_\infty=2K_c+6\,l\,K\,C_P$, so that $\omega_L(u)+C_P\,\omega_b(u)\le\omega_\infty$ and $\omega_G(u)\le2K_c$ for every $u\ge0$ by conclusion (a) there. Fix $\delta>0$. At every point of $[0,T]\times\Omega$, either $\rho_t\le\delta$, and then $\big(\omega_L(\rho_t)+C_P\omega_b(\rho_t)\big)\big(|\mathfrak{s}_t|^2+|\mathfrak{a}_t|^2\big)\le\big(\omega_L(\delta)+C_P\omega_b(\delta)\big)\big(|\mathfrak{s}_t|^2+|\mathfrak{a}_t|^2\big)$ since the moduli are nondecreasing, or $\rho_t>\delta$, and then, because $|\mathfrak{s}_t|^2+|\mathfrak{a}_t|^2=N\rho_t^2>N\delta^2$ and $(x+y)^2\le2x^2+2y^2$,

$$\big(\omega_L(\rho_t)+C_P\omega_b(\rho_t)\big)\big(|\mathfrak{s}_t|^2+|\mathfrak{a}_t|^2\big)\le\omega_\infty\,\frac{\big(|\mathfrak{s}_t|^2+|\mathfrak{a}_t|^2\big)^2}{N\delta^2}\le\frac{2\,\omega_\infty}{N\delta^2}\big(|\mathfrak{s}_t|^4+|\mathfrak{a}_t|^4\big).$$

Adding the two bounds and taking expectations, using Step 1 and (M),

$$\mathbb{E}\Big[\big(\omega_L(\rho_t)+C_P\omega_b(\rho_t)\big)\big(|\mathfrak{s}_t|^2+|\mathfrak{a}_t|^2\big)\Big]\le2M_2\big(\omega_L(\delta)+C_P\omega_b(\delta)\big)+\frac{4\,\omega_\infty M}{N\delta^2}\,;$$

the integral over $[0,T]$ of the left-hand side is well defined and finite by conclusion (b) of the expansion, and is at most $T$ times the constant right-hand side by monotonicity. Similarly, since $d(\Sigma_T,S_T)=N^{-1/2}|\mathfrak{s}_T|$,

$$\mathbb{E}\Big[\omega_G\big(d(\Sigma_T,S_T)\big)|\mathfrak{s}_T|^2\Big]\le\omega_G(\delta)\,M_2+\frac{2K_c\,M}{N\delta^2}.$$

By conclusion (c) of the expansion,

$$|R_N|\ \le\ \frac{l+m}{2}\,T\Big(2M_2\big(\omega_L(\delta)+C_P\omega_b(\delta)\big)+\frac{4\omega_\infty M}{N\delta^2}\Big)+\frac{l}{2}\Big(\omega_G(\delta)M_2+\frac{2K_cM}{N\delta^2}\Big).$$

Given $\varepsilon''>0$, conclusion (a) of the expansion provides $\delta>0$ with $\omega_L(\delta)+C_P\omega_b(\delta)$ and $\omega_G(\delta)$ so small that the $\delta$-terms sum to at most $\varepsilon''/2$, and then $N_1$ with the $1/N$-terms at most $\varepsilon''/2$ for $N\ge N_1$; hence $|R_N|\le\varepsilon''$ for $N\ge N_1$, and $(R_N)$ has limit $0$.

Step 4 ($\mathrm{II}_N$ and $\mathrm{III}_N$ vanish). By the componentwise estimate $|x^\gamma|\le|x|$ of the componentwise calculus toolkit and $|Z^{\gamma\delta}_s|\le C_Z$, at every point $|\mathfrak{s}_s\cdot Z_se_s|\le\sum_{\gamma,\delta}C_Z|\mathfrak{s}^\gamma_s||e^\delta_s|\le C_Z\,l^2\,|\mathfrak{s}_s||e_s|$, so by conclusion (b) of the completion-of-squares theorem, $|\mathfrak{s}_s\cdot Z_se_s|\le C_Zl^2c_e\,N^{-1/2}|\mathfrak{s}_s|\big(|\mathfrak{s}_s|^2+|\mathfrak{a}_s|^2\big)$. Pointwise, $(|\mathfrak{s}|-|\mathfrak{s}|^2)^2\ge0$ gives $|\mathfrak{s}|^3\le\tfrac12(|\mathfrak{s}|^2+|\mathfrak{s}|^4)$, and $(|\mathfrak{s}|-|\mathfrak{a}|^2)^2\ge0$ gives $|\mathfrak{s}||\mathfrak{a}|^2\le\tfrac12(|\mathfrak{s}|^2+|\mathfrak{a}|^4)$; hence with Step 1 and (M), $\mathbb{E}\big[|\mathfrak{s}_s|(|\mathfrak{s}_s|^2+|\mathfrak{a}_s|^2)\big]\le M_2+M=:M_3$ for every $s$. Therefore $|2\,\mathbb{E}[\mathfrak{s}_s\cdot Z_se_s]|\le2\,\mathbb{E}\big[|\mathfrak{s}_s\cdot Z_se_s|\big]\le2C_Zl^2c_eM_3N^{-1/2}$ for every $s$ (monotonicity, and $\pm X\le|X|$), so by monotonicity of the interval integral $|\mathrm{II}_N|\le2TC_Zl^2c_eM_3\,N^{-1/2}$, and $(\mathrm{II}_N)$ has limit $0$. For $\mathrm{III}_N$: with $c_\Theta$ as in the covariance deviation lemma, its clause (b) and Step 1 give, for every $s$,

$$\Big|\sum_{\gamma,\delta=1}^{l}Z^{\gamma\delta}_s\Big(\mathbb{E}\big[\Theta^{\gamma\delta}(\Sigma_s,\alpha_s)\big]-\Theta^{\star\gamma\delta}_s\Big)\Big|\ \le\ C_Z\,l^2\,\frac{c_\Theta}{\sqrt{N}}\big(2M_2\big)^{1/2},$$

so $|\mathrm{III}_N|\le TC_Zl^2c_\Theta(2M_2)^{1/2}N^{-1/2}$, and $(\mathrm{III}_N)$ has limit $0$. Combining Steps 3--4 with the triangle inequality, $(r_N)$ has limit $0$. Together with the finiteness assertions of Step 2 this proves conclusion (a).

Step 5 (filtering bound). Fix $N$ and $t$, and write $\mathcal{K}=R_t^{-1}W_t^T$, a real matrix with $m$ rows and $l$ columns with entries bounded by $C_K$ (conclusion (a) of the completion-of-squares theorem). Each component of $\mathfrak{s}_t$ is bounded: every coordinate of the empirical state measure and of $S_t$ lies in $[0,1]$, both lying in the probability simplex at every $\omega$, so $|\mathfrak{s}^\gamma_t|\le\sqrt{N}$ at every $\omega$, and bounded random variables are square-integrable by monotonicity. Each component of $\mathfrak{a}_t$ satisfies $\mathbb{E}[(\mathfrak{a}^j_t)^2]\le\mathbb{E}[|\mathfrak{a}_t|^2]\le M_2$ (componentwise estimate and Step 1), hence is square-integrable, and is almost surely equal to a $\mathcal{G}^{(N)}_t$-measurable square-integrable random variable by claim 2 of the observation-adaptedness lemma, applied with the mean-field trajectory pair $(S,A)$ of the common data. Set $X=-\mathcal{K}\mathfrak{s}_t$ componentwise, so $X^i=-\sum_{\gamma=1}^{l}\mathcal{K}^{i\gamma}\mathfrak{s}^\gamma_t$ is square-integrable by the closure properties of the square-integrability definition, and $Y=\mathfrak{a}_t$; then $Y-X=u_t$ componentwise, and $\mathbb{E}[u_t\cdot R_tu_t]=\mathbb{E}[(Y-X)\cdot(R_t(Y-X))]$, the entry pairing of the completion-of-squares theorem and the index formula of the conditional mean-square optimality lemma being the identical double sum. Each $R_t$ is symmetric positive definite (conclusion (a) of the completion-of-squares theorem), hence positive semidefinite, so that lemma applies with $k=m$, $\mathcal{G}=\mathcal{G}^{(N)}_t$, and weight $R_t$: fixing conditional expectations $\mu^\gamma$ of $\mathfrak{s}^\gamma_t$ given $\mathcal{G}^{(N)}_t$, so that $\varepsilon^\gamma_t=\mathfrak{s}^\gamma_t-\mu^\gamma$ — each $\mu^\gamma$ being $\mathcal{G}^{(N)}_t$-measurable and square-integrable by the defining conditions of the conditional-expectation definition, so that each $\varepsilon^\gamma_t$ is square-integrable by the closure properties of the square-integrability definition — and conditional expectations $\mu_X^i$ of $X^i$,

$$\mathbb{E}\big[u_t\cdot R_tu_t\big]\ \ge\ \mathbb{E}\big[\varepsilon_X\cdot(R_t\varepsilon_X)\big],\qquad \varepsilon_X^i=X^i-\mu_X^i .$$

By the linearity of conditional expectation (part 1 of the basic properties lemma, applied finitely many times), $-\sum_\gamma\mathcal{K}^{i\gamma}\mu^\gamma$ is a conditional expectation of $X^i$, so $\mu_X^i=-\sum_\gamma\mathcal{K}^{i\gamma}\mu^\gamma$ almost surely by the uniqueness assertion of the existence and uniqueness theorem; hence $\varepsilon_X^i=-\sum_\gamma\mathcal{K}^{i\gamma}\varepsilon^\gamma_t$ almost surely. For square-integrable random variables $\varphi,\tilde{\varphi},\psi$ with $\varphi$ almost surely equal to $\tilde{\varphi}$ (the letters $U$ and $V$ being reserved for the open sets of the extension and $M$ for the moment bound) one has $\mathbb{E}[\varphi\psi]=\mathbb{E}[\tilde{\varphi}\psi]$, since $\lVert\varphi-\tilde{\varphi}\rVert_2=0$ by the null-equivalence statement of the square-integrability definition and $|\mathbb{E}[(\varphi-\tilde{\varphi})\psi]|\le\lVert\varphi-\tilde{\varphi}\rVert_2\lVert\psi\rVert_2=0$ by the Cauchy--Schwarz inequality; a product both of whose factors are replaced by almost-sure equals requires two applications, one factor at a time. Applying this and the linearity of the integral entrywise,

$$\mathbb{E}\big[\varepsilon_X\cdot(R_t\varepsilon_X)\big]=\sum_{i,i'=1}^{m}R^{ii'}_t\sum_{\gamma,\delta=1}^{l}\mathcal{K}^{i\gamma}\mathcal{K}^{i'\delta}\,\mathbb{E}\big[\varepsilon^\gamma_t\varepsilon^\delta_t\big]=\sum_{\gamma,\delta=1}^{l}\big(\mathcal{K}^T R_t\,\mathcal{K}\big)^{\gamma\delta}\,\mathbb{E}\big[\varepsilon^\gamma_t\varepsilon^\delta_t\big],$$

by the entry formulas for matrix products and the transpose. Since $R_t$ is symmetric positive definite, $R_t^{-1}$ is symmetric positive definite by the invertibility lemma, and

$$\mathcal{K}^TR_t\mathcal{K}=(R_t^{-1}W_t^T)^TR_t(R_t^{-1}W_t^T)=W_tR_t^{-1}R_tR_t^{-1}W_t^T=W_tR_t^{-1}W_t^T=\Xi_t,$$

using the reverse-order law for the transpose of a product (claim 3 of the componentwise calculus toolkit), the involutivity of the transpose (immediate from its definition), and the symmetry of $R_t^{-1}$. The matrix $\Xi_t$ is symmetric, $\Xi_t^T=W_t(R_t^{-1})^TW_t^T=\Xi_t$, and positive semidefinite: for $x\in\mathbb{R}^l$, $x\cdot(\Xi_tx)=(W_t^Tx)\cdot\big(R_t^{-1}(W_t^Tx)\big)\ge0$, using the adjoint identity of claim 3 of the toolkit (a matrix moves across the dot product as its transpose) and the positive definiteness of $R_t^{-1}$. Finally $\sum_{\gamma\delta}\Xi^{\gamma\delta}_t\mathbb{E}[\varepsilon^\gamma_t\varepsilon^\delta_t]=\mathbb{E}[\varepsilon_t\cdot(\Xi_t\varepsilon_t)]\ge0$ by claim 1 of the expected quadratic form lemma and the pointwise nonnegativity of $\varepsilon_t(\omega)\cdot(\Xi_t\varepsilon_t(\omega))$ with monotonicity of the integral. If $\bar{\mu}^\gamma$ is another choice of conditional expectations, each $\bar{\mu}^\gamma$ is almost surely equal to $\mu^\gamma$ by the uniqueness assertion, so each product expectation $\mathbb{E}[\varepsilon^\gamma_t\varepsilon^\delta_t]$ is unchanged, again by two applications of the almost-sure substitution above; the middle quantity is therefore independent of the choice. This proves conclusion (b).

Step 6 (lower bound). By claim 1 of the expected quadratic form lemma, $\mathbb{E}[\mathfrak{s}_0\cdot Z_0\mathfrak{s}_0]=\sum_{\gamma\delta}Z_0^{\gamma\delta}\mathbb{E}[\mathfrak{s}_0^\gamma\mathfrak{s}_0^\delta]$. Let $\varepsilon'>0$. By hypothesis (I) there is $N_1$ such that $|\mathbb{E}[\mathfrak{s}^{(N),\gamma}_0\mathfrak{s}^{(N),\delta}_0]-\Pi_0^{\gamma\delta}|\le\varepsilon'/(2l^2C_Z+2)$ for all $\gamma,\delta$ and $N\ge N_1$, whence $\big|\mathbb{E}[\mathfrak{s}_0\cdot Z_0\mathfrak{s}_0]-\sum_{\gamma\delta}Z_0^{\gamma\delta}\Pi_0^{\gamma\delta}\big|\le l^2C_Z\cdot\varepsilon'/(2l^2C_Z+2)\le\varepsilon'/2$; and by conclusion (a) there is $N_2$ with $|r_N|\le\varepsilon'/2$ for $N\ge N_2$. For $N\ge N_0=\max(N_1,N_2)$, the identity of conclusion (a) gives

$$\mathcal{J}_N\ \ge\ \sum_{\gamma,\delta=1}^{l}Z^{\gamma\delta}_0\Pi^{\gamma\delta}_0-\tfrac{\varepsilon'}{2}\;+\;\int_{[0,T]}\mathbb{E}\big[u^{(N)}_s\cdot R_su^{(N)}_s\big]ds\;+\;\int_{[0,T]}\sum_{\gamma,\delta=1}^{l}Z^{\gamma\delta}_s\Theta^{\star\gamma\delta}_s\,ds\;-\;\tfrac{\varepsilon'}{2},$$

which is conclusion (c). $\square$