Mean-Square Linearization Residual of the State Fluctuation Process

Adopt the setting of the fluctuation processes of the controlled $N$-agent dynamics: a transition-rate family $\beta$ on $l$ states with control set $\mathcal{A}$, a nonempty subset of Euclidean space $\mathbb{R}^m$, and rate bound $B$, an observation-rate family $\tilde{\beta}$, a horizon $T>0$, an $N$-agent driving system $(\Omega,\mathcal{F},P)$, an observation-driven control policy $h$ which is $\mathcal{A}$-valued, a solution on $[0,T]$ with regular event $\Omega_0$, empirical state measure $\Sigma_t$, and control $\alpha_t$, a mean-field trajectory pair $(S,A)$ for $\beta$ with horizon $T$, and the fluctuation processes $\mathfrak{s}_t=\sqrt{N}(\Sigma_t-S_t)$ and $\mathfrak{a}_t=\sqrt{N}(\alpha_t-A_t)$. Assume that $\beta$ admits a twice continuously differentiable extension $(U,V,\bar{\beta})$ with derivative bound $K$, that the control set $\mathcal{A}$ is convex, and that

$$\mathcal{A}_2=\int_{[0,T]}\mathbb{E}\big[|\mathfrak{a}_t|^2\big]\,dt<\infty,$$

this integral being well defined in $[0,\infty]$ by clause (a) of the a priori second-moment bound; as in that lemma, the subscripted symbol $\mathcal{A}_2$ is distinct from the control set $\mathcal{A}$. Let $b$ be the aggregate state drift of $\beta$, let $\bar{b}$ be the extended aggregate state drift of $(U,V,\bar{\beta})$, adopt the partial-derivative notation $\partial_i$, $\partial_j\partial_i$ of the extension definition, and let $M_t=(M^\gamma_t)_{\gamma\in\{1,\dots,l\}}$ be as in the martingale decomposition. Write $\mathbb{E}$ for the expectation, $|\cdot|$ for the Euclidean norm (Euclidean distance to the origin), $\mathbf{1}_{D}$ for the function equal to $1$ on a set $D$ and $0$ off $D$, and set, for $s\in[0,T]$,

$$g_s=\sqrt{N}\,\big(b(\Sigma_s,\alpha_s)-b(S_s,A_s)\big)\in\mathbb{R}^l,\qquad z_s=(\mathfrak{s}_s,\mathfrak{a}_s)\in\mathbb{R}^{l+m},\qquad \Lambda=l\,(B+K)\,\sqrt{l\,(l+m)}.$$

Define, as in the statement of the completion-of-squares and coercivity theorem and with the same symbols, the linearization coefficient matrices: for $s\in[0,T]$, the real $l\times l$ matrix $E_s$ and the real $l\times m$ matrix $\mathsf{B}_s$ (the sans-serif $\mathsf{B}_s$ is distinct from the rate bound $B$, and the matrix $E_s$ is distinct from the expectation $\mathbb{E}$) with entries

$$E^{\gamma\sigma}_s=\partial_\sigma\bar{b}^\gamma(S_s,A_s)\quad(\gamma,\sigma\in\{1,\dots,l\}),\qquad\qquad \mathsf{B}^{\gamma j}_s=\partial_{l+j}\bar{b}^\gamma(S_s,A_s)\quad(\gamma\in\{1,\dots,l\},\ j\in\{1,\dots,m\}),$$

which are defined because $S_s$ lies in the probability simplex $\Delta^l$ and $A_s$ in $\mathcal{A}$, so that $(S_s,A_s)\in\Delta^l\times\mathcal{A}\subseteq U\times V$ (the inclusions $\Delta^l\subseteq U$ and $\mathcal{A}\subseteq V$ are part of the extension definition), and the partial derivatives of $\bar{b}$ exist there by the regularity of the extended aggregate state drift. Define the linearization residual, with the matrix-vector products $E_s\mathfrak{s}_s$ and $\mathsf{B}_s\mathfrak{a}_s$,

$$e_s=g_s-E_s\mathfrak{s}_s-\mathsf{B}_s\mathfrak{a}_s\in\mathbb{R}^l\qquad(s\in[0,T]).$$

Then:

(a) (Pointwise bounds.) At every point of $[0,T]\times\Omega$,

$$|e_s|\ \le\ \frac{3\,l\,\sqrt{l}\,K\,(l+m)}{2\,\sqrt{N}}\,|z_s|^2\qquad\text{and}\qquad |e_s|\ \le\ 2\,\Lambda\,|z_s|.$$

The first bound coincides with part (b) of the completion-of-squares and coercivity theorem (whose constant $c_e$ equals $\tfrac{3}{2}\,l\,\sqrt{l}\,(l+m)\,K$); neither bound uses the hypothesis $\mathcal{A}_2<\infty$.

(b) (Well-definedness.) Each map $(s,\omega)\mapsto\mathbf{1}_{\Omega_0}(\omega)\,e^\gamma_s(\omega)$ is measurable with respect to the product $\sigma$-algebra of the trace Borel $\sigma$-algebra on $[0,T]$ and $\mathcal{F}$, by the joint measurability of the state and control together with measurability of sequentially continuous functions of measurable Euclidean maps. The set $\Omega_{\mathfrak{a}}$ of those $\omega\in\Omega_0$ at which the Lebesgue integral $\int_{[0,T]}|\mathfrak{a}_s(\omega)|^2\,ds$ is finite is an event of probability $1$ (by the Tonelli theorem and $\mathcal{A}_2<\infty$), and at each $\omega\in\Omega_{\mathfrak{a}}$ the section $s\mapsto e^\gamma_s(\omega)$ is Lebesgue integrable over $[0,t]$ for every $t\in[0,T]$. Define the residual process

$$\mathcal{R}_t=\int_{[0,t]}e_s\,ds\ \text{ componentwise at each }\omega\in\Omega_{\mathfrak{a}},\qquad \mathcal{R}_t=0\ \text{ off }\Omega_{\mathfrak{a}}\qquad(t\in[0,T]).$$

Each $\mathcal{R}^\gamma_t$ is a random variable: the integrals over $[0,t]$ of the positive and negative parts of $\mathbf{1}_{\Omega_0}e^\gamma$ (product-measurable, being continuous functions of it) define measurable $[0,\infty]$-valued functions of $\omega$ by the Tonelli theorem, both finite on $\Omega_{\mathfrak{a}}$, and $\mathcal{R}^\gamma_t$ is the difference of the functions equal to them on $\Omega_{\mathfrak{a}}$ and to $0$ off $\Omega_{\mathfrak{a}}$.

(c) (Integral form of the fluctuation dynamics.) At every $\omega\in\Omega_{\mathfrak{a}}$ — an event of probability $1$ contained in the regular event $\Omega_0$, on which clause (a) of the martingale decomposition applies — and hence almost surely, for every $t\in[0,T]$,

$$\mathfrak{s}_t=\mathfrak{s}_0+\int_{[0,t]}\big(E_s\mathfrak{s}_s+\mathsf{B}_s\mathfrak{a}_s\big)\,ds+\sqrt{N}\,M_t+\mathcal{R}_t,$$

all integrals existing componentwise at such $\omega$.

(d) (Mean-square residual bound.) For every $t\in[0,T]$, with both sides valued in $[0,\infty]$,

$$\mathbb{E}\big[|\mathcal{R}_t|^2\big]\ \le\ t\int_{[0,t]}\mathbb{E}\Big[\min\Big(\frac{3\,l\,\sqrt{l}\,K\,(l+m)}{2\,\sqrt{N}}\,|z_s|^2,\ 2\,\Lambda\,|z_s|\Big)^{\!2}\Big]\,ds .$$

Here the maps $(s,\omega)\mapsto\mathbf{1}_{\Omega_0}(\omega)\min(\cdots)^2$ and $(s,\omega)\mapsto\mathbf{1}_{\Omega_0}(\omega)|z_s(\omega)|^4$ are product-measurable, being continuous functions of the product-measurable maps of the joint measurability lemma, so the integrands $s\mapsto\mathbb{E}[\min(\cdots)^2]$ and $s\mapsto\mathbb{E}[|z_s|^4]$ (unchanged by the $\mathbf{1}_{\Omega_0}$ modification, $\Omega_0$ having probability $1$) are measurable $[0,\infty]$-valued functions of $s$ by the Tonelli theorem. In particular

$$\mathbb{E}\big[|\mathcal{R}_t|^2\big]\ \le\ \frac{9\,l^3\,K^2\,(l+m)^2}{4\,N}\;t\int_{[0,t]}\mathbb{E}\big[|z_s|^4\big]\,ds .$$