Proof of Mean-Square Linearization Residual of the State Fluctuation Process

Throughout, adopt the notation of the statement ($g_s$, $z_s$, $\Lambda$, $E_s$, $\mathsf{B}_s$, $e_s$, $\Omega_{\mathfrak{a}}$, $\mathcal{R}_t$), write $x_s=(S_s,A_s)$ and $y_s=(\Sigma_s,\alpha_s)$ as points of $\mathbb{R}^{l+m}$ under the coordinate identification of the extension definition, so that $y_s-x_s=z_s/\sqrt{N}$ pointwise, and write $\bar{p}=(1/l,\dots,1/l)$, a point of the probability simplex $\Delta^l$. All integrals over $[0,t]$ below are $0$ for $t=0$; at $t=0$ the identity of clause (c) reduces to $M_0=0$, which is the $t=0$ case of the definition of $M^\gamma$ in clause (b) of the martingale decomposition, and both displays of clause (d) read $0\le0$. Accordingly, wherever an integral over $[0,t]$ is manipulated we take $t>0$.

Step 1: pointwise bounds (clause (a)). Fix $(s,\omega)\in[0,T]\times\Omega$. By the definition of the controlled $N$-agent dynamics, the state processes take values in $\{1,\dots,l\}$ at every point, so the empirical state measure $\Sigma_s(\omega)$ lies in $\Delta^l$; and $S_s\in\Delta^l$, the mean-field trajectory pair having $S:[0,T]\to\Delta^l$. Moreover $\alpha_s(\omega)\in\mathcal{A}$ at every point of $\Omega$ — the control process of a solution takes values in $\mathcal{A}$ by that definition's solution data, the policy being $\mathcal{A}$-valued — and $A_s\in\mathcal{A}$ (the trajectory pair having $A:[0,T]\to\mathcal{A}$). Hence $x_s$ and $y_s$ lie in $\Delta^l\times\mathcal{A}\subseteq\Delta^l\times V$, and every point of the segment $\{x_s+\tau(y_s-x_s):\tau\in[0,1]\}$ lies in $\Delta^l\times V$: the convex combination $(1-\tau)S_s+\tau\Sigma_s(\omega)$ of two points of the simplex has nonnegative entries with sum $(1-\tau)+\tau=1$, and the control coordinates form a convex combination of the two points $A_s$ and $\alpha_s(\omega)$ of $\mathcal{A}\subseteq V$, the set $V$ being convex by the extension definition. By clause (i) of the regularity of the extended aggregate state drift, $\bar{b}$ agrees with $b$ on $\Delta^l\times\mathcal{A}$ — which contains both $x_s$ and $y_s$ — and each $\bar{b}^\gamma$ is a $C^1$ map on the open set $U\times V$ whose partial derivatives are again $C^1$; also $\Delta^l\times V\subseteq U\times V$. Therefore the Taylor lemma applies to $f=\bar{b}^\gamma$ with $n=l+m$, $x=x_s$, $y=y_s$, $h=z_s(\omega)/\sqrt{N}$. By the definitions of $E_s$ and $\mathsf{B}_s$,

Part (ii) of the Taylor lemma with $M_2=3\,l\,K$ — a bound for all second-order partials of $\bar{b}^\gamma$ on $\Delta^l\times V$, hence on the segment, by clause (iii) of the drift regularity lemma — gives

and $|e_s|\le\sqrt{l}\,\max_\gamma|e^\gamma_s|$ (the elementary inequality $\sum_\gamma(e^\gamma_s)^2\le l\,\max_\gamma(e^\gamma_s)^2$) yields the first bound of clause (a). The Lipschitz estimate of clause (ii) of the drift regularity lemma — available because $\mathcal{A}$ is convex by hypothesis — applied to the points $x_s,y_s\in\Delta^l\times\mathcal{A}$, whose Euclidean distance is $|z_s|/\sqrt{N}$, gives $|g^\gamma_s|=\sqrt{N}\,\big|\bar{b}^\gamma(y_s)-\bar{b}^\gamma(x_s)\big|\le\sqrt{N}\cdot\sqrt{l+m}\;l\,(B+K)\,|z_s|/\sqrt{N}$, so $|g_s|\le\sqrt{l}\,\sqrt{l+m}\;l\,(B+K)\,|z_s|=\Lambda\,|z_s|$ by the same elementary inequality; and $|(E_s\mathfrak{s}_s+\mathsf{B}_s\mathfrak{a}_s)^\gamma|\le\sum_{i=1}^{l+m}|\partial_i\bar{b}^\gamma(x_s)|\,|z^i_s|\le l\,(B+K)\,\sqrt{l+m}\,|z_s|$, by clause (ii) and the elementary inequality $\sum_{i=1}^{l+m}|z^i_s|\le\sqrt{l+m}\,|z_s|$ (an instance of $2\lambda\mu\le\lambda^2+\mu^2$ summed over indices), so $|E_s\mathfrak{s}_s+\mathsf{B}_s\mathfrak{a}_s|\le\Lambda\,|z_s|$. The triangle inequality (the Euclidean distance is a metric) gives $|e_s|\le|g_s|+|E_s\mathfrak{s}_s+\mathsf{B}_s\mathfrak{a}_s|\le2\,\Lambda\,|z_s|$. No use of the hypothesis $\mathcal{A}_2<\infty$ was made.

Step 2: measurability and the residual process (clause (b)). Fix a point $a_0\in\mathcal{A}$ (the control set being nonempty) and define $\tilde{\Sigma}^\gamma_s=\mathbf{1}_{\Omega_0}\Sigma^\gamma_s+(1-\mathbf{1}_{\Omega_0})\,\bar{p}^\gamma$ and $\tilde{\alpha}^j_s=\mathbf{1}_{\Omega_0}\alpha^j_s+(1-\mathbf{1}_{\Omega_0})\,a_0^j$, so that $(\tilde{\Sigma}_s(\omega),\tilde{\alpha}_s(\omega))\in\Delta^l\times\mathcal{A}$ at every point (on $\Omega_0$ by Step 1, off $\Omega_0$ by construction). The maps $(s,\omega)\mapsto\mathbf{1}_{\Omega_0}(\omega)\Sigma^\gamma_s(\omega)$ and $(s,\omega)\mapsto\mathbf{1}_{\Omega_0}(\omega)\alpha^j_s(\omega)$ are measurable with respect to the product $\sigma$-algebra of the trace Borel $\sigma$-algebra on $[0,T]$ and $\mathcal{F}$, by clauses (b) and (c) of the joint measurability of the state and control; and $(s,\omega)\mapsto\mathbf{1}_{\Omega_0}(\omega)$ is product-measurable as the indicator of the measurable rectangle $[0,T]\times\Omega_0$ ($\Omega_0\in\mathcal{F}$ by the controlled-dynamics definition, and rectangles generate the product $\sigma$-algebra). Hence $\tilde{\Sigma}^\gamma$ and $\tilde{\alpha}^j$ are product-measurable. The function

is jointly continuous, hence sequentially continuous, on $[0,T]\times U\times V\subseteq\mathbb{R}^{1+l+m}$, since $\bar{b}^\gamma$ and its partial derivatives are continuous (clause (i) of the drift regularity lemma) and the trajectory pair $(S,A)$ is continuous; and, using $\bar{b}=b$ on $\Delta^l\times\mathcal{A}$ (Step 1), $\mathbf{1}_{\Omega_0}e^\gamma_s=\mathbf{1}_{\Omega_0}\cdot F^\gamma(s,\tilde{\Sigma}_s,\tilde{\alpha}_s)$ at every point of $[0,T]\times\Omega$, the arguments lying in the domain of $F^\gamma$ everywhere since $(\tilde{\Sigma}_s,\tilde{\alpha}_s)\in\Delta^l\times\mathcal{A}\subseteq U\times V$. By measurability of sequentially continuous functions of measurable Euclidean maps — applied to $F^\gamma$ composed with the measurable maps $(s,\omega)\mapsto s$, $\tilde{\Sigma}^\gamma$, $\tilde{\alpha}^j$, and once more to the product with $\mathbf{1}_{\Omega_0}$ — each $(s,\omega)\mapsto\mathbf{1}_{\Omega_0}e^\gamma_s$ is product-measurable. The same argument makes $(s,\omega)\mapsto\mathbf{1}_{\Omega_0}|\mathfrak{s}_s|$, $(s,\omega)\mapsto\mathbf{1}_{\Omega_0}|\mathfrak{a}_s|$, their squares, and, for Step 4, $(s,\omega)\mapsto\mathbf{1}_{\Omega_0}w_s$ with $w_s=\min\big(\tfrac{3l\sqrt{l}K(l+m)}{2\sqrt{N}}|z_s|^2,\,2\Lambda|z_s|\big)^2$ and $(s,\omega)\mapsto\mathbf{1}_{\Omega_0}|z_s|^4$, product-measurable.

By the Tonelli theorem (the trace Lebesgue measure of the toolkit and $P$ being finite, hence $\sigma$-finite, measures), $\Phi_{\mathfrak{a}}(\omega)=\int_{[0,T]}\mathbf{1}_{\Omega_0}(\omega)\,|\mathfrak{a}_s(\omega)|^2\,ds$ defines a measurable $[0,\infty]$-valued function of $\omega$ with $\mathbb{E}[\Phi_{\mathfrak{a}}]=\mathcal{A}_2$ (the expectations defining $\mathcal{A}_2$ are unchanged by the $\mathbf{1}_{\Omega_0}$ modification because $\Omega_0$ has probability $1$, as recorded in clause (a) of the a priori second-moment bound); for $\omega\in\Omega_0$ its defining section is $s\mapsto|\mathfrak{a}_s(\omega)|^2$, so $\Omega_{\mathfrak{a}}=\Omega_0\cap\{\omega:\Phi_{\mathfrak{a}}(\omega)<\infty\}$ is an event. For every natural number $n$ one has $\Phi_{\mathfrak{a}}\ge n\,\mathbf{1}_{\{\Phi_{\mathfrak{a}}=\infty\}}$ pointwise, so monotonicity and the integral of a simple function give $\mathcal{A}_2\ge n\,P(\Phi_{\mathfrak{a}}=\infty)$, forcing $P(\Phi_{\mathfrak{a}}=\infty)=0$; as $\Omega_0$ has probability $1$, so does $\Omega_{\mathfrak{a}}$.

Fix $\omega\in\Omega_{\mathfrak{a}}$ and $t\in(0,T]$. The sections $s\mapsto e^\gamma_s(\omega)$ are measurable on $[0,T]$: they are the differences of the sections at $\omega$ of the product-measurable maps $(\mathbf{1}_{\Omega_0}e^\gamma)^{+}$ and $(\mathbf{1}_{\Omega_0}e^\gamma)^{-}$ (product-measurable as continuous functions of $\mathbf{1}_{\Omega_0}e^\gamma$; sections of product-measurable $[0,\infty]$-valued maps are measurable, as in the Tonelli theorem), and $\mathbf{1}_{\Omega_0}(\omega)=1$; the same argument gives measurability of the sections at $\omega$ of $\mathbf{1}_{\Omega_0}|\mathfrak{s}_s|$, $\mathbf{1}_{\Omega_0}|\mathfrak{a}_s|$, and their squares. By Step 1 and the estimates $|\mathfrak{s}_s(\omega)|\le2\sqrt{N}$ (points of $\Delta^l$ have Euclidean norm at most $1$, their entries lying in $[0,1]$ with sum $1$, so that $\sum_\gamma(\Sigma^\gamma)^2\le\sum_\gamma\Sigma^\gamma=1$, and $|\Sigma_s-S_s|\le|\Sigma_s|+|S_s|\le2$ by the triangle inequality, the Euclidean distance being a metric), $|z_s|\le|\mathfrak{s}_s|+|\mathfrak{a}_s|$ (the square of the left side being the sum of the squares of the two blocks), and $|\mathfrak{a}_s|\le\tfrac12(1+|\mathfrak{a}_s|^2)$,

and moreover, by the second bound of clause (a),

both by monotonicity (using $|e^\gamma_s|^2\le|e_s|^2\le4\Lambda^2|z_s|^2=4\Lambda^2(|\mathfrak{s}_s|^2+|\mathfrak{a}_s|^2)$); so each section is Lebesgue integrable over $[0,t]$, with square-integrable modulus, and $\mathcal{R}_t$ is defined as in the statement. Each $\mathcal{R}^\gamma_t$ is a random variable: by the Tonelli theorem the integrals over $[0,t]$ of $(\mathbf{1}_{\Omega_0}e^\gamma)^{\pm}$ define measurable $[0,\infty]$-valued functions of $\omega$, finite on $\Omega_{\mathfrak{a}}$ by the first bound just displayed, and $\mathcal{R}^\gamma_t$ is the difference of the functions equal to them on $\Omega_{\mathfrak{a}}$ and to $0$ off $\Omega_{\mathfrak{a}}$.

Step 3: integral form (clause (c)). Let $\Omega_*=\Omega_{\mathfrak{a}}$, an event of probability $1$ (Step 2) contained in $\Omega_0$, on which clause (a) of the martingale decomposition applies. Fix $\omega\in\Omega_*$, $\gamma\in\{1,\dots,l\}$, and $t\in[0,T]$. First, the relevant sections are measurable and integrable over $[0,t]$ at $\omega$: the path $s\mapsto b^\gamma(\Sigma_s,\alpha_s)$ is measurable and bounded in absolute value by $2(l-1)B$ by clause (a) of the martingale decomposition; the function $s\mapsto b^\gamma(S_s,A_s)$ is continuous — this is part of clause 2 of the mean-field trajectory pair — hence measurable, and bounded in absolute value by $2(l-1)B$ as well, since by the definition of the aggregate state drift $b^\gamma(\Sigma,\alpha)$ is a sum over the $l-1$ states $\sigma\neq\gamma$ of terms $\Sigma^\sigma\beta(\sigma,\gamma,\Sigma,\alpha)-\Sigma^\gamma\beta(\gamma,\sigma,\Sigma,\alpha)$ with $0\le\Sigma^\sigma,\Sigma^\gamma\le1$ on $\Delta^l$ and $|\beta|\le B$ by the rate bound of the transition-rate family, so that $|b^\gamma|\le2(l-1)B$ at every point of $\Delta^l\times\mathcal{A}$; hence $s\mapsto g^\gamma_s(\omega)$ is measurable and bounded by $4(l-1)B\sqrt{N}$, and integrable over $[0,t]$; the section $s\mapsto e^\gamma_s(\omega)$ is measurable and integrable by Step 2; and the section $s\mapsto(E_s\mathfrak{s}_s+\mathsf{B}_s\mathfrak{a}_s)^\gamma(\omega)=g^\gamma_s(\omega)-e^\gamma_s(\omega)$ is measurable as their difference and dominated by $\Lambda\,|z_s(\omega)|$, integrable over $[0,t]$ as in Step 2. Now, by the definition of $M^\gamma$ in clause (b) of the martingale decomposition — whose integrand $\mathbf{1}_{\Omega_0}\,b^\gamma(\Sigma_s,\alpha_s)$ has section at $\omega$ equal to $b^\gamma(\Sigma_s,\alpha_s)$, since $\omega\in\Omega_*\subseteq\Omega_{\mathfrak{a}}\subseteq\Omega_0$ —

and by clause 2 of the mean-field trajectory pair, $S^\gamma_t=S^\gamma_0+\int_0^tb^\gamma(S_s,A_s)\,ds$ as a Riemann integral, which agrees with the Lebesgue integral over $[0,t]$ by claim 3 of the toolkit, applied for $t>0$ to the restriction of the continuous integrand — continuous by claim 1 of restriction stability — (both integrals are $0$ for $t=0$). Subtracting and multiplying by $\sqrt{N}$,

and by additivity of the Lebesgue integral (linearity) applied to the three integrable sections above,

Substituting into the previous display gives the identity of clause (c) at every $\omega\in\Omega_*$, hence almost surely.

Step 4: mean-square bound (clause (d)). Fix $t\in(0,T]$ (the case $t=0$ was disposed of in the preamble). At each $\omega\in\Omega_{\mathfrak{a}}$ and each $\gamma$, the sections $e^\gamma(\omega)$ are measurable with $\int_{[0,t]}(e^\gamma_s)^2\,ds<\infty$ (Step 2), so claim 4 (Cauchy--Schwarz) of the interval toolkit, applied to the pair $(1,e^\gamma)$ on $[0,t]$ — the constant function $1$ being measurable with $\int_{[0,t]}1^2\,ds=t<\infty$ — gives $(\mathcal{R}^\gamma_t)^2=\big(\int_{[0,t]}e^\gamma_s\,ds\big)^2\le t\int_{[0,t]}(e^\gamma_s)^2\,ds$; summing over $\gamma$ (linearity of the Lebesgue integral), $|\mathcal{R}_t|^2\le t\int_{[0,t]}|e_s|^2\,ds$ on $\Omega_{\mathfrak{a}}$, while $|\mathcal{R}_t|^2=0$ off $\Omega_{\mathfrak{a}}$. By Step 1, $|e_s|^2\le w_s$ at every point, $w_s$ being the square of the minimum of the two bounds of clause (a), so, pointwise on $\Omega$,

using $\Omega_{\mathfrak{a}}\subseteq\Omega_0$. The function $|\mathcal{R}_t|^2$ is a random variable (a continuous function of the random variables $\mathcal{R}^\gamma_t$ of Step 2), so taking expectations and using monotonicity and the Tonelli theorem for the product-measurable map $\mathbf{1}_{\Omega_0}w$ of Step 2,

the last equality because $w_s$ is a random variable (a continuous function of the components of $z_s$, again by the composition lemma) and expectations are unchanged by the $\mathbf{1}_{\Omega_0}$ modification, $\Omega_0$ having probability $1$, as recorded in clause (a) of the a priori second-moment bound and in clause (d) of the statement. This is the first display of clause (d). The second display follows from $w_s\le\frac{9\,l^3K^2(l+m)^2}{4N}\,|z_s|^4$ pointwise (the minimum is at most its first argument) by monotonicity of the expectation and of the integral, the map $s\mapsto\mathbb{E}[|z_s|^4]$ being measurable as recorded in the statement.\ $\square$