Proof of Moment Bounds for the Aggregate Compensated Counters of the Controlled N-Agent Dynamics

Fix a solution and an ordered pair $(\sigma,\gamma)$ of distinct states, and write $a_i=(i,\sigma\gamma)$ for the corresponding transition clock labels, $i\in\{1,\dots,N\}$, so that $\mathcal{N}^{\sigma\gamma}_t=\sum_iN^{a_i}_t$, $\mathfrak{A}^{\sigma\gamma}_t=\sum_i\mathcal{T}^{a_i}_t$, and $\mathfrak{M}^{\sigma\gamma}_t=\sum_iM^{a_i}_t$ in the notation of the compensated-counters lemma. We abbreviate $\mathfrak{M}=\mathfrak{M}^{\sigma\gamma}$, $\mathfrak{A}=\mathfrak{A}^{\sigma\gamma}$ while the pair is fixed (Steps 1--3); Step 4 releases it. Throughout we use the multiplier identities and interval estimates for this solution, with their constant $C_\star$ and $\bar{\beta}=\max(B,\tilde{B})$; by part (c) of that lemma every finite product of compensated counters at times in $[0,T]$ is integrable, so in particular $\mathfrak{M}_t^{\,2}$, $\mathfrak{M}_t^{\,4}$, $\mathfrak{M}_t^{\,6}$, and $\mathfrak{M}_t^{\,8}$ have finite expectations (each expands into a finite sum of such products), and all expectations split termwise below. $\Omega_0$ denotes the regular event; it has probability $1$, so expectations are unchanged by modifications off $\Omega_0$.

Step 1: part (a). Martingale. By part (a) of the compensated-counters lemma, each $M^{a_i}$ is a square-integrable martingale for the system filtration with $M^{a_i}_0=0$. A finite sum of square-integrable martingales is again one: adaptedness is preserved under finite sums, square-integrability is preserved by the triangle inequality for the mean-square norm, and the martingale property in the averaged form of the definition is linear in the process. Hence $\mathfrak{M}$ is a square-integrable martingale with $\mathfrak{M}_0=\sum_iM^{a_i}_0=0$.

Counting path. Work at an outcome in $\Omega_0$. Condition 3 of the solution definition requires that each individual counter, and also the grand total of all counters, coincide on $[0,T]$ with restrictions of counting paths; let $c_1,\dots,c_N$ be counting paths agreeing on $[0,T]$ with $t\mapsto N^{a_1}_t,\dots,t\mapsto N^{a_N}_t$, and $c_g$ one agreeing with the grand total. (The aggregate $\mathcal{N}^{\sigma\gamma}$ itself is not among the maps covered by condition 3; that is what we now prove.) Define $c(t)=\sum_{i=1}^{N}c_i(\min(t,T))$ for $t\ge0$. Then $c(0)=0$, $c$ takes values that are $0$ or natural numbers (finite sums of such), and $c$ is nondecreasing. Right-continuity: each $t\mapsto c_i(\min(t,T))$ is nondecreasing, and for nondecreasing functions the greatest lower bound over $s>t$ is the limit along any sequence decreasing to $t$, so the greatest lower bound of a finite sum is the sum of the greatest lower bounds; each summand is right-continuous (for $t\ge T$ it is constant, and for $t<T$ this is right-continuity of $c_i$ at $\min(t,T)=t$), hence so is $c$. Unit jumps: for $0<t\le T$, the least upper bounds over $[0,t)$ likewise add for finite sums of nondecreasing functions, so $c(t)-c(t-)=\sum_i(c_i(t)-c_i(t-))$; on $[0,T]$ the grand total is the pointwise sum of all counter paths, so $c_g(t)-c_g(t-)$ equals the sum of the jumps at $t$ of all the counting paths agreeing with the individual counters, each such jump is nonnegative, and $c_g(t)-c_g(t-)\le1$ by the unit-jump property; hence $c(t)-c(t-)\le1$. For $t>T$, $c$ is constant on $[T,\infty)$, so $c(t)-c(t-)=0$ there. Thus $c$ is a counting path agreeing with $t\mapsto\mathcal{N}^{\sigma\gamma}_t$ on $[0,T]$; since $\Omega_0$ has probability $1$, the counting-path claim holds almost surely.

Compensator bound. By condition 2 of the solution definition, each $\mathcal{T}^{a_i}_t$ is, at every outcome, the Lebesgue integral over $[0,t]$ of a function $g$ with values in $[0,B]$ (the range bound is part (vii)(b) of the existence theorem). Passing to zero extensions (claim 2 of that toolkit): let $0<r<t\le T$, and write $\tilde{g}$ for the zero extension of $g|_{[0,t]}$ and $\tilde{g}_r$ for that of $g|_{[0,r]}$. Pointwise on $\mathbb{R}$, $\tilde{g}=\tilde{g}_r+\tilde{g}\,\mathbf{1}_{(r,t]}$ (the two summands agree with $\tilde{g}$ on $[0,r]$ and on $(r,t]$ respectively and vanish elsewhere), the interval $(r,t]$ being Borel by claim 2 of the Borel generator lemma and the product measurable by claims 1 and 3 of the measurable-arithmetic lemma; integrating, and using claim 2 of the toolkit on both subintervals together with the linearity of the Lebesgue integral, $\mathcal{T}^{a_i}_t-\mathcal{T}^{a_i}_r=\int_{\mathbb{R}}\tilde{g}\,\mathbf{1}_{(r,t]}\,d\lambda$; and $0\le\int_{\mathbb{R}}\tilde{g}\,\mathbf{1}_{(r,t]}\,d\lambda\le B\,\lambda\big((r,t]\big)\le B\,(t-r)$ by its monotonicity, the indicator-integral lemma, monotonicity of a measure, and claim 1 of the toolkit (whose restricted measure agrees with $\lambda$, so that $\lambda([r,t])=t-r$). For $r=t$ the increment is $0$, and for $r=0$ the bound reads $0\le\mathcal{T}^{a_i}_t\le Bt$, directly by monotonicity ($\mathcal{T}^{a_i}_0$ is the integral over the $\lambda$-null set $\{0\}$, hence $0$). Hence $0\le \mathcal{T}^{a_i}_t-\mathcal{T}^{a_i}_r\le B(t-r)$ for all $0\le r\le t\le T$ at every outcome. Summing over $i$ gives $0\le\mathfrak{A}_t-\mathfrak{A}_r\le NB(t-r)$, as claimed; in particular $\mathfrak{A}_0=0$ and $0\le\mathfrak{A}_t\le NBt$ everywhere.

Step 2: the second moment. Integrability of $\mathfrak{M}_t^{2}$ and $\mathfrak{M}_t^{4}$ was noted above. By part (b) of the compensated-counters lemma with $r=0$ and $D=\Omega$, together with $M^{a_i}_0=0$ and $\mathcal{T}^{a_i}_0=0$,

Expanding $\mathfrak{M}_t^{2}=\sum_{i,j}M^{a_i}_tM^{a_j}_t$ and summing,

using the pathwise bound of Step 1. This proves the second-moment display of (b). By the Cauchy--Schwarz inequality (against the constant $1$), also $\mathbb{E}[|\mathfrak{M}_s|]\le\sqrt{NBs}$ for every $s\in[0,T]$, with $\sqrt{\cdot}$ the nonnegative square root.

Step 3: the fourth moment. Fix $t\in(0,T]$ (for $t=0$ both sides of the claimed bound vanish). For a natural number $n\ge1$ set $\delta=t/n$ and $r_q=q\delta$ for $q\in\{0,\dots,n\}$, and write $\Delta_q\mathfrak{M}=\mathfrak{M}_{r_{q+1}}-\mathfrak{M}_{r_q}$, $\Delta_qM^{a_i}=M^{a_i}_{r_{q+1}}-M^{a_i}_{r_q}$, $\Delta_q\mathfrak{A}=\mathfrak{A}_{r_{q+1}}-\mathfrak{A}_{r_q}$, $\Delta_q\mathcal{T}^{a_i}=\mathcal{T}^{a_i}_{r_{q+1}}-\mathcal{T}^{a_i}_{r_q}$. The counters and the consumed clock times at times up to $r_q$ are $\mathcal{F}^{\mathrm{sys}}_{r_q}$-measurable by part (iv) of the existence theorem, so $\mathfrak{M}_{r_q}$ is $\mathcal{F}^{\mathrm{sys}}_{r_q}$-measurable (claim 2 of the measurable-arithmetic lemma), and so are its powers $\mathfrak{M}^{2}_{r_q}$, $\mathfrak{M}^{3}_{r_q}$ and the constants (claims 1 and 3 of that lemma), as the multiplier hypotheses below require. Telescoping $\mathfrak{M}_t^{4}=\sum_{q=0}^{n-1}(\mathfrak{M}_{r_{q+1}}^{4}-\mathfrak{M}_{r_q}^{4})$ (with $\mathfrak{M}_0=0$) and expanding $\mathfrak{M}_{r_{q+1}}^{4}=(\mathfrak{M}_{r_q}+\Delta_q\mathfrak{M})^{4}$ by the binomial theorem,

all terms being integrable as noted. We treat the four terms with the multiplier lemma, whose square-integrability hypotheses hold in each case because the required expectations ($\mathbb{E}[\mathfrak{M}_{r_q}^{6}]$, $\mathbb{E}[\mathfrak{M}_{r_q}^{8}]$, $\mathbb{E}[\mathfrak{M}_{r_q}^{4}(M^{a_i}_{r_q})^{2}]$, and so on) are finite by its part (c), and whose measurability hypotheses were checked above.

Cubic multiplier term. $\mathbb{E}[\mathfrak{M}_{r_q}^{3}\,\Delta_qM^{a_i}]=0$ for each $i$ by part (a) of the multiplier lemma with the square-integrable $\mathcal{F}^{\mathrm{sys}}_{r_q}$-measurable multiplier $Z=\mathfrak{M}_{r_q}^{3}$; summing over $i$, the first term vanishes.

Quadratic multiplier term. With $Z=\mathfrak{M}_{r_q}^{2}$, part (b) of the multiplier lemma gives, for all $i,j$, $\mathbb{E}[Z\,\Delta_qM^{a_i}\,\Delta_qM^{a_j}]=\mathbf{1}_{\{i=j\}}\mathbb{E}[Z\,\Delta_q\mathcal{T}^{a_i}]$ (we write $\mathbf{1}_{\{\cdot\}}$ for the indicator equal to $1$ when the subscripted condition holds and $0$ otherwise). Summing over $i,j$ and using the pathwise bounds $0\le\Delta_q\mathfrak{A}\le NB\delta$ of Step 1, $Z\ge0$, monotonicity of the expectation, and Step 2,

Linear multiplier term. Expand $(\Delta_q\mathfrak{M})^{3}=\sum_{i_1,i_2,i_3}\Delta_qM^{a_{i_1}}\Delta_qM^{a_{i_2}}\Delta_qM^{a_{i_3}}$, the indices running over $\{1,\dots,N\}$. For the $N^{3}-N$ triples with indices not all equal, part (f) of the multiplier lemma with the integrable multiplier $\mathfrak{M}_{r_q}$ bounds each expectation in absolute value by $C_\star\,\mathbb{E}[|\mathfrak{M}_{r_q}|]\,\delta^{2}\le C_\star\sqrt{NBT}\,\delta^{2}$. For the $N$ diagonal triples, write $\mathbb{E}[\mathfrak{M}_{r_q}(\Delta_qM^{a_i})^{3}]=\mathbb{E}[\mathfrak{M}_{r_q}((\Delta_qM^{a_i})^{3}-\Delta_q\mathcal{T}^{a_i})]+\mathbb{E}[\mathfrak{M}_{r_q}\,\Delta_q\mathcal{T}^{a_i}]$; the first expectation is bounded in absolute value by $C_\star(1+NBT)\,\delta^{3/2}$ by part (e) of the multiplier lemma with power $k=3$ and $\mathbb{E}[\mathfrak{M}_{r_q}^{2}]\le NBT$, and the second by $B\delta\,\mathbb{E}[|\mathfrak{M}_{r_q}|]\le B\delta\,\sqrt{NBr_q}$ using $0\le\Delta_q\mathcal{T}^{a_i}\le B\delta$ pathwise (Step 1) and Step 2. Altogether

Constant multiplier term. Expand $(\Delta_q\mathfrak{M})^{4}=\sum_{i_1,i_2,i_3,i_4}\Delta_qM^{a_{i_1}}\Delta_qM^{a_{i_2}}\Delta_qM^{a_{i_3}}\Delta_qM^{a_{i_4}}$. For the $N^{4}-N$ quadruples with indices not all equal, part (f) with $Z=1$ bounds each expectation in absolute value by $C_\star\delta^{2}$. For the $N$ diagonal quadruples, part (d) with $Z=1$ and power $k=4$ gives $\mathbb{E}[(\Delta_qM^{a_i})^{4}]\le\mathbb{E}[\Delta_q\mathcal{T}^{a_i}]+C_\star\delta^{2}$. Hence

Summation. Sum the four contributions over $q\in\{0,\dots,n-1\}$. The quadratic term contributes at most $6(NB)^{2}\,\delta\sum_{q=0}^{n-1}r_q=6(NB)^{2}\,\delta^{2}\,n(n-1)/2\le3(NBt)^{2}$, since $n\delta=t$. Using $\sqrt{r_q}\le\sqrt{t}$, the linear term's leading part contributes at most $4NB\,\delta\cdot n\sqrt{NBt}=4(NBt)^{3/2}$; its remainder parts contribute at most $4N\,C_\star(1+NBT)\,t\,\sqrt{\delta}+4N^{3}C_\star\sqrt{NBT}\,t\,\delta$. The constant term contributes $\sum_q\mathbb{E}[\Delta_q\mathfrak{A}]=\mathbb{E}[\mathfrak{A}_t]\le NBt$ plus at most $(N+N^{4})C_\star\,t\,\delta$. Hence, for every $n$,

and $\varepsilon_n\to0$ as $n\to\infty$ (recall $\delta=t/n$). Since the left-hand side does not depend on $n$, $\mathbb{E}[\mathfrak{M}_t^{4}]\le NBt+3(NBt)^{2}+4(NBt)^{3/2}$. Finally, with $x=NBt\ge0$, $4x^{3/2}=4\sqrt{x\cdot x^{2}}\le2(x+x^{2})$, since $2\sqrt{uv}\le u+v$ for nonnegative $u,v$ (expand $(\sqrt{u}-\sqrt{v})^{2}\ge0$). Therefore

proving (b).

Step 4: part (c). Parts (a) and (b) are now established for every ordered pair of distinct states, so the pair $(\sigma,\gamma)$ fixed at the outset is released; from here on, fix $\gamma\in\{1,\dots,l\}$, and $\sigma$ denotes a summation variable. The derivation of the identity is adapted from Step 2 of the published proof of the martingale decomposition theorem. Work on $\Omega_0$. By part (b) of that theorem, $M^\gamma_t=\Sigma^\gamma_t-\Sigma^\gamma_0-\int_{[0,t]}\mathbf{1}_{\Omega_0}\,b^\gamma(\Sigma_s,\alpha_s)\,ds$, the integral existing at every outcome by its part (a); on $\Omega_0$, where we work, the indicator factor equals $1$. Summing condition 6 of the solution definition over $i$,

For any ordered pair $(\sigma',\gamma')$ of distinct states, the identity $\sum_i\eta^{i,\sigma'}_s\,\beta(\sigma',\gamma',\Sigma_s,\alpha_s)=N\,\Sigma^{\sigma'}_s\,\beta(\sigma',\gamma',\Sigma_s,\alpha_s)$ holds pointwise in $s$ on $\Omega_0$ (derived notation of the solution definition), so by the definition of the consumed clock times in condition 2 and the linearity of the Lebesgue integral,

By the formula of the aggregate state drift and linearity again,

Subtracting the last display from the preceding one and using $\mathfrak{M}^{\sigma\gamma}=\mathcal{N}^{\sigma\gamma}-\mathfrak{A}^{\sigma\gamma}$,

which is the claimed almost sure identity, $\Omega_0$ having probability $1$.

For the moment bound, first note the elementary inequality $(x_1+\cdots+x_n)^{4}\le n^{3}\sum_jx_j^{4}$ for real $x_1,\dots,x_n$: indeed $(x_1+\cdots+x_n)^{2}\le n\sum_jx_j^{2}$ (expand and use $2x_px_q\le x_p^{2}+x_q^{2}$), so $(x_1+\cdots+x_n)^{4}\le n^{2}(\sum_jx_j^{2})^{2}\le n^{2}\cdot n\sum_jx_j^{4}$. Applying it with the $2(l-1)$ summands of the identity and taking expectations (part (b) applies to every ordered pair of distinct states),

Finally, $|\sqrt{N}M_t|^{4}=N^{2}\big(\sum_\gamma(M^\gamma_t)^{2}\big)^{2}\le N^{2}\,l\sum_\gamma(M^\gamma_t)^{4}=l\,N^{-2}\sum_\gamma(N\,M^\gamma_t)^{4}$ by the same square inequality with $l$ summands, so

and since $N\ge1$ this is at most $6\,l^{2}(2(l-1))^{4}\,(Bt+(Bt)^{2})$. This proves (c). $\blacksquare$