Proof of The Control of the Controlled N-Agent Dynamics is Adapted to the Observation Filtration

Fix $t\in[0,T]$ and $j\in\{1,\dots,m\}$. Throughout, $\mathcal{G}_t$ is the $\sigma$-algebra generated by the random variables $\Upsilon^\upsilon_s$ ($0\le s\le t$, $\upsilon\in\{1,\dots,\tilde{l}\}$) together with every event of probability $0$, as in the solution definition.

Step 0 (preliminaries). We use repeatedly: finite sums, differences, products, and scalar multiples of $\mathcal{G}_t$-measurable real-valued functions on $\Omega$ are $\mathcal{G}_t$-measurable, and indicator functions of events of $\mathcal{G}_t$ are $\mathcal{G}_t$-measurable --- the rational-decomposition argument recorded in the stochastic-process definition and the level-set arguments of the Preliminaries of the square-integrability definition use only the generator criterion of the definition of a measurable function, and therefore apply verbatim with $\mathcal{G}_t$ in place of $\mathcal{F}$. Likewise, for countably many $\mathcal{G}_t$-measurable functions $X_1,X_2,\dots$, the pointwise infimum $\inf_nX_n$, when real-valued, is $\mathcal{G}_t$-measurable, since $\{\inf_nX_n<a\}=\bigcup_n\{X_n<a\}$ for every real $a$, and similarly $\sup_nX_n$ via $\{\sup_nX_n>a\}=\bigcup_n\{X_n>a\}$; in particular the maximum and minimum of two $\mathcal{G}_t$-measurable real-valued functions are $\mathcal{G}_t$-measurable, being the supremum and infimum of a two-element family. From the sets $\{X<a\}$ (respectively $\{X>a\}$) for all real $a$ one recovers the preimages of all Borel sets by complements and countable set operations, by the generator criterion.

Step 1 (observation totals). For $s\in[0,t]$ set $K'_s=N\sum_{\upsilon=1}^{\tilde{l}}\Upsilon^\upsilon_s$, a $\mathcal{G}_t$-measurable random variable. At every $\omega\in\Omega_0$ and every $s\in[0,t]$, condition 4 of the solution definition gives $\Upsilon^\upsilon_s=\frac1N\sum_{i=1}^N\tilde{N}^{i,\upsilon}_s$, so $K'_s=\sum_{i,\upsilon}\tilde{N}^{i,\upsilon}_s=\tilde{c}_s$, the observation total. For $k=0,1,2,\dots$ put $G_k=\{K'_t=k\}\in\mathcal{G}_t$; these events are pairwise disjoint, and we write $G_\infty=\Omega\setminus\bigcup_{k\ge0}G_k$.

Step 2 (counting-path facts and jump-time surrogates). Let $c$ be a counting path, let $k\ge1$ be a natural number with $c(t')\ge k$ for some $t'\ge0$, and let $\tau_k(c)$ be its $k$-th jump time. We record:

(i) for every $q\ge0$: $c(q)\ge k$ if and only if $q\ge\tau_k(c)$. Indeed, for every $s>\tau_k(c)$ the greatest-lower-bound property yields $s'\in[\tau_k(c),s)$ with $c(s')\ge k$, so $c(s)\ge k$ by monotonicity (clause 2 of the counting-path definition); then $c(\tau_k(c))\ge k$ by right-continuity (clause 3: $c(\tau_k(c))$ is the greatest lower bound of $\{c(s):s>\tau_k(c)\}$, a set with all members $\ge k$), and $c(q)\ge k$ for $q\ge\tau_k(c)$ by monotonicity. Conversely $c(q)\ge k$ puts $q$ in the set whose infimum is $\tau_k(c)$.

(ii) $\tau_k(c)>0$: otherwise $c(0)\ge k\ge1$ by (i), contradicting $c(0)=0$ (clause 1).

(iii) $c(\tau_k(c))=k$, and $\tau_k(c)$ is a jump time of $c$: every $s<\tau_k(c)$ has $c(s)\le k-1$ by (i) and integrality (clause 1), so $c(\tau_k(c)-)\le k-1$, while $c(\tau_k(c))\ge k$ by (i); the unit-jump bound (clause 4) gives $k\le c(\tau_k(c))\le c(\tau_k(c)-)+1\le k$, so $c(\tau_k(c))=k>c(\tau_k(c)-)$.

(iv) if also $c(t'')\ge k+1$ for some $t''$, then $\tau_k(c)<\tau_{k+1}(c)$: by (iii), $c(\tau_k(c))=k<k+1$, so $\tau_k(c)\ge\tau_{k+1}(c)$ would contradict (i) applied to $k+1$.

(v) for $T'>0$, the jump times of $c$ lying in $(0,T']$ are exactly $\tau_1(c)<\dots<\tau_{c(T')}(c)$: each $\tau_k(c)$ with $k\le c(T')$ lies in $(0,T']$ by (i)--(ii) and is a jump time by (iii). Conversely let $t''\in(0,T']$ be a jump time and put $k=c(t'')$. The set $\{c(s):0\le s<t''\}$ is a nonempty set of integers bounded above, so its least upper bound $c(t''-)$ is a member of the set (there is a member exceeding $c(t''-)-1$; a strictly larger member would be an integer in an interval of length less than $1$ above it, so that member is the least upper bound); in particular $c(t''-)$ is an integer, and clause 4 with $c(t''-)<c(t'')$ gives $c(t''-)=k-1$ and $k\ge1$. Every $s<t''$ then has $c(s)\le k-1<k$ while $c(t'')=k$, so $\tau_k(c)=t''$ by the definition of the infimum. Hence the jump times in $(0,T']$ all occur among $\tau_1(c),\dots,\tau_{c(T')}(c)$, which are strictly increasing by (iv), and the two finite lists coincide.

By condition 3 of the solution definition, $s\mapsto\tilde{c}_s$ coincides on $[0,T]$ with the restriction of a counting path $c$, and condition 5 lists the jump times of the observation total in $[0,T]$ as $\tau_1<\dots<\tau_{K_T}$; by fact (v) with $T'=T$ these are exactly $\tau_1(c)<\dots<\tau_{c(T)}(c)$ (jump times are positive by definition, so lying in $[0,T]$ means lying in $(0,T]$, and $c(T)=\tilde{c}_T=K_T$, so the two lists have the same length), and facts (i)--(iv) apply to them, with $c(q)=\tilde{c}_q$ for $q\in[0,T]$.

Now let $D=(\mathbb{Q}\cap[0,t])\cup\{t\}$, a countable set containing $t$. For $j'\ge1$ define

a well-defined element of $[0,t]$. For $a\le t$, $\{\tau'_{j'}<a\}=\bigcup_{q\in D,\,q<a}\{K'_q\ge j'\}$ (an infimum of a set is smaller than $a$ exactly when some member is), and for $a>t$, $\{\tau'_{j'}<a\}=\Omega$; in all cases an event of $\mathcal{G}_t$, so $\tau'_{j'}$ is $\mathcal{G}_t$-measurable. Set $\tau''_{j'}=\max(\tau'_1,\dots,\tau'_{j'})$, again $\mathcal{G}_t$-measurable, with $0\le\tau''_1\le\tau''_2\le\dots\le t$ at every $\omega$, so that for every $k\ge1$ the tuple $(\tau''_1,\dots,\tau''_k)$ lies in the record space $R_k(T)$ of the policy definition.

Claim: at every $\omega\in\Omega_0$ with $\tilde{c}_t(\omega)\ge j'$, $\tau'_{j'}(\omega)=\tau_{j'}(\omega)$. Indeed, by Step 1 and fact (i), for $q\in D$ one has $K'_q=\tilde{c}_q\ge j'$ exactly when $q\ge\tau_{j'}$, so the set in the infimum is $D\cap[\tau_{j'},t]$, which contains $t$. If $\tau_{j'}<t$, its infimum is $\tau_{j'}$: no member is smaller, and for every $q'>\tau_{j'}$ there are rational numbers in $(\tau_{j'},\min(q',t))$ by the density of the rationals in the real numbers; if $\tau_{j'}=t$, the set is $\{t\}$ with infimum $t$. Since the jump times increase in $j'$ (fact (iv)), also $\tau''_{j'}(\omega)=\tau_{j'}(\omega)$ for all $j'\le\tilde{c}_t(\omega)$ at every $\omega\in\Omega_0$.

Step 3 (channel surrogates). Fix $j'\ge1$; the construction below is carried out for every $j'$, and we suppress the index $j'$ on $d_n$, $e_n$, $U^\upsilon_n$, $L^\upsilon_n$, $V^\upsilon$, $W^\upsilon$, retaining it on the resulting $\upsilon'_{j'}$. For $n\in\mathbb{N}$ let $Q_n=\{i\,t\,2^{-n}:i=0,1,\dots,2^n\}\subseteq[0,t]$, a finite set containing $0$ and $t$. Define $d_n=\min\{q\in Q_n:q\ge\tau'_{j'}\}$ and $e_n=\max(\{q\in Q_n:q<\tau'_{j'}\}\cup\{0\})$; both are $\mathcal{G}_t$-measurable with finitely many values: each event $\{d_n=q\}$ and $\{e_n=q\}$ is a countable Boolean combination of events $\{\tau'_{j'}<a\}$ with $a$ real, all in $\mathcal{G}_t$ by Step 2, using $\{\tau'_{j'}\le q'\}=\bigcap_{n'\ge1}\{\tau'_{j'}<q'+1/n'\}$. For $\upsilon\in\{1,\dots,\tilde{l}\}$ set

$\mathcal{G}_t$-measurable random variables; since the events $\{d_n=q\}$, $q\in Q_n$, partition $\Omega$, and likewise the events $\{e_n=q\}$, one has $U^\upsilon_n=\Upsilon^\upsilon_{d_n}$ and $L^\upsilon_n=\Upsilon^\upsilon_{e_n}$ pointwise. Set

Each truncated term lies between $0$ and $\max(\Upsilon^\upsilon_t,0)$ at every $\omega$, so $V^\upsilon$ and $W^\upsilon$ are real-valued everywhere and $\mathcal{G}_t$-measurable by Step 0.

At every $\omega\in\Omega_0$: by condition 3 of the solution definition, each $s\mapsto\tilde{N}^{i,\upsilon}_s$ coincides on $[0,T]$ with the restriction of a counting path, so $s\mapsto\Upsilon^\upsilon_s=\frac1N\sum_i\tilde{N}^{i,\upsilon}_s$ is nondecreasing and nonnegative on $[0,T]$. For finitely many nondecreasing paths, at every time the greatest lower bound over strictly later times of the sum is the sum of the greatest lower bounds, and the least upper bound over strictly earlier times of the sum is the sum of the least upper bounds: given $\varepsilon''>0$, a single later (respectively earlier) time can be chosen that works for all summands at once, by monotonicity. Hence right-continuity (clause 3 of the counting-path definition) passes to $\Upsilon^\upsilon$ at every $s\in[0,T)$ --- and since the paths are nondecreasing, the greatest lower bound over all $s'>s$ is already attained over $s'\in(s,T]$, so $\Upsilon^\upsilon_s=\inf\{\Upsilon^\upsilon_{s'}:s<s'\le T\}$ --- the left limit $\Upsilon^\upsilon_{s-}=\sup\{\Upsilon^\upsilon_{s'}:0\le s'<s\}$ exists for $s\in(0,T]$, and

which is positive exactly when some observation counter with channel $\upsilon$ jumps at $s$, every summand being nonnegative. Moreover $0\le U^\upsilon_n\le\Upsilon^\upsilon_t$ and $0\le L^\upsilon_n\le\Upsilon^\upsilon_t$ on $\Omega_0$ (monotonicity, since $d_n,e_n\in[0,t]$), so the truncations are inactive there: $V^\upsilon=\inf_n\Upsilon^\upsilon_{d_n}$ and $W^\upsilon=\sup_n\Upsilon^\upsilon_{e_n}$ at every $\omega\in\Omega_0$.

If moreover $\tilde{c}_t(\omega)\ge j'$: then $\tau'_{j'}=\tau_{j'}$ (Step 2), and $d_n\ge\tau_{j'}$ with $d_n-\tau_{j'}\le t2^{-n}$, so every term satisfies $\Upsilon^\upsilon_{d_n}\ge\Upsilon^\upsilon_{\tau_{j'}}$ by monotonicity, and $\inf_n\Upsilon^\upsilon_{d_n}=\Upsilon^\upsilon_{\tau_{j'}}$: either $d_n=\tau_{j'}$ for some $n$, or $d_n>\tau_{j'}$ for all $n$, in which case $d_n\le t$ forces $\tau_{j'}<t\le T$, right-continuity applies at $\tau_{j'}\in[0,T)$, and for every $s>\tau_{j'}$ one has $d_n\le s$ for all large $n$, so the infimum equals the displayed greatest lower bound $\Upsilon^\upsilon_{\tau_{j'}}$. And $\tau_{j'}>0$ by fact (ii) of Step 2, so for $n$ with $t2^{-n}<\tau_{j'}$ one has $e_n<\tau_{j'}$ and $\tau_{j'}-e_n\le t2^{-n}$, whence $\sup_n\Upsilon^\upsilon_{e_n}=\sup\{\Upsilon^\upsilon_s:0\le s<\tau_{j'}\}=\Upsilon^\upsilon_{\tau_{j'}-}$ by monotonicity.

Define $\upsilon'_{j'}$ as the least $\upsilon\in\{1,\dots,\tilde{l}\}$ with $V^\upsilon>W^\upsilon$, and $\upsilon'_{j'}=1$ if there is no such $\upsilon$; each event $\{\upsilon'_{j'}=\upsilon\}$ is a finite Boolean combination of the events $\{V^{\upsilon''}>W^{\upsilon''}\}\in\mathcal{G}_t$. At every $\omega\in\Omega_0$ with $\tilde{c}_t(\omega)\ge j'$: by condition 5 of the solution definition exactly one observation counter jumps at $\tau_{j'}$, and its channel is $\upsilon_{j'}$; by the displayed jump identity, $\Upsilon^\upsilon_{\tau_{j'}}-\Upsilon^\upsilon_{\tau_{j'}-}>0$ exactly when $\upsilon=\upsilon_{j'}$; hence $V^\upsilon>W^\upsilon$ exactly for $\upsilon=\upsilon_{j'}$, and $\upsilon'_{j'}=\upsilon_{j'}$.

Step 4 (assembly). Fix $k\ge1$ and a mark vector $v\in\{1,\dots,\tilde{l}\}^k$, and write $\{\upsilon'=v\}$ for the event $\{\upsilon'_1=v_1\}\cap\dots\cap\{\upsilon'_k=v_k\}$. The policy definition makes $(t',\tau)\mapsto h^j_k(t',\tau,v)$ measurable for the $\sigma$-algebra generated by the relatively open subsets of $[0,T]\times R_k(T)$. The insertion map $\iota:R_k(T)\to[0,T]\times R_k(T)$, $\iota(\tau)=(t,\tau)$ with the present $t$ fixed, pulls relatively open sets back to relatively open sets: if $U$ is an open subset of $\mathbb{R}^{1+k}$ and $\tau\in R_k(T)$ with $(t,\tau)\in U$, there is an open box around $(t,\tau)$ inside $U$, whose slice at first coordinate $t$ is an open box around $\tau$ whose intersection with $R_k(T)$ is contained in the preimage; so the preimage of $U\cap([0,T]\times R_k(T))$ is relatively open in $R_k(T)$. Since the collection of subsets of $[0,T]\times R_k(T)$ whose $\iota$-preimage lies in the $\sigma$-algebra generated by the relatively open subsets of $R_k(T)$ is a $\sigma$-algebra containing the relatively open sets, the slice $g^\circ_{k,v}:\tau\mapsto h^j_k(t,\tau,v)$, which is $h^j_k(\cdot,\cdot,v)\circ\iota$, is measurable for the $\sigma$-algebra generated by the relatively open subsets of $R_k(T)$. Next, the map $\omega\mapsto\tau''(\omega)=(\tau''_1(\omega),\dots,\tau''_k(\omega))\in R_k(T)$ pulls relatively open sets back to $\mathcal{G}_t$: every open subset of $\mathbb{R}^k$ is the union of the countably many open boxes with rational vertices that it contains (around every point of an open set there is such a box, by the definition of openness via balls and the density of the rationals), and the preimage of an open box is a finite intersection of events $\{a<\tau''_{j'}<b\}\in\mathcal{G}_t$, while the preimage of $U\cap R_k(T)$ equals the preimage of $U$ because the map takes values in $R_k(T)$. Since the collection of subsets of $R_k(T)$ whose preimage under $\omega\mapsto\tau''(\omega)$ lies in $\mathcal{G}_t$ is a $\sigma$-algebra containing the relatively open sets, the composite $g_{k,v}:\omega\mapsto g^\circ_{k,v}(\tau''(\omega))=h^j_k\big(t,(\tau''_1(\omega),\dots,\tau''_k(\omega)),v\big)$ is $\mathcal{G}_t$-measurable.

Define

where at each $\omega$ at most one summand is nonzero (the events $G_k$ are pairwise disjoint and, for fixed $k$, the mark events $\{\upsilon'=v\}$ over $v$ partition $\Omega$), and the value is $0$ on $G_\infty$. This is a well-defined real-valued function, and it is $\mathcal{G}_t$-measurable: for every Borel set $A$,

a countable union of events of $\mathcal{G}_t$ (the conditions $h^j_0(t)\in A$ and $0\in A$ select $\emptyset$ or the displayed event).

Step 5 (identity on the regular event). Let $\omega\in\Omega_0$ and put $k=\tilde{c}_t(\omega)=K'_t(\omega)$ (Step 1), so $\omega\in G_k$. If $k=0$, condition 5 of the solution definition gives $\alpha^j_t(\omega)=h^j_0(t)=\alpha'^{\,j}_t(\omega)$. If $k\ge1$, then for $1\le j'\le k$ Steps 2--3 give $\tau''_{j'}(\omega)=\tau_{j'}(\omega)$ and $\upsilon'_{j'}(\omega)=\upsilon_{j'}(\omega)$, so with $v=(\upsilon_1(\omega),\dots,\upsilon_k(\omega))$ the active summand is $g_{k,v}(\omega)=h^j_k\big(t,(\tau_1(\omega),\dots,\tau_k(\omega)),v\big)=\alpha^j_t(\omega)$ by condition 5. This proves conclusion 1, the almost-sure equality following from $P(\Omega_0)=1$.

Step 6 (square-integrability transfer). Let $X$ be square-integrable, let $Y$ be a random variable with $P(X=Y)=1$, and let $A'$ be the complement of $\{X=Y\}$, an event of probability $0$. The nonnegative random variable $Y^2\mathbf{1}_{A'}$ vanishes off $A'$; every nonnegative simple function bounded above by it vanishes off $A'$ and hence has integral $0$, so its expectation satisfies $\mathbb{E}[Y^2\mathbf{1}_{A'}]=0$ by the definition of the integral of a nonnegative random variable. Pointwise $Y^2\le X^2+Y^2\mathbf{1}_{A'}$ (off $A'$ the two agree; on $A'$ the second summand dominates), so by the additivity and monotonicity of the integral on nonnegative random variables in the linearity and monotonicity theorem, $\mathbb{E}[Y^2]\le\mathbb{E}[X^2]<\infty$, and $Y$ is square-integrable; exchanging the roles of $X$ and $Y$ gives $\mathbb{E}[X^2]\le\mathbb{E}[Y^2]$, so the mean-square norms agree. This proves conclusion 2.

Step 7 (fluctuation control). With $(S,A)$ and $\mathfrak{a}_t$ as in conclusion 3, $A^j_t$ is a real number, and $\sqrt{N}(\alpha'^{\,j}_t-A^j_t)$ is $\mathcal{G}_t$-measurable by the closure of $\mathcal{G}_t$-measurability under scalar multiples and addition of constants (Step 0); at every $\omega\in\Omega_0$ it equals $\sqrt{N}(\alpha^j_t(\omega)-A^j_t)=\mathfrak{a}^j_t(\omega)$ by conclusion 1 and the definition of the control fluctuation process. The square-integrability transfer is conclusion 2 applied to $X=\mathfrak{a}^j_t$. $\square$