Proof of Measurable Reconstruction of the Controlled N-Agent Dynamics from Observation Records

Throughout, a transition clock index is a pair $c=(i,\sigma\gamma)$ with $\sigma\neq\gamma$, and the transition clock indices are listed once and for all as a finite sequence. For a counting path $y$ and $m\ge1$, $\tau_m(y)$ denotes the $m$-th jump time; since $y$ is right-continuous with values in the natural numbers and $0$, one has $\{\tau_m(Y^c)\le s\}=\{Y^c_s\ge m\}$ for every $s\ge0$, so each $\tau_m(Y^c)$ is a $\mathcal{T}$-measurable $[0,\infty]$-valued map. The jump times of a counting path are strictly increasing where finite, by the unit-jump property. We also record that the map $(r,s)\mapsto a^r(s)$ is measurable with respect to $\mathcal{R}\otimes\mathcal{B}_{[0,T]}$: on the cell $C_{k,v}$ and the region $\{(r,s):k_r(s)=j\}$ (a member of $\mathcal{R}\otimes\mathcal{B}_{[0,T]}$, being described by the coordinate conditions $t_j\le s$ and, when $j<k$, $s<t_{j+1}$), $a^r(s)$ is the composition of the measurable map $(r,s)\mapsto(s,(t_1,\dots,t_j))$ into $[0,T]\times R_j(T)$ with $h_j(\cdot,\cdot,(v_1,\dots,v_j))$, which is measurable for the trace $\sigma$-algebras as in The Record-Frozen Control Path and Record-Frozen Policy. Moreover $a^r$ is $\mathcal{A}$-valued: by The Record-Frozen Control Path and Record-Frozen Policy, $a^r(s)=h_{k_r(s)}\bigl(s,(t_1,\dots,t_{k_r(s)}),(v_1,\dots,v_{k_r(s)})\bigr)$ with $(t_1,\dots,t_{k_r(s)})\in R_{k_r(s)}(T)$, so $a^r(s)\in\mathcal{A}$ because $h$ is $\mathcal{A}$-valued; in particular $\beta(\sigma,\gamma,\cdot,a^r(s))$ is defined, and the record-frozen policy $\hat{h}^r$, each of whose members takes the value $a^r(s)$, is $\mathcal{A}$-valued as well.

Part 1: construction. Fix $(r,\omega)\in\mathbf{R}\times\Omega$ with $r=(k,t,v)$. We define stage data: times $S_0\le S_1\le\dots$, states $x^i_p\in\{1,\dots,l\}$, consumed counts $m^c_p$, and consumed levels $L^c_p\ge0$, starting from $S_0=0$, $x^i_0=\varsigma^i_0(\omega)$, $m^c_0=0$, $L^c_0=0$. Given stage $p$ with $S_p\le T$: let $\Sigma_p\in\Delta^l$ have components $\frac{1}{N}\sum_i\mathbf{1}\{x^i_p=\gamma\}$, and for each transition clock index $c=(i,\sigma\gamma)$ define the tentative rate on $[S_p,T]$ by $a^c_p(s)=\mathbf{1}\{x^i_p=\sigma\}\,\beta(\sigma,\gamma,\Sigma_p,a^r(s))$, a $[0,B]$-valued measurable function of $s$ (the map $s\mapsto(\Sigma_p,a^r(s))$ into $\Delta^l\times\mathcal{A}$ has measurable components by the preliminary remark, and $\beta(\sigma,\gamma,\cdot,\cdot)$ is sequentially continuous on $\Delta^l\times\mathcal{A}$ by condition 2 of Transition-Rate Family, so Sequentially Continuous Functions of Measurable Euclidean Maps are Measurable applies with $E=\Delta^l\times\mathcal{A}$), and the tentative consumed level $\hat{A}^c_p(s)=L^c_p+\int_{[S_p,s]}a^c_p\,du$ for $s\in[S_p,T]$, a continuous nondecreasing function by claim 1 of Cumulative-Rate Time Change: Regularity, Substitution, and Crossing Times. Let $\theta^c_p=\tau_{m^c_p+1}(Y^c(\omega))$ and let $\kappa^c_p$ be the least $s\in[S_p,T]$ with $\hat{A}^c_p(s)\ge\theta^c_p$ if such $s$ exists (the set is closed by continuity, so it has a least member), and $\kappa^c_p=+\infty$ otherwise. If every $\kappa^c_p=+\infty$, the recursion terminates with $P=p$. Otherwise put $S_{p+1}=\min_c\kappa^c_p$, let $c_{p+1}=(i^*,\sigma^*\gamma^*)$ be the first minimizer in the fixed listing, and set: $x^{i^*}_{p+1}=\gamma^*$ and $x^i_{p+1}=x^i_p$ for $i\neq i^*$; $m^{c_{p+1}}_{p+1}=m^{c_{p+1}}_p+1$ and $m^c_{p+1}=m^c_p$ otherwise; $L^c_{p+1}=\hat{A}^c_p(S_{p+1})$ for every $c$.

The recursion terminates: every stage increments some $m^c$, and $m^c_p\le Y^c_{BT}(\omega)$ throughout, since a crossing forces $\tau_{m^c_p+1}(Y^c)\le\hat{A}^c_p(S_{p+1})\le BT$ (the levels telescope: $L^c_{p+1}=\hat{A}^c_p(S_{p+1})$ is the integral over $[0,S_{p+1}]$ of the stagewise rates, all bounded by $B$), whence $Y^c_{BT}\ge m^c_p+1$; and $Y^c_{BT}(\omega)$ is a natural number or $0$. So $P$ is finite, bounded by the sum of the $Y^c_{BT}(\omega)$.

Define $p(s)=\max\{p\le P: S_p\le s\}$ for $s\in[0,T]$, and set $\eta^{r,i,\gamma}_s(\omega)=\mathbf{1}\{x^i_{p(s)}=\gamma\}$, $\sigma^{r,i}_s(\omega)=x^i_{p(s)}$, and $\tilde{A}^{r,i,\upsilon}_s(\omega)=\int_{[0,s]}\tilde{\beta}\bigl(\sigma^{r,i}_u(\omega),\upsilon,\Sigma^r_u(\omega)\bigr)\,du,$ which lies in $[0,\tilde{B}T]$. By construction, for every $(r,\omega)$: exactly one $\gamma$ has $\eta^{r,i,\gamma}_s=1$; the paths $s\mapsto\sigma^{r,i}_s$ are piecewise constant and right-continuous with at most $P$ changes, starting at $\varsigma^i_0(\omega)$; and the displayed identity holds. Taking $G=\mathbf{R}\times\Omega$ (so that $\Omega_G=\Omega$ serves in the statement), which lies in $\mathcal{R}\otimes\mathcal{T}$ with all sections of probability one, this establishes claim (c), and the sections required in claims (d) and (f) are trivially contained in $G$.

Part 2: measurability (claims (a) and (b)). By induction on $p$, the maps $(r,\omega)\mapsto S_p$ (with value $+\infty$ for $p>P$), $x^i_p$, $m^c_p$, $L^c_p$ are $\mathcal{R}\otimes\mathcal{T}$-measurable. For the inductive step: the map $((r,\omega),s)\mapsto a^c_p(s)$ is measurable (finitely many cases in the stage states, each a composition as in Part 1, jointly in $(r,s)$ by the preliminary remark); hence for fixed $s$ the map $(r,\omega)\mapsto\hat{A}^c_p(s)$ is measurable, by the Tonelli theorem applied to the bounded jointly measurable integrand, and $\hat{A}^c_p$ is jointly measurable in $((r,\omega),s)$, being the pointwise nonincreasing limit in $n$ of the maps $((r,\omega),s)\mapsto\hat{A}^c_p(d_n(s))$ with $d_n(s)$ the least point of the grid $\{jT2^{-n}\}$ that is $\ge s$ (each grid map is measurable, being constant in $s$ on each grid interval; the limit is an infimum over $n$ by monotonicity of $\hat{A}^c_p$, hence measurable). Continuity of $\hat{A}^c_p$ gives $\{(r,\omega):\kappa^c_p\le s\}=\{\hat{A}^c_p(s)\ge\theta^c_p\}\cap\{S_p\le s\}$, so $\kappa^c_p$ is measurable; so are $S_{p+1}$ (a finite minimum), the identity of the first minimizer (finitely many comparisons), and the updated stage data.

Claim (a): $\eta^{r,i,\gamma}_s=\sum_{p}\mathbf{1}\{S_p\le s<S_{p+1}\}\,\mathbf{1}\{x^i_p=\gamma\}$ (with $S_{P+1}=+\infty$), and for a measurable $[0,\infty]$-valued map $g$ on $\mathbf{R}\times\Omega$ the set $\{((r,\omega),s):g(r,\omega)\le s\}$ is measurable, being the intersection over $n$ of the unions over rationals $q$ of $\{g<q\}\times\{s>q-1/n\}$; likewise for strict inequalities. Hence $\eta$, $\sigma^{r,i}$, and (by the same discretization as above) $\tilde{A}^{r,i,\upsilon}$ are measurable in $((r,\omega),s)$, which is claim (a) after the elementwise identification of triples with nested pairs.

Claim (b): on the union $U_j$ of the cells with at least $j$ events (a countable union of cells, hence in $\mathcal{R}$), the coordinate $t_j:U_j\to(0,T]$ is $\mathcal{R}$-measurable (its cellwise preimages correspond under the transport to coordinate preimages in the restricted $\mathcal{B}_k$). The pairing $(r,\omega)\mapsto((r,t_j(r)),\omega)$ is measurable into the product of claim (a) (preimages of rectangles are $\{r\in C\}\cap\{t_j\in B\}$ intersected with $\omega$-sets), so $(r,\omega)\mapsto\tilde{A}^{r,i,\upsilon}_{t_j(r)}(\omega)$ is measurable. For the left limits, the maps $(r,\omega)\mapsto\eta^{r,i,\gamma}_{(t_j(r)-1/n)\vee0}(\omega)$ are measurable for each $n$; the paths being piecewise constant, the limit as $n\to\infty$ exists and equals $\eta^{r,i,\gamma}_{t_j(r)-}$, and the set where the limit equals $1$ is $\bigcup_m\bigcap_{n\ge m}\{\eta^{r,i,\gamma}_{(t_j(r)-1/n)\vee0}=1\}$, which is measurable.

Part 3: pathwise identification. Fix $\omega$, a record $r$, and suppose given families of paths on $[0,T]$ — occupation indicators $\bar\eta^{i,\gamma}$, states $\bar\sigma^i$, counters $\bar{N}^c=Y^c(\bar{A}^c)$ with $\bar{A}^c$ the consumed times formed from the paths as in condition 2 of Solution of the Controlled N-Agent Dynamics — satisfying, at this $\omega$, conditions 1 and 6 of Solution of the Controlled N-Agent Dynamics with control path $a^r$, the counting-structure requirement of condition 3 for the transition counters and their total (for a genuine solution this follows from condition 3: the grand total has unit jumps and the observation total is nondecreasing, so the transition total also coincides with the restriction of a counting path), and $\bar\sigma^i_0=\varsigma^i_0(\omega)$. Then these paths coincide on $[0,T]$ with the Part 1 construction at $(r,\omega)$. Proof by induction over the construction stages: suppose the paths agree with the construction on $[0,S_p]$, that each $\bar{A}^c(S_p)=L^c_p$, and that the number of jumps of $Y^c$ consumed by time $S_p$ is $m^c_p$. On $(S_p,\min(S_{p+1},T)]$, no transition counter jumps before $S_{p+1}$: a jump of $\bar{N}^c$ at time $s$ requires $\bar{A}^c$ to reach $\tau_{m^c_p+1}(Y^c)$ at $s$, because $\bar{A}^c$ is continuous (bounded integrand) and nondecreasing, and $Y^c\circ\bar{A}^c$ increases exactly when $\bar{A}^c$ passes a jump time of $Y^c$; let $s^*$ be the first time in $(S_p,T]$ at which some transition counter jumps (attained, the counters having finitely many jumps; $s^*=+\infty$ if none). By condition 6 the states are constant on $[S_p,s^*)$, so the consumed-time integrands agree there with the tentative rates $a^c_p$, whence $\bar{A}^c=\hat{A}^c_p$ on $[S_p,s^*)$ and, by continuity, at $s^*$; a jump at $s^*$ means $\hat{A}^c_p$ reaches the threshold $\tau_{m^c_p+1}(Y^c)$ at $s^*$, and conversely the first crossing produces a jump, so $s^*=S_{p+1}$, matching the construction. Exactly one counter jumps at $S_{p+1}$: if two clocks crossed at $S_{p+1}$, the transition total would jump by two there, which the counting-structure requirement of condition 3 excludes; so the crossing clock is unique, the min-index rule selects it, and condition 6 forces the state update of the construction. Under these hypotheses no two clocks cross at the same time and no threshold is already attained at a stage time, so $S_{p+1}>S_p$ at every stage and the intervals above are nonempty. The induction starts at $S_0=0$ and, after the final stage, the same argument shows that no further jumps occur, so the paths agree on $[0,T]$.

Part 4: claim (d). Fix $r$. The record-frozen policy $\hat{h}^r$ is $\mathcal{A}$-valued by the preliminary remark, so part (ii) of Existence, Uniqueness, and Regularity for the Controlled N-Agent Dynamics supplies a solution of the controlled $N$-agent dynamics on $[0,T]$ for $\hat{h}^r$; let $\Omega^r$ be its regular event, $P(\Omega^r)=1$. By condition 5 for $\hat{h}^r$, its control satisfies $\alpha_s=a^r(s)$ for all $s$ at every $\omega\in\Omega^r$. At every $\omega\in\Omega^r$ the solution's paths satisfy the hypotheses of Part 3, so they coincide with the Part 1 construction at $(r,\omega)$; consequently its occupation indicators equal $\eta^{r,i,\gamma}$, its consumed observation clock times equal $\tilde{A}^{r,i,\upsilon}$ (the same integrals of identical paths), and its observation processes equal $\Upsilon^{r,\upsilon}_s=\frac{1}{N}\sum_i\tilde{Y}^{i,\upsilon}_{\tilde{A}^{r,i,\upsilon}_s}$, on $\Omega^r$. Therefore the processes named in claim (d), together with the regular event $\Omega^r$, satisfy conditions 1 and 3–6 of Solution of the Controlled N-Agent Dynamics at every $\omega\in\Omega^r$ (they there take the same values as the solution's processes), and the measurability requirements of condition 2 hold for them by Part 2 (sections of jointly measurable maps, multiplied by the indicator of $\Omega^r$). Each $\Upsilon^{r,\upsilon}_s=\frac1N\sum_i\tilde{Y}^{i,\upsilon}_{\tilde{A}^{r,i,\upsilon}_s}$ is a random variable on $(\Omega,\mathcal{F},P)$: fix $i$, $\upsilon$, $s$ and a natural number $j$, and write $\tau_j(\omega)$ for the $j$-th jump time of the counting path $u\mapsto\tilde{Y}^{i,\upsilon}_u(\omega)$. For real $t\ge0$ one has $\{\tau_j\le t\}=\{\tilde{Y}^{i,\upsilon}_t\ge j\}$ (if $\tilde{Y}^{i,\upsilon}_t\ge j$ then $t$ belongs to the set whose greatest lower bound is $\tau_j$; if $\tau_j\le t$ then $\tilde{Y}^{i,\upsilon}_u\ge j$ for all $u>\tau_j$ by monotonicity, hence $\tilde{Y}^{i,\upsilon}_{\tau_j}\ge j$ by right-continuity and $\tilde{Y}^{i,\upsilon}_t\ge j$), so $\{\tau_j\le t\}$ is an event, and $\tau_j>0$ since the path vanishes at $0$. Hence, with $q$ ranging over the positive rational numbers and $n$ over the natural numbers with $q-1/n\ge0$,

$$\bigl\{\tilde{Y}^{i,\upsilon}_{\tilde{A}^{r,i,\upsilon}_s}\ge j\bigr\}=\{\tau_j\le\tilde{A}^{r,i,\upsilon}_s\}=\bigcap_{q}\Bigl(\{\tilde{A}^{r,i,\upsilon}_s\ge q\}\cup\bigcup_{n}\{\tilde{Y}^{i,\upsilon}_{q-1/n}\ge j\}\Bigr)$$

is an event, $\tilde{A}^{r,i,\upsilon}_s$ being a random variable by Part 2 (the second equality: if $\tau_j\le\tilde{A}^{r,i,\upsilon}_s$ then for every $q$ either $\tilde{A}^{r,i,\upsilon}_s\ge q$ or $\tau_j<q$, the latter meaning $\tau_j\le q-1/n$ for some $n$; conversely, if $\tau_j>\tilde{A}^{r,i,\upsilon}_s$, a rational $q$ strictly between them (density of the rationals) violates the condition). As $\tilde{Y}^{i,\upsilon}_{\tilde{A}^{r,i,\upsilon}_s}$ takes values in $\{0\}\cup\mathbb{N}$, it is a random variable, and so is the average $\Upsilon^{r,\upsilon}_s$. Thus the families of claim (d) are families of random variables satisfying conditions 1--4, and clause (vii) of Existence, Uniqueness, and Regularity for the Controlled N-Agent Dynamics gives that their consumed clock times are defined on all of $\Omega$ and bounded ((vii)(b)), that their transition and observation counters are random variables ((vii)(c)), and that the channel $\upsilon_j$ used in condition 5 is well defined ((vii)(d)). This is claim (d).

Part 5: claim (e). Let $r,r'$ and $s$ be as in the claim. By induction over stages: the recursions at $(r,\omega)$ and $(r',\omega)$ produce identical stage data for every stage with time $\le s$, and identical tentative data on $[S_p,\min(S_{p+1},s)]$: the only dependence on the record is through $a^r(u)$ for $u$ in the relevant interval, and for $u\le s$ one has $k_r(u)=k_{r'}(u)$ with identical entries, so $a^r(u)=a^{r'}(u)$; whether a crossing occurs at a time $\le s$, and where, is determined by the tentative data on $[0,s]$. Hence $\eta^{r,i,\gamma}_u(\omega)=\eta^{r',i,\gamma}_u(\omega)$ for $u\in[0,s]$, and the $\tilde{A}$-integrands agree on $[0,s]$, so $\tilde{A}^{r,i,\upsilon}_u(\omega)=\tilde{A}^{r',i,\upsilon}_u(\omega)$ for $u\le s$. (This holds for every $\omega$; $G=\mathbf{R}\times\Omega$.)

Part 6: claim (f). Measurability of $W$. By claim 4(b) of Assembly of Measure Spaces: Restriction, Transport, One-Point Spaces, and Countable Disjoint Unions it suffices that $W^{-1}(A)\in\mathcal{G}_T$ for $A$ in the $\sigma$-algebra of each cell. The event $\Omega_0$ and its complement lie in $\mathcal{G}_T$, since $P(\Omega_0)=1$ and $\mathcal{G}_T$ contains every event of probability zero by its definition in Solution of the Controlled N-Agent Dynamics. For $A\subseteq C_{k,v}$ corresponding under the transport to the Borel set $B\subseteq D_k(T)$: $W^{-1}(A)=\Omega_0\cap\{K_T=k\}\cap\{\upsilon_1=v_1,\dots,\upsilon_k=v_k\}\cap\{(\tau_1,\dots,\tau_k)\in B\},$ which lies in $\mathcal{G}_T$ by part (iv) of Existence, Uniqueness, and Regularity for the Controlled N-Agent Dynamics; for $A\ni r_\emptyset$ one adds the $\mathcal{G}_T$-events $\Omega_0\cap\{K_T=0\}$ and $\Omega\setminus\Omega_0$. Consistency. Let $\omega\in\Omega_0$ and put $r=W(\omega)$. By condition 5, the solution's control path at $\omega$ is $u\mapsto h_{K_u}(u,\tau_1,\dots,\tau_{K_u},\upsilon_1,\dots,\upsilon_{K_u})$, and since the entries of $r$ with times $\le u$ are exactly the events counted by $K_u$, this equals $a^{r}(u)$ for every $u\in[0,T]$. The solution's paths at $\omega$ therefore satisfy the hypotheses of Part 3 with the record $r$, so they coincide with the construction at $(W(\omega),\omega)$, and the consumed observation clock times coincide with $\tilde{A}^{W(\omega),i,\upsilon}$. As $(W(\omega),\omega)\in G$ trivially, these properties hold at every $\omega\in\Omega_0$, an event of probability one, as claim (f) requires.