Proof of Existence, Uniqueness, and Regularity for the Controlled N-Agent Dynamics

Throughout, $\beta$, $\tilde{\beta}$, $T$, and $h$ are as in the statement, and clock labels $a$ range over the triples $(i,\sigma\gamma)$ with $\sigma\neq\gamma$ and the pairs $(i,\upsilon)$. Set $\bar{\beta}=\max(B,\tilde{B})$. Given a state vector $x\in\{1,\dots,l\}^N$, write $\Sigma(x)\in\Delta^l$ for its empirical vector, and define the configuration rates $\lambda^{(i,\sigma\gamma)}(x,\alpha)=\mathbf{1}_{\{x_i=\sigma\}}\beta(\sigma,\gamma,\Sigma(x),\alpha)$ and $\lambda^{(i,\upsilon)}(x,\alpha)=\tilde{\beta}(x_i,\upsilon,\Sigma(x))$, all bounded by $\bar{\beta}$. We use freely two facts about counting paths $c$. (F1): $\tau_k(c)\le u$ if and only if $c(u)\ge k$; indeed, if $s>\tau_k(c)$ there is $t\in[\tau_k(c),s]$ with $c(t)\ge k$, so $c(s)\ge k$ by monotonicity, and then right-continuity gives $c(\tau_k(c))\ge k$, while conversely $c(u)\ge k$ puts $u$ in the defining set. (F2): the finite jump times are strictly increasing; if $\tau_k(c)=\tau_{k+1}(c)=t<\infty$ then $c(t)\ge k+1$ while $c(s)\le k-1$ for all $s<t$ by (F1), contradicting unit jumps. (Compare part (a) of Jump Times of the Homogeneous Poisson Process: Finiteness and Exponential Interarrival Law.)

Part A: proof of (i). Let $M$ be a homogeneous Poisson process with rate $1$ on some probability space, which exists by Existence of the Inhomogeneous Poisson Process. Every increment $M_{q'}-M_q$ ($q<q'$ rational) has the Poisson distribution, hence is almost surely a nonnegative integer; intersecting over all rational pairs gives an almost sure event $\Omega'$ on which $q\mapsto M_q$ is nondecreasing over the rationals with nonnegative integer values ($M_0=0$ by the definition). On $\Omega'$ define $Y_u=\inf\{M_q: q\ \text{rational},\ q>u\}$; off $\Omega'$ set $Y\equiv0$. Then every path of $Y$ is nonnegative-integer valued, nondecreasing, and right-continuous ($\inf_{s>u}Y_s=\inf_{q>u}M_q=Y_u$), with $Y_0=0$. For fixed $u$ and rationals $q_n\downarrow u$: on $\Omega'$, $M_{q_n}$ is nonincreasing, and almost surely each $M_{q_n}-M_u$ is a nonnegative integer, so $M_{q_n}-M_u$ is eventually equal to some integer $c\ge0$ with $P(c\ge1)\le\inf_nP(M_{q_n}-M_u\ge1)=\inf_n(1-e^{-(q_n-u)})=0$; hence $Y_u=M_u$ almost surely, so $Y$ is a modification of $M$ and therefore again a homogeneous Poisson process with rate $1$ (its finite families of increments have the same joint distributions, since modification preserves them). Unit jumps: fix a natural $M_0'$ and rational $\delta\in(0,1]$; if some path of $Y$ has a jump of size $\ge2$ at $u\in(0,M_0']$, then the rational cell $(t_{k},t_{k+1}]$ of size $\delta$ containing $u$ satisfies $Y_{t_{k+1}}-Y_{t_k}\ge2$. For Poisson parameter $\mu$, $P(K\ge2)=1-e^{-\mu}(1+\mu)\le1-(1-\mu)(1+\mu)=\mu^2$, using $e^{-\mu}\ge1-\mu$ (for $\mu\ge1$ trivially; for $\mu<1$ from the geometric series $\frac{1}{1-\mu}=\sum_j\mu^j\ge\sum_j\mu^j/j!=e^{\mu}$, by the series form of the exponential and its properties). Hence $P(\text{a jump of size}\ge2\ \text{in}\ (0,M_0'])\le(M_0'/\delta)\delta^2=M_0'\delta$ for every rational $\delta$, so this probability is $0$; intersecting over $M_0'\in\mathbb{N}$ and redefining $Y\equiv0$ on the exceptional null event (the zero path is a counting path, and modification on a null event changes no distribution) yields a rate-$1$ homogeneous Poisson process all of whose paths are counting paths.

Now take the finite probability space $\{1,\dots,l\}^N$ with the power-set $\sigma$-algebra and the measure with weights $p_x$, and for each of the $Nl(l-1)+N\tilde{l}$ clock labels a copy of the space just constructed. Form the finite iterated product with the product $\sigma$-algebra and the product measure. The coordinate liftings of the initial-state variable and of the clock processes have the required distributions (their finite-dimensional events are rectangles with full factors elsewhere), all their paths are counting paths, and the family of $\sigma$-algebras consisting of $\sigma(\varsigma^1_0,\dots,\varsigma^N_0)$ and the individual clock $\sigma$-algebras is independent, because each of these $\sigma$-algebras is contained in the preimage of its own factor and the product measure multiplies over rectangles. This is an $N$-agent driving system with the prescribed initial law, proving (i).

Part B: the canonical construction. Fix a driving system and the policy $h$. For every $\omega$ define recursively: $T_0=0$, $x_0=\varsigma_0(\omega)$, empty record, $A^a_{(0)}=0$ and $k^a_0=0$ for every clock label. Given the step-$n$ data $(T_n\le T,\ x_n,\ \text{record}\ \rho_n,\ A^a_{(n)},\ k^a_n)$, let $\alpha_n(s)=h_{|\rho_n|}(s,\rho_n)$ for $s\in[T_n,T]$ (a measurable function of $s$ by Observation-Driven Control Policy, whose sections are measurable because $s\mapsto(s,\tau^\circ)$ pulls relatively open sets back to relatively open sets), and define the continuous nondecreasing functions

the integrands being measurable by Sequentially Continuous Functions of Measurable Euclidean Maps are Measurable (each $\lambda^a(x_n,\cdot)$ is sequentially continuous by Transition-Rate Family and Observation-Rate Family) and bounded by $\bar{\beta}$, so that $\phi^a_n$ is $\bar{\beta}$-Lipschitz by the integral toolkit. Set $t^a_n=\inf\{t\in(T_n,T]:\ \phi^a_n(t)\ge\tau_{k^a_n+1}(Y^a(\omega))\}$ (infimum of the empty set being $+\infty$), $T_{n+1}=\min_a t^a_n$, and the winner set $W_{n+1}=\{a:\ t^a_n=T_{n+1}\}$ when $T_{n+1}\le T$. If $T_{n+1}>T$ or $T_{n+1}=+\infty$ the construction is complete and all data are extended constantly to $[T_n,T]$. Otherwise update: $A^a_{(n+1)}=\phi^a_n(T_{n+1})$ for every $a$; for each winner, increase its count by one; for a transition winner $(i,\sigma\gamma)$ replace the $i$-th coordinate of the state vector by $\gamma$; for an observation winner $(i,\upsilon)$ append $(T_{n+1},\upsilon)$ to the record; if $|W_{n+1}|\ge2$, mark the step $\mathrm{BAD}$ and apply the winners' updates in some fixed order. Since each step increases $\sum_ak^a$ by at least one and $k^a\le Y^a_{\bar{\beta}T}(\omega)<\infty$ (consumed clock time never exceeds $\bar{\beta}T$, and a count of $k$ forces $\tau_k(Y^a)\le\bar{\beta}T$, i.e. $Y^a_{\bar{\beta}T}\ge k$ by (F1)), the recursion terminates for every $\omega$ after finitely many steps. Define $\sigma^i_t$ as the $i$-th coordinate of the current state vector, $\Upsilon^\upsilon_t=\frac1N\#\{\text{record entries with channel}\ \upsilon\ \text{and time}\le t\}$, $\alpha_t=h_{K_t}(t,\text{record up to }t)$, and let $\Omega_0$ be the event that no step is marked $\mathrm{BAD}$.

All step data are random variables, by induction on $n$: $\omega\mapsto\phi^a_n(t)$ is measurable for fixed $t$ (Tonelli, applied to the jointly measurable integrand built from the finitely many measurable step-$n$ variables), and $(t,\omega)\mapsto\phi^a_n(t)$ is jointly measurable, being continuous in $t$ and measurable in $\omega$ (pointwise limit of grid discretizations in $t$); since $\phi^a_n$ is continuous and nondecreasing, $\{t^a_n\le t\}=\{\phi^a_n(t)\ge\tau_{k^a_n+1}(Y^a)\}$, and $\tau_k(Y^a)$ is measurable because $\{\tau_k(Y^a)\le u\}=\{Y^a_u\ge k\}$ by (F1); the updates are finite case distinctions. In particular $\Omega_0\in\mathcal{F}$.

Part C: on $\Omega_0$ the construction is a solution. Conditions 1--6 of Solution of the Controlled N-Agent Dynamics hold at every $\omega\in\Omega_0$. Condition 1 is immediate. For condition 2: the paths $s\mapsto\eta^{i,\sigma}_s$, $s\mapsto\Sigma_s$, $s\mapsto\alpha_s$ are constant between event times with measurable values and measurable breakpoints, so all required maps are jointly measurable in $(s,\omega)$ (each is a finite sum of terms $\mathbf{1}_{\{T_n\le s<T_{n+1}\}}\cdot(\text{step-}n\ \text{expression})$, jointly measurable by the previous paragraph and Sequentially Continuous Functions of Measurable Euclidean Maps are Measurable applied on the product space), and on $\Omega_0$ the resulting consumed clock times of the definition coincide with the recursion values: between $T_n$ and $T_{n+1}$ no clock crosses its next level, so no counter jumps, the state and record are frozen, the actual rates equal the frozen rates, and additivity of the integral gives $A^{a}_t=\phi^a_n(t)$ for $t\in[T_n,T_{n+1}]$. For condition 3: each $N^a_t=Y^a_{A^a_t}$ is nondecreasing, right-continuous (the time change is continuous and the clock path right-continuous), integer-valued, and has unit jumps, since a jump of $N^a$ at $t$ has size $Y^a_{A^a_t}-Y^a_{A^a_t-}\le1$; distinct clocks never jump at the same time on $\Omega_0$, because a ring of clock $a$ occurs exactly at those event times where $a$ is the winner (crossings occur only at candidate times, by minimality of $T_{n+1}$), winners are unique on $\Omega_0$, and one clock cannot ring twice at one time by (F2). Hence every individual counter, the observation total, and the grand total restrict counting paths. Conditions 4, 5, 6 hold by construction (the record is precisely the jump-time/channel sequence of the observation total; the state identity telescopes over the transition events).

Part D: $P(\Omega_0)=1$ via the event-chain invariant. For each clock let $\xi^a_1,\xi^a_2,\dots$ be the interarrival times of $Y^a$; by Jump Times of the Homogeneous Poisson Process: Finiteness and Exponential Interarrival Law and the independence of the driving family, for every finite set of pairs $(a,j)$ the variables $\xi^a_j$ and the initial states are jointly independent, with $P(\xi^a_j>u)=e^{-u}$: events generated by distinct clocks factorize by driving-system independence, within one clock the $\xi^a_j$ are independent by the cited lemma, and the two-level factorization $P(D_0\cap\bigcap_a E_a)=P(D_0)\prod_aP(E_a)=P(D_0)\prod_a\prod_jP(E_{a,j})$ combines them. Work on the almost sure event where, for every clock, all jump times are finite and strictly increasing (part (a) of the cited lemma), so that $\tau_j(Y^a)=\xi^a_1+\dots+\xi^a_j$.

Let $D_n=(\varsigma_0,(T_1,W_1),\dots,(T_n,W_n))$, let $R_n$ be the event that the construction performs at least $n$ steps with all of $W_1,\dots,W_n$ singletons, and define the residual levels $\rho^a_n=\tau_{k^a_n+1}(Y^a)-A^a_{(n)}\ge0$ on $R_n$. Note $R_n$ and the counts $k^a_n$ are functions of $D_n$, and $(D_n,\rho^a_n)$ is a function of $\varsigma_0$ and the array entries $\xi^a_j$ with $j\le k^a_n+1$. We prove by induction on $n$:

for all bounded measurable $\Phi\ge0$ and all $v_a\ge0$. Since the indicator products over rays form a $\pi$-system, $(\star_n)$ says exactly that, restricted to $R_n$, the joint law of $(D_n,(\rho^a_n)_a)$ is the product of the law of $D_n$ on $R_n$ with independent unit exponentials, by Dynkin's Pi-Lambda Theorem; in particular $(\star_n)$ extends to all bounded measurable functions of $(D_n,(\rho^a_n)_a)$ that factor through this pair, and Tonelli applies to the product law. The base case $n=0$ is the joint independence and exponential law above ($D_0=\varsigma_0$, $\rho^a_0=\xi^a_1$, $R_0=\Omega$).

For the step, fix $n$. Given $D_n=d$ on $R_n$ (which determines $T_n$, $x_n$, the record, the counts $\kappa_a=k^a_n$, and the frozen consumption functions $c^a(d,t)=\phi^a_n(t)-A^a_{(n)}$, continuous, nondecreasing, $\bar{\beta}$-Lipschitz), clock $a$ rings first at $t^a=\inf\{t>T_n:\ c^a(d,t)\ge\rho^a_n\}$, so that $\{t^a\le t\}=\{c^a(d,t)\ge\rho^a_n\}$, and $(T_{n+1},W_{n+1})$ is a measurable function of $(d,(\rho^a_n)_a)$. Ties are null: for $b\neq a$, on $\{t^a\le T,\ t^b=t^a\}$ continuity of $c^b(d,\cdot)$ forces $\rho^b_n=c^b(d,t^a)$; for fixed $d$ and fixed $\rho^a_n$ this pins $\rho^b_n$ to a single value, of probability zero under the atomless exponential law, so by Tonelli under the product law of $(\star_n)$, $P(R_n\cap\{\text{step }n{+}1\ \mathrm{BAD}\})=0$.

Propagation: fix a winner label $a$ and a count vector $\kappa$, let $E_a$ be the event that the chain is tie-free through step $n+1$ with $k^\cdot_n=\kappa$, winner $W_{n+1}=\{a\}$, and $T_{n+1}\le T$; up to the null tie set, $E_a$ is the event that $R_n$ holds, $k^\cdot_n=\kappa$, $t^a\le T$, and $\rho^b_n>c^b(D_n,t^a)$ for every $b\neq a$. On $E_a$: $D_{n+1}=(D_n,T_{n+1},\{a\})$ with $T_{n+1}=t^a$, $\rho^a_{n+1}=\xi^a_{\kappa_a+2}$, and $\rho^b_{n+1}=\rho^b_n-c^b(D_n,t^a)$ for $b\neq a$. Let $\Phi'\ge0$ be bounded measurable and $v_b\ge0$. The variable $\mathbf{1}_{E_a}\Phi'(D_{n+1})\prod_{b\neq a}\mathbf{1}_{\{\rho^b_{n+1}>v_b\}}$ is a measurable function of $(\varsigma_0,\ \xi^{a'}_j:\ j\le\kappa_{a'}+1)$, while $\xi^a_{\kappa_a+2}$ is independent of those variables by the joint independence of the array; hence

For the remaining factor, use the product law of $(\star_n)$ and Tonelli, integrating the loser coordinates at fixed $(d,\rho^a_n)$ (which determine $t^a$, $D_{n+1}$, and the values $c^b(d,t^a)$): the joint constraint of $E_a$ and survival is $\rho^b_n>c^b(d,t^a)+v_b$ for each $b\neq a$, and

where $\mathrm{Exp}$ is the law with survival function $v\mapsto e^{-v}$. The constant factor $\prod_{b\neq a}e^{-v_b}$ splits out of the outer integral over $(d,\rho^a_n)$, and what remains is the identical iterated integral with all $v_b=0$, namely $\mathbb{E}[\mathbf{1}_{E_a}\Phi'(D_{n+1})]$. Hence

The events $E_a$ over winner labels $a$ and count vectors $\kappa$ partition $R_{n+1}$ up to null sets, so summing yields $(\star_{n+1})$. Finally, the construction performs only finitely many steps and each step is $\mathrm{BAD}$ with probability zero, so $P(\Omega_0)\ge1-\sum_nP(R_n\cap\{\text{step }n{+}1\ \mathrm{BAD}\})=1$. With Part C, the constructed collection together with the regular event $\Omega_0$ (intersected with the almost sure array event above) is a solution; existence in (ii) is proved.

Part E: uniqueness. Let $(\sigma^i,\Upsilon^\upsilon,\alpha)$ be any solution with regular event $\Omega_0'$. Fix $\omega\in\Omega_0'\cap\{\text{all clock jump times finite and strictly increasing}\}$ (an almost sure event). By condition 3 the grand total restricts a counting path; let $S_1<S_2<\dots$ be its jump times in $[0,T]$. We argue by induction on $j$ with the hypothesis that all event times, jumping labels, states, record entries, consumed clock times, and counters of the solution coincide with those of the canonical construction up to and including the $j$-th event ($j=0$: both start from $\varsigma_0(\omega)$ with empty record and zero consumed times). On $[S_j,S_{j+1})$ no counter jumps, so by conditions 5 and 6 the states and record are constant, the consumed clock times are the integrals of the frozen rates, i.e. coincide with the construction's $\phi^a_j$, and each counter $N^a_t=Y^a_{A^a_t}$ stays constant until $A^a$ reaches $\tau_{k^a+1}(Y^a)$; hence $S_{j+1}$ equals the constructed $T_{j+1}$, the jumping clock is the constructed winner, and the state/record update agrees, propagating the hypothesis. After the finitely many events the two collections agree at every $t\in[0,T]$ at this $\omega$. Since this holds almost surely, any two solutions are indistinguishable (both agree almost surely with the construction). This proves (ii).

Part F: proof of (iii). On the regular event, $A^{i,\sigma\gamma}_t\le Bt$ and $\tilde{A}^{i,\upsilon}_t\le\tilde{B}t$ by condition 2, so monotonicity of clock paths gives $N^{i,\sigma\gamma}_t=Y^{i,\sigma\gamma}_{A^{i,\sigma\gamma}_t}\le Y^{i,\sigma\gamma}_{Bt}$ termwise, and likewise for observation counters; summing gives the display. Each dominating variable is Poisson with parameter $Bt$ or $\tilde{B}t$, hence square-integrable by Moments of the Poisson Distribution, and $0\le N^a_t\le Y^a_{Bt}$ almost surely forces square-integrability of each counter.

Part G: proof of (iv). The counters and initial states generate $\mathcal{F}^{\mathrm{sys}}_t$, so they are adapted; $\eta^{i,\gamma}_t$ is $\mathcal{F}^{\mathrm{sys}}_t$-measurable by condition 6 ($\eta^{i,\gamma}_0=\mathbf{1}_{\{\varsigma^i_0=\gamma\}}$), hence so are $\sigma^i_t$, $\Sigma^\gamma_t$, and $\Upsilon^\upsilon_t$ (condition 4). For the consumed clock times: the event times $S_j\wedge t$ and jumping labels up to $t$ are $\mathcal{F}^{\mathrm{sys}}_t$-measurable ($\{S_j\le s\}$ is the event that the grand total at $s$ is at least $j$, for $s\le t$; the label is identified by which counter increases across $S_j$, expressible with countably many counter variables at rational times and at $S_j$ via right-continuity), the frozen rates on each inter-event interval are measurable functions of the event data, and $A^a_t$ is the resulting finite sum of integrals, hence $\mathcal{F}^{\mathrm{sys}}_t$-measurable. For the control: $K_t$, the observation event times $\tau_j\le t$, and the channels $\upsilon_j$ are $\mathcal{G}_t$-measurable by the same argument applied to the observation processes ($N\Upsilon^\upsilon$ are counting-path restrictions on the regular event, and $\mathcal{G}_t$ contains the null events), which is the second assertion of (iv); and $\alpha^j_t=\sum_{k}\mathbf{1}_{\{K_t=k\}}h^j_k(t,\tau_1,\dots,\tau_k,\upsilon_1,\dots,\upsilon_k)$ is a countable sum of compositions of the measurable policy functions with $\mathcal{G}_t$-measurable vectors (the map $\omega\mapsto(t,\tau_1,\dots,\tau_k)$ pulls the generating relatively open boxes back to $\mathcal{G}_t$-events), hence $\mathcal{G}_t$-measurable.

Part H: proof of (v). As in Part C, $(t,\omega)\mapsto(\Sigma_t,\alpha_t)\mathbf{1}_{\Omega_0}$ is jointly measurable, so by Sequentially Continuous Functions of Measurable Euclidean Maps are Measurable applied on the product space with $g=L$ (sequentially continuous by Population Cost Data), the map $(t,\omega)\mapsto L(\Sigma_t,\alpha_t)\mathbf{1}_{\Omega_0}$ is jointly measurable and bounded below by $-C_L$; its sections in $t$ are measurable, giving the almost sure path measurability, and the truncated maps $\omega\mapsto\int_{[0,T]}\min(L(\Sigma_t,\alpha_t),n)\,dt$ are $\mathcal{F}$-measurable by Tonelli (applied to the nonnegative integrand $\min(L,n)+C_L$). Letting $n\to\infty$: the sublevel sets of the increasing limit are countable intersections of sublevel sets of the truncations, so the limit is measurable in the stated sense; $G(\Sigma_T)$ is measurable by Sequentially Continuous Functions of Measurable Euclidean Maps are Measurable, the sum is bounded below by $-C_LT-C_G$ using both lower bounds, and the expectation is the monotone limit of the expectations of the truncations, which exists in $(-\infty,+\infty]$.

Part I: proof of (vi). First note that every $F\in\mathcal{F}^{\mathrm{sys}}_r$ differs by a null event from an event in the $\sigma$-algebra generated by the generators alone (initial states and counters up to $r$): the collection of events with this property is a $\sigma$-algebra containing the generators and the null events. It therefore suffices to prove the claim for $F$ in the unaugmented $\sigma$-algebra, and, the collection of $F$ for which the claimed $H$ exists being a $\sigma$-algebra, for $F$ a finite intersection of generator preimages. By Part E we may replace the solution by the canonical construction of Part B up to a null event, which is absorbed by the symmetric-difference formulation. Run the construction a second time with each clock path $Y^a$ replaced by the stopped path $u\mapsto Y^a_{u\wedge c_a}$ (again a counting path). By induction over the steps, as long as every consumed clock time stays $\le c_a$, the two runs read their clocks only at arguments $\le c_a$, where the paths agree, so all step data coincide; moreover the stopped run's data are, by Part B, measurable with respect to $\mathcal{H}$ (the $\sigma$-algebra of the statement), since the stopped paths are $\mathcal{H}$-measurable at every argument. The construction's event $C$ coincides with the corresponding event for the stopped run (on either version of $C$ the runs agree, by the induction just described applied up to time $r$), so the construction's $C$ lies in $\mathcal{H}$; the solution's $C$ agrees with it up to a null event, which proves the first clause of (vi). Likewise every counter variable $N^a_s$, $s\le r$, agrees on $C$ with its stopped-run counterpart, which is $\mathcal{H}$-measurable; hence for $F$ a finite intersection of generator preimages, $F\cap C$ equals $H\cap C$ for an $\mathcal{H}$-event $H$, up to the null event where solution and construction differ. This proves (vi). $\blacksquare$