Solution of the Controlled N-Agent Dynamics

Let $N$, $l$, $\tilde{l}$, $m$ be natural numbers with $N\ge1$, $l\ge2$, $\tilde{l}\ge1$, $m\ge1$. Let $\mathcal{A}$ be a nonempty subset of Euclidean space $\mathbb{R}^m$. Fix a transition-rate family $\beta$ on $l$ states with control set $\mathcal{A}$ and rate bound $B$, an observation-rate family $\tilde{\beta}$ on $l$ states with $\tilde{l}$ observation channels and rate bound $\tilde{B}$, a real number $T>0$, an $N$-agent driving system $(\Omega,\mathcal{F},P)$ with initial states $\varsigma^i_0$, transition clocks $Y^{i,\sigma\gamma}$, and observation clocks $\tilde{Y}^{i,\upsilon}$, and an observation-driven control policy $h=(h_k)_{k\ge0}$ with horizon $T$, control dimension $m$, and $\tilde{l}$ channels which is $\mathcal{A}$-valued.

Throughout, $\sigma$, $\gamma$, $\gamma'$ denote state labels in $\{1,\dots,l\}$, $\upsilon$ an observation channel in $\{1,\dots,\tilde{l}\}$, $i$ an agent index in $\{1,\dots,N\}$, and $\sigma^i$ the state process of agent $i$.

A solution of the controlled $N$-agent dynamics on $[0,T]$ consists of families of random variables indexed by $t\in[0,T]$ on $(\Omega,\mathcal{F},P)$ (stochastic processes with time restricted to $[0,T]$), namely state processes $\sigma^i=(\sigma^i_t)$ taking values in $\{1,\dots,l\}$, observation processes $\Upsilon^\upsilon=(\Upsilon^\upsilon_t)$ taking real values, and a control process $\alpha=(\alpha_t)$ taking values in $\mathcal{A}$ (each component $\alpha^j$ a real-valued process), together with an event $\Omega_0\in\mathcal{F}$ with $P(\Omega_0)=1$, called the regular event, such that the measurability requirement of condition 2 below holds, and conditions 1, 3, 4, 5 and 6 below hold at every $\omega\in\Omega_0$. The conditions are a joint requirement on the whole collection: condition 2 refers to the control appearing in condition 5, and condition 5 to the counters of condition 3.

Derived notation: the occupation indicators are $\eta^{i,\gamma}_t=1$ if $\sigma^i_t=\gamma$ and $\eta^{i,\gamma}_t=0$ otherwise; the empirical state measure is $\Sigma_t=(\Sigma^1_t,\dots,\Sigma^l_t)$ with $\Sigma^\gamma_t=\frac{1}{N}\sum_{i=1}^N\eta^{i,\gamma}_t$, a point of the probability simplex $\Delta^l$.

1. (State regularity.) For each $i$: $\sigma^i_0=\varsigma^i_0$, and the path $t\mapsto\sigma^i_t$ is piecewise constant and right-continuous: there are a count $K^{(i)}$, either zero or a natural number, and times $0<t^{(i)}_1<\dots<t^{(i)}_{K^{(i)}}\le T$ such that $t\mapsto\sigma^i_t$ is constant on $[0,t^{(i)}_1)$, constant on $[t^{(i)}_k,t^{(i)}_{k+1})$ for each $k\in\{1,\dots,K^{(i)}-1\}$, and constant on $[t^{(i)}_{K^{(i)}},T]$, with the convention that for $K^{(i)}=0$ the path is constant on all of $[0,T]$.

2. (Joint measurability and time changes.) For all $i$, all ordered pairs $(\sigma,\gamma)$ with $\sigma\neq\gamma$, and all $\upsilon$, the maps

are measurable with respect to the product $\sigma$-algebra of the trace Borel $\sigma$-algebra on $[0,T]$ and $\mathcal{F}$, where $\mathbf{1}_{\Omega_0}$ is the function equal to $1$ on $\Omega_0$ and $0$ off $\Omega_0$. For every $\omega\in\Omega$ the consumed clock times are

the Lebesgue integrals over the compact interval $[0,t]$ of the sections in $s$ of the two maps just displayed.

3. (Counting structure.) Define the transition counters and observation counters on all of $\Omega$ by

It is required that each of the maps $t\mapsto N^{i,\sigma\gamma}_t$, each of the maps $t\mapsto\tilde{N}^{i,\upsilon}_t$, the observation total $t\mapsto\tilde{c}_t=\sum_{i=1}^N\sum_{\upsilon=1}^{\tilde{l}}\tilde{N}^{i,\upsilon}_t$, and the grand total obtained by adding to $\tilde{c}_t$ the sum of all transition counters, coincides on $[0,T]$ with the restriction of a counting path.

4. (Observation identity.) For all $\upsilon$ and $t\in[0,T]$:

5. (Control identity.) Let $K_t=\tilde{c}_t$ be the number of observation events up to time $t$, let $\tau_1<\dots<\tau_{K_T}$ be the jump times of the observation total in $[0,T]$, and for each $j\in\{1,\dots,K_T\}$ let $\upsilon_j\in\{1,\dots,\tilde{l}\}$ be the unique channel such that some observation counter with channel $\upsilon_j$ jumps at $\tau_j$. Then for every $t\in[0,T]$:

where the right-hand side is $h_0(t)$ on the event $K_t=0$.

6. (State identity.) For all $i$, all $\gamma$, and all $t\in[0,T]$:

Given a solution, the observation filtration $(\mathcal{G}_t)_{t\in[0,T]}$ is defined by letting $\mathcal{G}_t$ be the $\sigma$-algebra generated by the random variables $\Upsilon^\upsilon_s$ with $0\le s\le t$ and $\upsilon\in\{1,\dots,\tilde{l}\}$ together with every event of $\mathcal{F}$ of probability zero, and the system filtration $(\mathcal{F}^{\mathrm{sys}}_t)_{t\in[0,T]}$ is defined by letting $\mathcal{F}^{\mathrm{sys}}_t$ be the $\sigma$-algebra generated by the initial states $\varsigma^1_0,\dots,\varsigma^N_0$ and the random variables $N^{i,\sigma\gamma}_s$ and $\tilde{N}^{i,\upsilon}_s$ for $0\le s\le t$ and all indices, together with every event of $\mathcal{F}$ of probability zero. Each of these two families is a filtration with time index restricted to $[0,T]$.