Existence, Uniqueness, and Regularity for the Controlled N-Agent Dynamics

Let $N$, $l$, $\tilde{l}$, $m$ be natural numbers with $N\ge1$, $l\ge2$, $\tilde{l}\ge1$, $m\ge1$, let $\mathcal{A}$ be a nonempty subset of Euclidean space $\mathbb{R}^m$, let $B$ and $\tilde{B}$ be nonnegative real numbers, let $\beta$ be a transition-rate family on $l$ states with control set $\mathcal{A}$ and rate bound $B$, let $\tilde{\beta}$ be an observation-rate family on $l$ states with $\tilde{l}$ observation channels and rate bound $\tilde{B}$, and let $T>0$ be a real number.

(i) (Driving systems exist.) For every family $(p_x)$ of nonnegative real numbers indexed by $x\in\{1,\dots,l\}^N$ with $\sum_x p_x=1$, there exists an $N$-agent driving system whose initial states satisfy $P(\varsigma^1_0=x_1,\dots,\varsigma^N_0=x_N)=p_x$ for every $x=(x_1,\dots,x_N)$.

(ii) (Existence and uniqueness of solutions.) For every $N$-agent driving system $(\Omega,\mathcal{F},P)$ and every observation-driven control policy $h$ with horizon $T$, control dimension $m$ and $\tilde{l}$ channels which is $\mathcal{A}$-valued, there exists a solution of the controlled $N$-agent dynamics on $[0,T]$; and any two solutions $(\sigma^i,\Upsilon^\upsilon,\alpha)$ and $(\hat{\sigma}^i,\hat{\Upsilon}^\upsilon,\hat{\alpha})$ are indistinguishable: almost surely, for all $t\in[0,T]$, $\sigma^i_t=\hat{\sigma}^i_t$ for all $i$, $\Upsilon^\upsilon_t=\hat{\Upsilon}^\upsilon_t$ for all $\upsilon$, and $\alpha_t=\hat{\alpha}_t$.

In clauses (iii)--(vii) below, fix an $N$-agent driving system $(\Omega,\mathcal{F},P)$ and an $\mathcal{A}$-valued observation-driven control policy $h$ as in (ii), and let solution mean a solution of the controlled $N$-agent dynamics on $[0,T]$ for these data, with regular event $\Omega_0$ and with $\Sigma$, the consumed clock times $\mathcal{T}^{i,\sigma\gamma}$ and $\tilde{\mathcal{T}}^{i,\upsilon}$, the counters $N^{i,\sigma\gamma}$ and $\tilde{N}^{i,\upsilon}$, the observation total $\tilde{c}$, the counts $K_t=\tilde{c}_t$, the jump times $\tau_1<\dots<\tau_{K_T}$ of the observation total, the filtrations $(\mathcal{G}_t)$ and $(\mathcal{F}^{\mathrm{sys}}_t)$, and the remaining notation as in that definition.

(iii) (Event count bound.) For any solution, almost surely, for all $t\in[0,T]$,

where the sums run over $i\in\{1,\dots,N\}$, ordered pairs $(\sigma,\gamma)$ with $\sigma\neq\gamma$, and $\upsilon\in\{1,\dots,\tilde{l}\}$. In particular, since each variable on the right has the Poisson distribution and finite second moment, every counter $N^{i,\sigma\gamma}_t$ and $\tilde{N}^{i,\upsilon}_t$ is square-integrable.

(iv) (Adaptedness.) For any solution, the processes $\sigma^i$, $\eta^{i,\gamma}$, $\Sigma^\gamma$, $N^{i,\sigma\gamma}$, $\tilde{N}^{i,\upsilon}$, $\mathcal{T}^{i,\sigma\gamma}$, $\tilde{\mathcal{T}}^{i,\upsilon}$, and $\Upsilon^\upsilon$ are adapted to the system filtration $(\mathcal{F}^{\mathrm{sys}}_t)_{t\in[0,T]}$ of the solution; for every $t\in[0,T]$, the observation-event count $K_t$, the observation event times $\tau_j$ with $j\le K_t$, and their channels $\upsilon_j$ with $j\le K_t$ are measurable with respect to the observation filtration $\mathcal{G}_t$ of the solution; and each component $\alpha^j_t$ of the control is $\mathcal{G}_t$-measurable.

(v) (Cost well-definedness.) Let $(L,G)$ be population cost data on $l$ states with control dimension $m$, and let $C_L$ and $C_G$ be the nonnegative constants of that definition, so that $L\ge-C_L$ and $G\ge-C_G$ everywhere. For any solution, almost surely the path $t\mapsto L(\Sigma_t,\alpha_t)$ is measurable on $[0,T]$ with the trace Borel $\sigma$-algebra and is bounded below by $-C_L$, so that the Lebesgue integral $\int_{[0,T]}L(\Sigma_t,\alpha_t)\,dt$ exists in $(-\infty,+\infty]$; on the exceptional event of probability zero where that path fails to be measurable, the integral is defined to be $0$, so that the map

is defined on all of $\Omega$. That map is measurable as an extended-real-valued map (the sets on which it is at most $a$ are events for every real $a$), it is bounded below by the constant $-C_LT-C_G$, and therefore its expectation is well defined in $(-\infty,+\infty]$ as the limit of the expectations of its truncations at level $n$ as $n\to\infty$.

(vi) (Clock-reading bound.) For any solution, fix $r\in[0,T]$ and nonnegative real numbers $c_{i,\sigma\gamma}$ (one for each transition clock) and $c_{i,\upsilon}$ (one for each observation clock), and let $C$ be the event that $\mathcal{T}^{i,\sigma\gamma}_r\le c_{i,\sigma\gamma}$ and $\tilde{\mathcal{T}}^{i,\upsilon}_r\le c_{i,\upsilon}$ for all indices. Let $\mathcal{H}$ denote the $\sigma$-algebra generated by the initial states $\varsigma^1_0,\dots,\varsigma^N_0$ together with the clock variables $Y^{i,\sigma\gamma}_u$ for $0\le u\le c_{i,\sigma\gamma}$ and $\tilde{Y}^{i,\upsilon}_u$ for $0\le u\le c_{i,\upsilon}$ (all indices). Then the event $C$ agrees up to an event of probability zero with an event of $\mathcal{H}$, and for every event $F\in\mathcal{F}^{\mathrm{sys}}_r$ there is an event $H\in\mathcal{H}$ such that the symmetric difference of $F\cap C$ and $H\cap C$ is an event of probability zero.

(vii) (Well-posedness of the defining data.) Let state processes $\sigma^i$, observation processes $\Upsilon^\upsilon$, a control process $\alpha$, and an event $\Omega_0\in\mathcal{F}$ with $P(\Omega_0)=1$ satisfy conditions 1--4 of the definition of a solution, with the derived quantities named there. Then:

(a) for every $\omega\in\Omega$ and every $t\in[0,T]$ the empirical state measure $\Sigma_t(\omega)$ lies in the probability simplex $\Delta^l$;

(b) the two integrands displayed in condition 2 take values in $[0,B]$ and $[0,\tilde{B}]$ respectively; consequently the consumed clock times are defined for every $\omega\in\Omega$, satisfy $0\le\mathcal{T}^{i,\sigma\gamma}_t\le Bt$ and $0\le\tilde{\mathcal{T}}^{i,\upsilon}_t\le\tilde{B}t$ for all $t\in[0,T]$, vanish identically off $\Omega_0$, and are $\mathcal{F}$-measurable in $\omega$ for each fixed $t$;

(c) each transition counter $N^{i,\sigma\gamma}_t$ and each observation counter $\tilde{N}^{i,\upsilon}_t$ is a random variable on $(\Omega,\mathcal{F},P)$, and all of them vanish off $\Omega_0$;

(d) at every $\omega\in\Omega_0$ and for each $j\in\{1,\dots,K_T\}$ there is exactly one channel $\upsilon\in\{1,\dots,\tilde{l}\}$ such that some observation counter with channel $\upsilon$ jumps at $\tau_j$, so that the channel $\upsilon_j$ appearing in condition 5 is well defined;

(e) each of the families $(\mathcal{G}_t)_{t\in[0,T]}$ and $(\mathcal{F}^{\mathrm{sys}}_t)_{t\in[0,T]}$ is a filtration with time index restricted to $[0,T]$, and $\mathcal{G}_t\subseteq\mathcal{F}^{\mathrm{sys}}_t$ for every $t\in[0,T]$.