The Realized Control of the Controlled N-Agent Dynamics as a Random Element of the Control Set

Adopt the setting of the controlled $N$-agent dynamics: natural numbers $N\ge1$, $l\ge2$, $\tilde{l}\ge1$ and $m\ge1$; a nonempty convex subset $\mathcal{A}$ of Euclidean space $\mathbb{R}^{m}$, called the control set, which is compact for the topology determined by the Euclidean distance; a transition-rate family $\beta$ on $l$ states with control set $\mathcal{A}$; an observation-rate family $\tilde{\beta}$ on $l$ states with $\tilde{l}$ channels; a real number $T>0$; and an $N$-agent driving system $(\Omega,\mathcal{F},P)$ with initial states $\varsigma^{1}_{0},\dots,\varsigma^{N}_{0}$.

By Heine-Borel Theorem in $\mathbb{R}^n$ the set $\mathcal{A}$ is bounded, so there is a real number $R\ge0$ with $|a|\le R$ for every $a\in\mathcal{A}$, where $|\cdot|$ denotes the Euclidean norm. Let $h=(h_{k})_{k\ge0}$ be an observation-driven control policy with horizon $T$, control dimension $m$ and $\tilde{l}$ channels, with record spaces $R_{k}(T)$, and assume $h$ is $\mathcal{A}$-valued.

Let $(\sigma^{i},\Upsilon^{\upsilon},\alpha)$ be a solution of the controlled $N$-agent dynamics on $[0,T]$ for these data, with regular event $\Omega_{0}$, empirical state measure $\Sigma$, observation counters $\tilde{N}^{i,\upsilon}$ and observation total $\tilde{c}$; such a solution exists by claim (ii) of the existence and uniqueness theorem. Thus $\Omega_{0}$ is an event of probability $1$ at each of whose points the six conditions defining a solution hold, each counter $\tilde{N}^{i,\upsilon}_{t}$ is a random variable that vanishes at every point outside $\Omega_{0}$, and $\tilde{c}_{t}=\sum_{i=1}^{N}\sum_{\upsilon=1}^{\tilde{l}}\tilde{N}^{i,\upsilon}_{t}$ for $t\in[0,T]$.

Adopt the notation of the Lebesgue space $L^{2}([0,T];\mathbb{R}^{d})$ in the case $d=m$, including the convention of claim 5 there by which an element of that space is denoted by the same symbol as a representative of it. Let $\mathcal{U}_{\mathcal{A}}$ be the set of $\mathcal{A}$-valued controls and let $\rho$, $d_{\Delta}$, $X$ and $d_{X}$ be as in the compactness lemma for the simplex, the control set and their product, the metric $\rho$ being formed from a fixed sequence $(w_{r})_{r\in\mathbb{N}}$ in $L^{2}([0,T];\mathbb{R}^{m})$ whose set of terms is dense there for the metric of that space, as in claim 1 of the weak metrizability and compactness theorem. Write $\mathcal{B}_{[0,T]}$ and $\lambda_{[0,T]}$ for the restricted Borel $\sigma$-algebra and Lebesgue measure on $[0,T]$, and let $\mathcal{B}_{[0,T]}\otimes\mathcal{F}$ be the product $\sigma$-algebra. Fix $a_{0}\in\mathcal{A}$.

Then the following hold.

1. (A measurable observation record.) For each natural number $j\ge1$ put

$$\tau_{j}=\inf\{t\in[0,T]:\tilde{c}_{t}\ge j\},$$

the greatest lower bound of the displayed set when it is nonempty, with the convention $\tau_{j}=T+1$ when it is empty. Then each $\tau_{j}$ is a random variable with values in $[0,T]\cup\{T+1\}$, one has $\tau_{1}(\omega)\le\tau_{2}(\omega)\le\cdots$ for every $\omega\in\Omega$, and

$$\tilde{c}_{t}(\omega)\ge j\quad\text{if and only if}\quad\tau_{j}(\omega)\le t$$

for every $\omega\in\Omega$, every $j\ge1$ and every $t\in[0,T]$. There are moreover random variables $\upsilon_{j}$, indexed by the natural numbers $j\ge1$, with values in $\{1,\dots,\tilde{l}\}$, such that for every $\omega\in\Omega_{0}$ and every $j$ with $1\le j\le\tilde{c}_{T}(\omega)$ the number $\tau_{j}(\omega)$ is the $j$-th jump time of the observation total and $\upsilon_{j}(\omega)$ is the channel attached to that jump time in condition 5 of the definition of a solution. Fix once and for all such a family $(\upsilon_{j})_{j\ge1}$; claims 2--5 below refer to it.

2. (The realized control.) Define $\hat{\alpha}:[0,T]\times\Omega\to\mathbb{R}^{m}$ by

$$\hat{\alpha}(t,\omega)=h_{k}\bigl(t,\tau_{1}(\omega),\dots,\tau_{k}(\omega),\upsilon_{1}(\omega),\dots,\upsilon_{k}(\omega)\bigr)\quad\text{with }k=\tilde{c}_{t}(\omega)$$

for $\omega\in\Omega_{0}$, the right-hand side being $h_{0}(t)$ when $k=0$, and by $\hat{\alpha}(t,\omega)=a_{0}$ for $\omega\notin\Omega_{0}$. Then the argument of $h_{k}$ displayed above lies in $[0,T]\times R_{k}(T)\times\{1,\dots,\tilde{l}\}^{k}$, so that $\hat{\alpha}$ is well defined; furthermore $\hat{\alpha}(t,\omega)\in\mathcal{A}$ for every $t\in[0,T]$ and every $\omega\in\Omega$; one has $\hat{\alpha}(t,\omega)=\alpha_{t}(\omega)$ for every $t\in[0,T]$ and every $\omega\in\Omega_{0}$; and every component of $\hat{\alpha}$ is measurable with respect to $\mathcal{B}_{[0,T]}\otimes\mathcal{F}$ and the Borel $\sigma$-algebra of the real line.

3. (Each path is an $\mathcal{A}$-valued control.) For every $\omega\in\Omega$ the path $t\mapsto\hat{\alpha}(t,\omega)$ is square-integrable and every one of its values lies in $\mathcal{A}$. Writing $\hat{\alpha}(\omega)$ both for that path and for the element of $L^{2}([0,T];\mathbb{R}^{m})$ it represents, one has $\hat{\alpha}(\omega)\in\mathcal{U}_{\mathcal{A}}$. Since every value of the path lies in $\mathcal{A}$, the path is in particular a representative of $\hat{\alpha}(\omega)$ that is $\mathcal{A}$-valued at every point of $[0,T]$, which is what claim 2 of the flow stability lemma calls an admissible representative.

4. (Weak distances are measurable.) For every $\zeta\in\mathcal{U}_{\mathcal{A}}$ the map $\omega\mapsto\rho\bigl(\hat{\alpha}(\omega),\zeta\bigr)$ is a random variable.

5. (Random elements.) The map $\omega\mapsto\hat{\alpha}(\omega)$ is a random element of $(\mathcal{U}_{\mathcal{A}},\rho)$; the map $\Sigma_{0}$ is a random element of $(\Delta^{l},d_{\Delta})$; and the map $\omega\mapsto\bigl(\Sigma_{0}(\omega),\hat{\alpha}(\omega)\bigr)$ is a random element of $(X,d_{X})$.