Reason: First published version. Proof that the realized control is jointly measurable, via measurability of the observation record and a countable measurable partition of the time-chance product on which the policy acts through a fixed record, and that the pair of the initial empirical state measure and the realized control is a random element of the product space.
(D1) (Arithmetic.) If f1,…,fd:Ξ→R are measurable, E⊆Rd is nonempty with (f1(ξ),…,fd(ξ))∈E for every ξ, and g:E→R is sequentially continuous on E, then g(f1,…,fd) is measurable, by Sequentially Continuous Functions of Measurable Euclidean Maps are Measurable. Since sums, differences, products, absolute values, maxima and minima of finitely many real arguments are given by sequentially continuous functions on Rd, all of these operations preserve measurability of real-valued maps; the same device applied to the indicator of a set A∈H, which is measurable because its preimages are ∅, A, Ξ∖A and Ξ, shows that a product of a measurable map with such an indicator is measurable. In particular {f1≤f2}={f2−f1≥0}∈H for measurable f1,f2.
(D2) (Piecewise measurability.) Let (En)n be a countable family in H with ⋃nEn=Ξ, and let f:Ξ→R be such that En∩f−1(A)∈H for every n and every A∈B(R). Then f is measurable, since f−1(A)=⋃n(En∩f−1(A)).
(D3) (Sublevel criterion.) Let f:Ξ→R be bounded, say 0≤f≤K, and suppose {f≤q}∈H for every real q. Then f is measurable. Indeed, for each natural number n let
Step 1: the observation totals are counting paths. By condition 3 of Solution of the Controlled N-Agent Dynamics each N~ti,υ is a random variable, so by (D1) so are the channel subtotalsc~tυ=∑i=1NN~ti,υ and the observation total c~t=∑υ=1l~c~tυ, for each fixed t∈[0,T].
Fix ω∈Ω. For every i and υ the map t↦N~ti,υ(ω) on [0,T] agrees with the restriction of a counting path: at points of Ω0 this is required by condition 3, and outside Ω0 the counter vanishes identically, which is the restriction of the counting path that is identically zero. In particular each such map is nondecreasing, takes values in the nonnegative integers, vanishes at t=0, and satisfies the right-continuity property of claim 3 of Counting Path and Its Jump Times. For a nondecreasing integer-valued map that property says exactly that the map is constant on some interval [t,t+η] with η>0 to the right of each t<T; a finite sum of such maps is again nondecreasing, integer-valued, zero at t=0, and constant to the right of each t<T on the intersection of finitely many such intervals. Hence for every ω the maps t↦c~tυ(ω) and t↦c~t(ω) on [0,T] are nondecreasing, integer-valued, vanish at t=0, and are right-continuous in that sense.
Step 2: the times τj and τnυ. Let c be any map [0,T]→R with the four properties just listed, let j≥1 be a natural number, and put θ=inf{t∈[0,T]:c(t)≥j}, with θ=T+1 when this set is empty. We claim that for q∈[0,T],
c(q)≥jif and only ifθ≤q.
If c(q)≥j then q belongs to the set E={t∈[0,T]:c(t)≥j}, so θ≤q. Conversely suppose θ≤q; then E is nonempty, and by monotonicity of c it is an up-set: if t∈E and t≤t′≤T then t′∈E.
We claim θ∈E. Suppose not. Then θ<T: for if θ=T, then E is a nonempty up-set contained in [0,T] with greatest lower bound T, so E={T}, whence θ=T∈E, a contradiction. Also, since θ is the greatest lower bound of E and θ∈/E, every s with θ<s≤T satisfies s∈E: there is a point of E in [θ,s], and E is an up-set. Hence c(s)≥j for all such s, while c(θ)<j, so c is not constant on any interval [θ,θ+η] with η>0. This contradicts the right-continuity property, which for a nondecreasing integer-valued map on [0,T] provides exactly such an interval at every point θ<T. So θ∈E after all, and then c(q)≥c(θ)≥j by monotonicity, since θ≤q.
Apply this to c=c~(ω). The resulting θ is τj(ω), so the displayed equivalence of claim 1 holds for every ω∈Ω, every j≥1 and every t∈[0,T]. The sets {t∈[0,T]:c~t≥j} decrease as j increases, so their infima increase and τ1(ω)≤τ2(ω)≤⋯. Also τj takes values in [0,T]∪{T+1} and, for every real q,
all of which lie in F because c~q is a random variable. Since 0≤τj≤T+1, device (D3) shows that τj is a random variable. For υ∈{1,…,l~} and a natural number n≥1 the same argument applied to c=c~υ(ω) shows that
τnυ=inf{t∈[0,T]:c~tυ≥n}(taken to be T+1 when the set is empty)
is a random variable with values in [0,T]∪{T+1}, and that c~tυ(ω)≥n if and only if τnυ(ω)≤t.
Step 3: the channels υj. For j≥1 put
υj=min{υ∈{1,…,l~}:τnυ=τj for some natural number n≥1}
when that set is nonempty, and υj=1 otherwise. For each υ the set Aj,υ=⋃n≥1{τnυ=τj} lies in F, since each {τnυ=τj}={τnυ≤τj}∩{τj≤τnυ} lies in F by (D1) and the union is countable; and {υj=υ} is a finite intersection of members of F and of their complements, formed from the Aj,υ′. As υj takes finitely many values, it is a random variable.
Now fix ω∈Ω0 and j with 1≤j≤c~T(ω). By Step 1 the map t↦c~t(ω) agrees on [0,T] with the restriction of a counting path c, and condition 3 requires this; since c(t)≥j for some t≤T, the j-th jump time of c in the sense of Counting Path and Its Jump Times is the infimum of {t≥0:c(t)≥j}, which equals τj(ω) because the qualifying times already occur in [0,T]. As c~0(ω)=0<j we have τj(ω)>0. Condition 5 lists τ1<⋯<τKT as the jump times of the observation total, so the number τj(ω) is the j-th of them, as asserted.
It remains to identify υj(ω) with the channel of condition 5. Write θ=τj(ω). For a channel υ, say that c~υjumps at θ if c~θυ(ω) exceeds c~sυ(ω) for every s∈[0,θ). If c~υ jumps at θ, then with n=c~θυ(ω) we have c~sυ(ω)<n for s<θ and c~θυ(ω)≥n, so τnυ(ω)=θ. Conversely if τnυ(ω)=θ for some n≥1 then, by the equivalence of Step 2, c~sυ(ω)<n for every s<θ while c~θυ(ω)≥n, so c~υ jumps at θ. Thus Aj,υ is, at ω, exactly the condition that c~υ jumps at θ. Next, c~υ jumps at θ precisely when some observation counter with channel υ jumps at θ. Indeed, c~υ is the sum over i of the counters N~i,υ, each nondecreasing. If some N~i,υ jumps at θ, so that N~θi,υ(ω)>N~si,υ(ω) for every s∈[0,θ), then adding to this strict inequality the inequalities N~θi′,υ(ω)≥N~si′,υ(ω) for the remaining indices i′, using claim 3 of Elementary Order Arithmetic in an Ordered Field, gives c~θυ(ω)>c~sυ(ω) for every s∈[0,θ). Conversely, if no counter with channel υ jumps at θ, then for each i there is si∈[0,θ) with N~θi,υ(ω)=N~sii,υ(ω), and taking s to be the largest of these finitely many numbers, which lies in [0,θ) since θ>0, monotonicity gives N~θi,υ(ω)=N~si,υ(ω) for every i and hence c~θυ(ω)=c~sυ(ω), so c~υ does not jump at θ. Condition 5 states that exactly one channel has this property and calls it υj. Hence the set in the definition of υj(ω) is a singleton, whose unique element is the channel of condition 5, and the minimum picks it out. This proves claim 1.
Step 4: α^ is well defined, A-valued, and agrees with α on Ω0. Let ω∈Ω0, t∈[0,T] and k=c~t(ω). If k≥1 then k≤c~T(ω) by monotonicity, so by claim 1 the numbers τ1(ω)≤⋯≤τk(ω) are jump times in [0,T], and τk(ω)≤t by the equivalence of claim 1 applied with j=k. Hence (τ1(ω),…,τk(ω))∈Rk(T) and (t,τ1(ω),…,τk(ω))∈[0,T]×Rk(T), so the value of hk is defined; for k=0 the value h0(t) is defined. Thus α^ is well defined. All values of h lie in A because h is A-valued, and a0∈A, so α^(t,ω)∈A for every t and ω. Finally, condition 5 of Solution of the Controlled N-Agent Dynamics states that at every point of Ω0,
αt=hKt(t,τ1,…,τKt,υ1,…,υKt),Kt=c~t,
with τ1,…,τKt and υ1,…,υKt the jump times of the observation total and their channels; by claim 1 these are exactly the values of our random variables τj and υj for j≤Kt≤c~T(ω). Hence α^(t,ω)=αt(ω) for every t∈[0,T] and every ω∈Ω0.
Step 5: joint measurability of α^. The maps (t,ω)↦t and, for each j, (t,ω)↦τj(ω) are G-measurable: preimages of Borel sets are A×Ω and [0,T]×τj−1(A) respectively, which lie in G by Product Sigma-Algebra. Hence for each j the set {(t,ω):τj(ω)≤t} lies in G by (D1), and therefore so does
with Q0 the complement of {(t,ω):τ1(ω)≤t}; the displayed identity is the equivalence of claim 1.
Set E∗=[0,T]×(Ω∖Ω0) and, for each k≥0 and each v∈{1,…,l~}k (the empty tuple when k=0),
Ek,v=Qk∩([0,T]×(Ω0∩{υ1=v1}∩⋯∩{υk=vk}))∈G.
These sets, together with E∗, form a countable family covering [0,T]×Ω. On E∗ every component of α^ is constant, so its preimages meet E∗ in ∅ or E∗.
Step 7: claim 4. Let ζ∈UA and let its representative, again written ζ, be admissible, so that ∣ζ(t)∣≤R for every t. Let (wr)r∈N be the sequence used in the definition of ρ in the weak metrizability and compactness theorem. Fix r and put
By condition 1 of Solution of the Controlled N-Agent Dynamics we have σ0i=ς0i on Ω0, and in general each occupation indicator η0i,γ is the indicator of an event, so each component Σ0γ=N1∑i=1Nη0i,γ is a random variable by (D1). All values of Σ0 lie in Δl. By claim 2 of the compactness lemma the space (Δl,dΔ) is separable; let DΔ be a countable dense subset. For q∈DΔ the map ω↦dΔ(Σ0(ω),q)=∣Σ0(ω)−q∣ is a random variable by (D1), since x↦∣x−q∣ is sequentially continuous on Rl. So claim 3 of Borel Sets and Measurable Maps in a Separable Metric Space makes Σ0 a random element of (Δl,dΔ).