The Ising Population Model: A Worked Instance of the Fluctuation Theory

This item exhibits a worked instance of the fluctuation theory for controlled, partially observed populations, and is meant to be read on its own; the references beside each display say where that display is established.

1. (The model.)¶ Fix positive real numbers $\chi$, $\psi$, $q$, $q_{0}$, $\mu$, $T$ and rate bounds $0<\underline{a}<1<\bar{a}$, and adopt the Ising population data with these parameters. Each of $N$ agents occupies one of two states, down and up, and the state of the population at time $t$ is recorded by the profile $\Sigma_{t}=(\Sigma^{1}_{t},\Sigma^{2}_{t})$ in the probability simplex $\Delta^{2}$, the two coordinates being the fractions of agents in the two states.

The control is two-dimensional and is nothing but the pair of transition rates: while the control takes the value $\alpha=(\alpha^{1},\alpha^{2})$, each down agent moves up at rate $\alpha^{1}$ and each up agent moves down at rate $\alpha^{2}$. The rates are confined to the box $\mathcal{A}=[\underline{a},\bar{a}]^{2}$, so they can be neither switched off nor made arbitrarily fast. The population is charged the running cost

and no terminal cost. Here $\phi$ is the regularised entropic rate cost: a nonnegative convex function of a single rate, vanishing exactly at the rest rate $1$ and with second derivative $1/u$ on $[\underline{a},\bar{a}]$, which is the curvature of the entropic rate cost. The first term therefore prices the effort of driving each agent away from the rest rate, weighted by the share of the population subject to that rate, and $\chi$ sets how cheap that effort is. The second term is a preference for an even split, of strength $\psi$.

The controller does not see the agents. Each agent in state $\sigma$ emits observations in the matching channel $\sigma$ at rate $q+q_{0}$ and in the other channel at the background rate $q_{0}$, and an admissible policy is a measurable function of the record of those two counting channels alone. It is the presence of the background rate $q_{0}>0$ that keeps both channels informative at every profile, including the two extreme ones.

That these data are of the kind the theory requires, namely an affine-controlled transition-rate family on $2$ states with compact convex control set and rate bound $\bar{a}$, an observation-rate family with $2$ channels and rate bound $q+q_{0}$, and population cost data convex in the control, each with a twice continuously differentiable extension, is the content of claims The Ising Population Model Instantiates the Data of the Fluctuation Theory §control-set, The Ising Population Model Instantiates the Data of the Fluctuation Theory §rates, The Ising Population Model Instantiates the Data of the Fluctuation Theory §rate-extension, The Ising Population Model Instantiates the Data of the Fluctuation Theory §observations, The Ising Population Model Instantiates the Data of the Fluctuation Theory §observation-extension, The Ising Population Model Instantiates the Data of the Fluctuation Theory §cost-data and The Ising Population Model Instantiates the Data of the Fluctuation Theory §cost-extension.

2. (The mean-field equilibrium.)¶ The aggregate state drift of these rates is

(claim The Ising Population Model Instantiates the Data of the Fluctuation Theory §rates): the population moves along $v$ at the net rate at which agents cross, and it stands still exactly when the two flows balance. With unit rates the flows balance at the even split, so the constant maps

form a stationary mean-field triple, with vanishing co-state (claims The Even-Split Equilibrium of the Ising Population Model is a Stationary Mean-Field Triple and is Optimal §pair and The Even-Split Equilibrium of the Ising Population Model is a Stationary Mean-Field Triple and is Optimal §triple).

This equilibrium is optimal, not merely stationary, and for a reason worth naming: the running cost is nonnegative, and it vanishes at exactly one state and control, namely at the even split with unit rates (claim The Ising Population Model Instantiates the Data of the Fluctuation Theory §cost-data). So the mean-field problem started from the even split has value $0$, attained by holding the rates at $1$ forever (claim The Even-Split Equilibrium of the Ising Population Model is a Stationary Mean-Field Triple and is Optimal §optimality). Every deviation, whether moving the rates off $1$ or letting the population drift off balance, is paid for.

3. (The fluctuation problem is scalar.)¶ At finite $N$ the profile does not sit at the equilibrium: it fluctuates at scale $N^{-1/2}$, and the fluctuation $\mathfrak{s}_{t}=\sqrt{N}(\Sigma_{t}-S_{t})$ is a multiple of $v$, since any difference of two profiles is (clause The Ising Population Data §dimensions). The linear-quadratic data governing it are, at every $t\in[0,T]$,

with cost coefficients

(claims The Fluctuation LQG Data of the Ising Equilibrium and Its Joint Coercivity §coefficients and The Fluctuation LQG Data of the Ising Equilibrium and Its Joint Coercivity §lqg-data), where $\mathsf{n}=(1,1)^{\top}$ and $I$ is the identity.

Every one of these matrices acts on the line spanned by $v$ as multiplication by a number: from $v^{\top}v=2$ and $\mathsf{n}^{\top}v=0$ one reads off $\mathcal{E}_{t}v=-2v$, $\mathcal{B}_{t}v=-v$, $Q_{t}v=\psi\,v$ and $\tilde{\mathcal{E}}_{t}v=q\,v$. So the whole problem is a scalar one: a fluctuation that relaxes at rate $2$, is driven by noise of intensity $\Theta^{\star}_{t}$, is steered by a control whose price is $R_{t}$, and is observed through a channel of gain $q$. The observations enter the value only through the single number

(claim The Fluctuation LQG Data of the Ising Equilibrium and Its Joint Coercivity §information), which measures how much the counts reveal about the imbalance; the background rate contributes to $\tilde{\mathcal{E}}_{t}$ only along $\mathsf{n}$, a direction the fluctuation never occupies, so it adds noise without adding information.

4. (The two Riccati equations, solved.)¶ The control cost and the filtering error are governed by two scalar Riccati equations, which can be written down explicitly. With

the solutions of

are

and the matrix families are $Z_{t}=z_{t}vv^{\top}+\mu(T-t)\mathsf{n}\mathsf{n}^{\top}$ and $\Pi_{t}=p_{t}vv^{\top}$, the latter being the error covariance of the approximate Kalman filter (claims The Control and Filter Riccati Families of the Ising Equilibrium in Closed Form §scalar-control, The Control and Filter Riccati Families of the Ising Equilibrium in Closed Form §scalar-filter, The Control and Filter Riccati Families of the Ising Equilibrium in Closed Form §control-riccati and The Control and Filter Riccati Families of the Ising Equilibrium in Closed Form §kalman).

Both run to their equilibrium values exponentially fast, at rates $4\Delta$ and $2\Gamma$:

(the same two claims), so over a long horizon the pair sits at $\bigl((\Delta-1)/(4\chi),\ (\Gamma+2)^{-1}\bigr)$ except near the two ends. The filtering equilibrium $(\Gamma+2)^{-1}$ falls as $q$ grows: if $0<q<q'$ and $\tilde{d}$, $\tilde{d}'$ are the corresponding values of $2q^{2}/(q+2q_{0})$, then $\tilde{d}<\tilde{d}'$, because the inequality $q^{2}(q'+2q_{0})<q'^{2}(q+2q_{0})$ rearranges to $qq'(q-q')+2q_{0}(q^{2}-q'^{2})<0$, which holds since $q<q'$; and a larger $\tilde{d}$ gives a larger $\Gamma$ and hence a smaller $(\Gamma+2)^{-1}$. More measurement, less residual uncertainty.

5. (What the theorem says here.)¶ Write $J^{N}[h]$ for the cost of an admissible policy $h$ in the $N$-agent problem and $J^{MF}$ for the mean-field cost of the equilibrium; the object the theory compares is the recentred cost $\mathcal{J}_{N}$, which measures at scale $N^{-1}$ how much a policy loses against the equilibrium. Provided the initial profiles are deterministic and converge to the even split fast enough, which is what hypotheses (D0) and (DK) of The Fluctuation LQG Value Is the Asymptotically Optimal Value of the Recentred N-Agent Cost over Admissible Families of Observation-Driven Policies, and Is Attained by the Approximate Kalman Policies, under Deterministic Initial States ask and what The Asymptotically Optimal Value of the Recentred N-Agent Cost of the Ising Population Model carries as assumptions, that corollary records in its claims The Asymptotically Optimal Value of the Recentred N-Agent Cost of the Ising Population Model §applies, The Asymptotically Optimal Value of the Recentred N-Agent Cost of the Ising Population Model §lower-bound, The Asymptotically Optimal Value of the Recentred N-Agent Cost of the Ising Population Model §attainment and The Asymptotically Optimal Value of the Recentred N-Agent Cost of the Ising Population Model §value that every hypothesis of that theorem holds for this model, and that

is the asymptotically optimal value: no family of observation-driven policies can hold $\mathcal{J}_{N}$ below $V^{*}-\varepsilon$ for large $N$, and the approximate Kalman policies drive $\mathcal{J}_{N}$ to $V^{*}$. The two terms of the integrand are the two entrywise pairings of The Control and Filter Riccati Families of the Ising Equilibrium in Closed Form §pairings, and they are the two sources of unavoidable cost: the noise the population itself generates, priced by $Z_{t}$ against $\Theta^{\star}_{t}$, and the part of the fluctuation the observations fail to resolve, priced by $\Xi_{t}$ against the filtering error $\Pi_{t}$. Neither involves the convexity weight $\mu$, which enters only the extension of the cost off the simplex (The Asymptotically Optimal Value of the Recentred N-Agent Cost of the Ising Population Model §applies).

6. (Scope.)¶ The sign of the interaction is essential: the cost above rewards splitting, and the ferromagnetic model, which rewards congregation, is not an instance of this example.