Proof of Existence and Basic Estimates at a Maximiser of the Gauge-Doubled Difference on the Wasserstein Space

Each result cited is universally quantified over the data in its own statement and is applied here to the data named in the statement of the lemma. The real line carries the metric $d_{\mathbb{R}}$ with $d_{\mathbb{R}}(s,t)=|s-t|$ of The Absolute Value Metric on the Real Numbers. Throughout, by Basic Properties of a Coercive Penalty Pair: a Lower Bound for the Penalty, Semicontinuity, and Exact Delta-Envelopes (iv),

$$\Phi_{\alpha}(\mu,\nu)=u(\mu)-\delta\,\mathcal{E}(\mu)-v(\nu)-\delta\,\mathcal{E}(\nu)-\tfrac{\alpha}{2}\,\rho(\mu,\nu)^{2}\qquad\bigl((\mu,\nu)\in\mathcal{D}\times\mathcal{D},\ \alpha\text{ positive}\bigr).$$

The compatibility of the order with addition and its transitivity, antisymmetry and totality are axioms of Ordered Field; mixed transitivity is claim 2 of Elementary Order Arithmetic in an Ordered Field. The claims are proved in the order 2, 3, 1, 4, claim 1 using claim 2.

Monotonicity of $\iota$. For $m,n\in\mathbb{N}$ with $m\le n$ one has $\iota(m)\le\iota(n)$. Indeed, by the trichotomy of the order of $\mathbb{N}$ (claim 3 of Properties of the Order on the Natural Numbers) either $m=n$, and then $\iota(m)=\iota(n)$, or $m<n$, and then $\iota(m)<\iota(n)$ by claim 6 of Properties of the Canonical Map from the Natural Numbers to an Ordered Field; in both cases $\iota(m)\le\iota(n)$.

A bound valid on all of $\mathcal{D}\times\mathcal{D}$. Let $\alpha$ be positive and $(\mu,\nu)\in\mathcal{D}\times\mathcal{D}$. The square $\rho(\mu,\nu)^{2}$ is nonnegative by Nonnegativity of Squares in an Ordered Field, so $0\le\tfrac{\alpha}{2}\rho(\mu,\nu)^{2}$ by claim 5 of Elementary Arithmetic in an Ordered Field applied with the nonnegative multiplier $\tfrac{\alpha}{2}$ and $x\cdot0=0$ (claim 1 of Zero Products and Elementary Identities in a Field). From $u(\mu)\le b$ and $b'\le v(\nu)$, the latter equivalent to $-v(\nu)\le-b'$ by claim 3 of Elementary Arithmetic in an Ordered Field used in both directions, we obtain

$$(\dagger)\qquad\Phi_{\alpha}(\mu,\nu)\le u(\mu)-\delta\,\mathcal{E}(\mu)-v(\nu)-\delta\,\mathcal{E}(\nu)\le b-b'-\delta\,\mathcal{E}(\mu)-\delta\,\mathcal{E}(\nu).$$

Also $\rho(\mu_{0},\mu_{0})=0$ by the metric axioms of Metric Space, so $\tfrac{\alpha}{2}\rho(\mu_{0},\mu_{0})^{2}=0$ and

$$(\ddagger)\qquad\Phi_{\alpha}(\mu_{0},\mu_{0})=u(\mu_{0})-v(\mu_{0})-2\delta\,\mathcal{E}(\mu_{0}),$$

the collection of the two equal penalty terms using the distributivity of Field.

Claim 2. Let $\alpha$ be positive, let $\eta$ be nonnegative and let $(\hat{\mu},\hat{\nu})\in\mathcal{D}\times\mathcal{D}$ satisfy $\Phi_{\alpha}(\mu_{0},\mu_{0})-\eta\le\Phi_{\alpha}(\hat{\mu},\hat{\nu})$. Combining this with $(\dagger)$ at $(\hat{\mu},\hat{\nu})$ and with $(\ddagger)$ gives

$$u(\mu_{0})-v(\mu_{0})-2\delta\,\mathcal{E}(\mu_{0})-\eta\le b-b'-\delta\,\mathcal{E}(\hat{\mu})-\delta\,\mathcal{E}(\hat{\nu}),$$

and adding $\delta\mathcal{E}(\hat{\mu})+\delta\mathcal{E}(\hat{\nu})-u(\mu_{0})+v(\mu_{0})+2\delta\mathcal{E}(\mu_{0})+\eta$ to both sides yields the displayed inequality of claim 2,

$$\delta\,\mathcal{E}(\hat{\mu})+\delta\,\mathcal{E}(\hat{\nu})\le b-b'-u(\mu_{0})+v(\mu_{0})+2\delta\,\mathcal{E}(\mu_{0})+\eta=:K_{\eta}.$$

Since $e_{0}\le\mathcal{E}(\hat{\nu})$, multiplying by the nonnegative $\delta$ (claim 5 of Elementary Arithmetic in an Ordered Field) gives $\delta e_{0}\le\delta\mathcal{E}(\hat{\nu})$, whence $\delta\mathcal{E}(\hat{\mu})\le K_{\eta}-\delta e_{0}$. Multiplying by the positive $\delta^{-1}$ (claim 7 of Elementary Order Arithmetic in an Ordered Field and claim 5 of Elementary Arithmetic in an Ordered Field) and using $\delta^{-1}\delta=1$ gives

$$\mathcal{E}(\hat{\mu})\le\delta^{-1}K_{\eta}-e_{0}=\delta^{-1}\bigl(b-b'-u(\mu_{0})+v(\mu_{0})+2\delta\,\mathcal{E}(\mu_{0})\bigr)-e_{0}+\delta^{-1}\eta=c_{0}+\delta^{-1}\eta,$$

the middle identity by the distributivity and commutativity of Field. Interchanging the roles of $\hat{\mu}$ and $\hat{\nu}$ and using $e_{0}\le\mathcal{E}(\hat{\mu})$ gives $\mathcal{E}(\hat{\nu})\le c_{0}+\delta^{-1}\eta$ in the same way.

Claim 3. In the situation of claim 2 both $\hat{\mu}$ and $\hat{\nu}$ lie in $\mathcal{D}$ and satisfy $\mathcal{E}\le c_{0}+\delta^{-1}\eta$, so Coercive Penalty Pairs on the Wasserstein Space (iii), read at the level $c=c_{0}+\delta^{-1}\eta$, gives $M_{2}(\hat{\mu})\le R$ and $M_{2}(\hat{\nu})\le R$. That level is determined by $b,b',u(\mu_{0}),v(\mu_{0}),\mathcal{E}(\mu_{0}),\delta,e_{0}$ and $\eta$ alone, so neither it nor $R$ depends on $\alpha$.

Claim 1. Let $\alpha\in\mathbb{R}$ be positive. By $(\dagger)$ and $e_{0}\le\mathcal{E}$ on $\mathcal{D}$, every value of $\Phi_{\alpha}$ satisfies $\Phi_{\alpha}(\mu,\nu)\le b-b'-2\delta e_{0}$, so the set $\{\Phi_{\alpha}(\mu,\nu):(\mu,\nu)\in\mathcal{D}\times\mathcal{D}\}$, which is nonempty because $\mathcal{D}$ is, is bounded above and has a supremum $M\in\mathbb{R}$ by that clause.

For $n\in\mathbb{N}$ the inverse $\iota(n)^{-1}$ exists and is positive by claim 3 of Properties of the Canonical Map from the Natural Numbers to an Ordered Field. Moreover $1\le n$ by claim 4 of Properties of the Order on the Natural Numbers, so $\iota(1)\le\iota(n)$ by the monotonicity of $\iota$ established above, and $\iota(1)=1$ by claim 1 of Properties of the Canonical Map from the Natural Numbers to an Ordered Field; multiplying $1\le\iota(n)$ by the nonnegative $\iota(n)^{-1}$ (claim 5 of Elementary Arithmetic in an Ordered Field) gives

$$\iota(n)^{-1}\le1\qquad(n\in\mathbb{N}).$$

By claim 3 of Approximation Property of the Supremum and the Infimum in $\mathbb{R}$, applied with the positive $\iota(n)^{-1}$, the set of pairs $(\mu,\nu)\in\mathcal{D}\times\mathcal{D}$ with $M-\iota(n)^{-1}<\Phi_{\alpha}(\mu,\nu)$ is nonempty for every $n\in\mathbb{N}$, so Axiom of Countable Choice provides a sequence of pairs $(\mu_{n},\nu_{n})\in\mathcal{D}\times\mathcal{D}$ with

$$M-\iota(n)^{-1}<\Phi_{\alpha}(\mu_{n},\nu_{n})\le M,$$

the second inequality because $M$ is an upper bound of the set of values. As $\Phi_{\alpha}(\mu_{0},\mu_{0})\le M$ and $\iota(n)^{-1}\le1$, we get $\Phi_{\alpha}(\mu_{0},\mu_{0})-1\le M-\iota(n)^{-1}\le\Phi_{\alpha}(\mu_{n},\nu_{n})$, so claim 2 with $\eta=1$ gives $\mathcal{E}(\mu_{n})\le c_{1}$ and $\mathcal{E}(\nu_{n})\le c_{1}$, where $c_{1}=c_{0}+\delta^{-1}$. Thus both sequences lie in

$$K=\{\sigma\in\mathcal{D}:\mathcal{E}(\sigma)\le c_{1}\},$$

which is sequentially compact in $(\mathcal{P}_{2}(\mathbb{R}^{d}),\rho)$ by Coercive Penalty Pairs on the Wasserstein Space (ii) read at the level $c_{1}$.

Hence there are $\hat{\mu}\in K$ and a strictly increasing sequence $(n_{k})_{k\in\mathbb{N}}$ in $\mathbb{N}$ such that $(\mu_{n_{k}})_{k\in\mathbb{N}}$ converges to $\hat{\mu}$ in $(\mathcal{P}_{2}(\mathbb{R}^{d}),\rho)$; applying sequential compactness once more to the sequence $(\nu_{n_{k}})_{k\in\mathbb{N}}$ in $K$ gives $\hat{\nu}\in K$ and a strictly increasing $(k_{j})_{j\in\mathbb{N}}$ such that $(\nu_{n_{k_{j}}})_{j\in\mathbb{N}}$ converges to $\hat{\nu}$. By A Subsequence of a Convergent Sequence Has the Same Limit, applied to the convergent sequence $(\mu_{n_{k}})_{k\in\mathbb{N}}$ and the indices $(k_{j})_{j\in\mathbb{N}}$, the sequence $(\mu_{n_{k_{j}}})_{j\in\mathbb{N}}$ converges to $\hat{\mu}$. Write $m_{j}=n_{k_{j}}$.

The real sequence $(\Phi_{\alpha}(\mu_{n},\nu_{n}))_{n\in\mathbb{N}}$ converges to $M$ in $(\mathbb{R},d_{\mathbb{R}})$: from the displayed near-maximising inequality, $0\le M-\Phi_{\alpha}(\mu_{n},\nu_{n})\le\iota(n)^{-1}$, so $d_{\mathbb{R}}(\Phi_{\alpha}(\mu_{n},\nu_{n}),M)\le\iota(n)^{-1}$ by claim 6 of Properties of the Absolute Value in an Ordered Field, applied to $x=\Phi_{\alpha}(\mu_{n},\nu_{n})-M$ and $c=\iota(n)^{-1}$, the required bounds $-\iota(n)^{-1}\le x$ and $x\le0\le\iota(n)^{-1}$ being claim 3 of Elementary Arithmetic in an Ordered Field applied to the displayed inequalities. Given a positive $\varepsilon'$, claim 2 of The Archimedean Property of the Real Numbers provides $N\in\mathbb{N}$ with $1<\iota(N)\varepsilon'$, whence for $N\le n$ one has $\iota(N)\le\iota(n)$ by the monotonicity of $\iota$ established above, then $1<\iota(n)\varepsilon'$ and, multiplying by the positive $\iota(n)^{-1}$ (claim 10 of Elementary Order Arithmetic in an Ordered Field), $\iota(n)^{-1}<\varepsilon'$. Applying A Subsequence of a Convergent Sequence Has the Same Limit twice, the sequence $(\Phi_{\alpha}(\mu_{m_{j}},\nu_{m_{j}}))_{j\in\mathbb{N}}$ converges to $M$ as well.

The five estimates. Let $\varepsilon\in\mathbb{R}$ be positive.

Since $u$ is continuous at $\hat{\mu}$ relative to $\mathcal{P}_{2}(\mathbb{R}^{d})$ in $(\mathcal{P}_{2}(\mathbb{R}^{d}),\rho)$, Continuous Map Between Metric Spaces provides a positive $r_{1}$ such that $|u(\sigma)-u(\hat{\mu})|<\varepsilon$, and hence $u(\sigma)<u(\hat{\mu})+\varepsilon$ by claim 3 of Properties of the Absolute Value in an Ordered Field and mixed transitivity, for every $\sigma$ with $\rho(\hat{\mu},\sigma)<r_{1}$. Likewise there is a positive $r_{2}$ such that $|v(\sigma)-v(\hat{\nu})|<\varepsilon$, and hence $-v(\sigma)<-v(\hat{\nu})+\varepsilon$, for every $\sigma$ with $\rho(\hat{\nu},\sigma)<r_{2}$.

By Basic Properties of a Coercive Penalty Pair: a Lower Bound for the Penalty, Semicontinuity, and Exact Delta-Envelopes (ii) and Lower Semicontinuous Function on a Subset of a Metric Space, applied at $\hat{\mu}$ with the positive $\delta^{-1}\varepsilon$, there is a positive $r_{3}$ such that $\mathcal{E}(\hat{\mu})-\delta^{-1}\varepsilon<\mathcal{E}(\sigma)$ for every $\sigma\in\mathcal{D}$ with $\rho(\hat{\mu},\sigma)<r_{3}$; multiplying by the positive $\delta$ (claim 10 of Elementary Order Arithmetic in an Ordered Field) and using $\delta\delta^{-1}=1$ gives $\delta\mathcal{E}(\hat{\mu})-\varepsilon<\delta\mathcal{E}(\sigma)$, that is, $-\delta\mathcal{E}(\sigma)<-\delta\mathcal{E}(\hat{\mu})+\varepsilon$. In the same way there is a positive $r_{4}$ with $-\delta\mathcal{E}(\sigma)<-\delta\mathcal{E}(\hat{\nu})+\varepsilon$ for every $\sigma\in\mathcal{D}$ with $\rho(\hat{\nu},\sigma)<r_{4}$.

For the last estimate write $s=\rho(\hat{\mu},\hat{\nu})$, a nonnegative real number, and put

$$\varepsilon_{0}=\varepsilon\bigl(2\alpha(1+s)\bigr)^{-1},$$

which is positive because $\alpha$ and $1+s$ are positive (for the latter, $0<1\le1+s$) and by claims 5 and 7 of Elementary Order Arithmetic in an Ordered Field. Let $\sigma,\tau\in\mathcal{P}_{2}(\mathbb{R}^{d})$ satisfy $\rho(\hat{\mu},\sigma)<\varepsilon_{0}$ and $\rho(\hat{\nu},\tau)<\varepsilon_{0}$. The triangle inequality and the symmetry of $\rho$ (metric axioms of Metric Space), used twice, give $s\le\rho(\hat{\mu},\sigma)+\rho(\sigma,\tau)+\rho(\tau,\hat{\nu})$ and hence $s-2\varepsilon_{0}<\rho(\sigma,\tau)$. We claim that

$$s^{2}-4\varepsilon_{0}s\le\rho(\sigma,\tau)^{2}.$$

If $2\varepsilon_{0}\le s$, then $0\le s-2\varepsilon_{0}<\rho(\sigma,\tau)$, so $(s-2\varepsilon_{0})^{2}<\rho(\sigma,\tau)^{2}$ by claim 1 of Monotonicity of Squaring on the Nonnegative Elements of an Ordered Field, and $(s-2\varepsilon_{0})^{2}=s^{2}-4\varepsilon_{0}s+4\varepsilon_{0}^{2}$ by claim 5 of Zero Products and Elementary Identities in a Field, the last summand being nonnegative by Nonnegativity of Squares in an Ordered Field, so $s^{2}-4\varepsilon_{0}s\le(s-2\varepsilon_{0})^{2}$. If instead $s<2\varepsilon_{0}$, then multiplying by the nonnegative $s$ gives $s^{2}\le2\varepsilon_{0}s$, so $s^{2}-4\varepsilon_{0}s\le-2\varepsilon_{0}s\le0\le\rho(\sigma,\tau)^{2}$, the middle step because $2\varepsilon_{0}s$ is nonnegative. Multiplying the claimed inequality by the nonnegative $\tfrac{\alpha}{2}$ and using claim 3 of Elementary Arithmetic in an Ordered Field in both directions gives

$$-\tfrac{\alpha}{2}\rho(\sigma,\tau)^{2}\le-\tfrac{\alpha}{2}s^{2}+2\alpha\varepsilon_{0}s .$$

Finally $2\alpha\varepsilon_{0}s=\varepsilon\,s\,(1+s)^{-1}\le\varepsilon$, since $s\le1+s$ gives $s(1+s)^{-1}\le1$ on multiplying by the positive $(1+s)^{-1}$, and multiplying that by the nonnegative $\varepsilon$ preserves the inequality; so

$$-\tfrac{\alpha}{2}\rho(\sigma,\tau)^{2}\le-\tfrac{\alpha}{2}s^{2}+\varepsilon .$$

Conclusion of claim 1. Let $r$ be the least of $r_{1},r_{2},r_{3},r_{4},\varepsilon_{0}$, obtained by repeated use of claim 9 of Elementary Order Arithmetic in an Ordered Field; it is positive. Since $(\mu_{m_{j}})_{j}$ converges to $\hat{\mu}$ and $(\nu_{m_{j}})_{j}$ converges to $\hat{\nu}$, and since $(\Phi_{\alpha}(\mu_{m_{j}},\nu_{m_{j}}))_{j}$ converges to $M$, given a positive $\varepsilon''$ there are three thresholds in $\mathbb{N}$ beyond which the corresponding distances are smaller than $r$, $r$ and $\varepsilon''$; let $J$ be the largest of the three, which exists by the trichotomy of the order of $\mathbb{N}$ (claim 3 of Properties of the Order on the Natural Numbers), and let $j\in\mathbb{N}$ satisfy $J\le j$. Then $\rho(\hat{\mu},\mu_{m_{j}})<r$ and $\rho(\hat{\nu},\nu_{m_{j}})<r$, so all five estimates apply with $\sigma=\mu_{m_{j}}$ and $\tau=\nu_{m_{j}}$, and adding them gives

$$\Phi_{\alpha}(\mu_{m_{j}},\nu_{m_{j}})\le\Phi_{\alpha}(\hat{\mu},\hat{\nu})+5\varepsilon .$$

Also $d_{\mathbb{R}}(\Phi_{\alpha}(\mu_{m_{j}},\nu_{m_{j}}),M)<\varepsilon''$, so $M<\Phi_{\alpha}(\mu_{m_{j}},\nu_{m_{j}})+\varepsilon''$ by claim 3 of Properties of the Absolute Value in an Ordered Field, and therefore $M\le\Phi_{\alpha}(\hat{\mu},\hat{\nu})+5\varepsilon+\varepsilon''$. As $\varepsilon''$ was an arbitrary positive number, Comparison of Real Numbers with Arbitrary Positive Slack (i) gives $M\le\Phi_{\alpha}(\hat{\mu},\hat{\nu})+5\varepsilon$. Given now an arbitrary positive $\varepsilon'''$, taking $\varepsilon=\varepsilon'''\iota(5)^{-1}$, positive by claims 3 and 7 quoted above, gives $5\varepsilon=\varepsilon'''$ and hence $M\le\Phi_{\alpha}(\hat{\mu},\hat{\nu})+\varepsilon'''$; so $M\le\Phi_{\alpha}(\hat{\mu},\hat{\nu})$ by Comparison of Real Numbers with Arbitrary Positive Slack (i) once more.

Since $K\subseteq\mathcal{D}$, the pair $(\hat{\mu},\hat{\nu})$ lies in $\mathcal{D}\times\mathcal{D}$, so $\Phi_{\alpha}(\hat{\mu},\hat{\nu})\le M$ as well. Hence $\Phi_{\alpha}(\mu,\nu)\le M=\Phi_{\alpha}(\hat{\mu},\hat{\nu})$ for every $(\mu,\nu)\in\mathcal{D}\times\mathcal{D}$, which is claim 1.

Claim 4. Let $L$ be nonnegative, let $\alpha$ be positive and let $(\hat{\mu},\hat{\nu})$ be a maximiser as stated; write $s=\rho(\hat{\mu},\hat{\nu})$. The pairs $(\hat{\mu},\hat{\mu})$ and $(\hat{\nu},\hat{\nu})$ lie in $\mathcal{D}\times\mathcal{D}$, so $\Phi_{\alpha}(\hat{\mu},\hat{\mu})\le\Phi_{\alpha}(\hat{\mu},\hat{\nu})$ and $\Phi_{\alpha}(\hat{\nu},\hat{\nu})\le\Phi_{\alpha}(\hat{\mu},\hat{\nu})$. Using the displayed form of $\Phi_{\alpha}$ and $\rho(\hat{\mu},\hat{\mu})=\rho(\hat{\nu},\hat{\nu})=0$, these read

$$u(\hat{\mu})-v(\hat{\mu})-2\delta\mathcal{E}(\hat{\mu})\le u(\hat{\mu})-v(\hat{\nu})-\delta\mathcal{E}(\hat{\mu})-\delta\mathcal{E}(\hat{\nu})-\tfrac{\alpha}{2}s^{2},$$ $$u(\hat{\nu})-v(\hat{\nu})-2\delta\mathcal{E}(\hat{\nu})\le u(\hat{\mu})-v(\hat{\nu})-\delta\mathcal{E}(\hat{\mu})-\delta\mathcal{E}(\hat{\nu})-\tfrac{\alpha}{2}s^{2}.$$

Adding them, the terms $-2\delta\mathcal{E}(\hat{\mu})-2\delta\mathcal{E}(\hat{\nu})$ occur on both sides and cancel, leaving

$$u(\hat{\mu})-v(\hat{\mu})+u(\hat{\nu})-v(\hat{\nu})\le2u(\hat{\mu})-2v(\hat{\nu})-\alpha s^{2},$$

that is, $\alpha s^{2}\le\bigl(u(\hat{\mu})-u(\hat{\nu})\bigr)+\bigl(v(\hat{\mu})-v(\hat{\nu})\bigr)$. By claim 3 of Properties of the Absolute Value in an Ordered Field and the two hypotheses, $u(\hat{\mu})-u(\hat{\nu})\le|u(\hat{\mu})-u(\hat{\nu})|\le Ls$ and $v(\hat{\mu})-v(\hat{\nu})\le|v(\hat{\mu})-v(\hat{\nu})|\le Ls$, so

$$\alpha s^{2}\le2Ls .$$

If $s=0$ then $\alpha s=0\le2L$, since $L$ is nonnegative and $2L$ is then nonnegative as well. Otherwise $s$ is positive, being nonnegative and different from $0$, and multiplying by the positive $s^{-1}$ (claim 7 of Elementary Order Arithmetic in an Ordered Field and claim 5 of Elementary Arithmetic in an Ordered Field) and using $s^{2}s^{-1}=s$ gives $\alpha s\le2L$. In both cases

$$\alpha s\le2L .$$

Both sides are nonnegative, so claim 2 of Monotonicity of Squaring on the Nonnegative Elements of an Ordered Field gives $\alpha^{2}s^{2}\le4L^{2}$, and multiplying by the positive $\alpha^{-1}$ gives $\alpha s^{2}\le4L^{2}\alpha^{-1}$. This is claim 4.