Proof of Existence, Penalty Bounds and Monotonicity of the Maximum of the Wasserstein-Doubled Difference of Bounded Functions

Each result cited is universally quantified over the data in its own statement, and is applied to the data named here. We write $W$ for $W_{2}$, which is a metric on $\mathcal{P}_{2}(\mathbb{R}^{d})$ by The Quadratic Wasserstein Distance is a Metric on the Wasserstein Space §metric, so that it is symmetric, vanishes on the diagonal and satisfies the triangle inequality (Metric Space); $1/n$ is the multiplicative inverse of the positive real attached to $n\in\mathbb{N}$ (The Real Numbers: Standing Notation and Background §numbers), and the sequence $(1/n)_{n\in\mathbb{N}}$ converges to $0$: given a positive $\varepsilon$, The Archimedean Property of the Real Numbers provides $N\in\mathbb{N}$ with $1/N<\varepsilon$, and for $n\ge N$ the real attached to $n$ is at least the positive real attached to $N$ (claims 3 and 6 of Properties of the Canonical Map from the Natural Numbers to an Ordered Field), so $0<1/n\le1/N<\varepsilon$ by Order Reversal under Reciprocals, and Summability of the Reciprocals of the Squares §reciprocal. Fix positive $\delta,\alpha$. The set $\mathcal{D}$ contains the nonempty $\mathcal{D}_{\Sigma}$ (Penalty Pairs on the Wasserstein Space: the Penalty, Its Score, and Their Domains §pair, Penalty Pairs on the Wasserstein Space: the Penalty, Its Score, and Their Domains §nonempty), so $\mathcal{D}\times\mathcal{D}$ is nonempty. As in the statement, $\delta e_{0}\le\delta\,\mathcal{E}(\sigma)$ for every $\sigma\in\mathcal{D}$ (claim 5 of Elementary Arithmetic in an Ordered Field).

Step 0 (Envelope bounds). By Basic Properties of the Delta-Envelopes on the Wasserstein Space §semicontinuity, $u^{-}_{\delta}$ is upper and $v^{+}_{\delta}$ lower semicontinuous on $\mathcal{D}$ relative to $\mathcal{D}$ in $(\mathcal{P}_{2}(\mathbb{R}^{d}),W)$, and $u(\sigma)-\delta\,\mathcal{E}(\sigma)\le u^{-}_{\delta}(\sigma)$ and $v^{+}_{\delta}(\sigma)\le v(\sigma)+\delta\,\mathcal{E}(\sigma)$ for $\sigma\in\mathcal{D}$. By The Delta-Envelopes of Bounded Functions and Their Monotonicity in the Weight, for a Wasserstein-Coercive Penalty Pair §bounded,

Since $0\le\tfrac{\alpha}{2}W(\mu,\nu)^{2}$ (claim 5 of Elementary Arithmetic in an Ordered Field and claim 2 of Nonnegativity of Squares in an Ordered Field), (0a) gives, for $(\mu,\nu)\in\mathcal{D}\times\mathcal{D}$,

Claim 1. By (0b) the nonempty set of values of $\Psi_{\delta,\alpha}$ is bounded above by $b-b'-2\delta e_{0}$, so it has a least upper bound $M=M(\delta,\alpha)\in\mathbb{R}$. For each $n\in\mathbb{N}$, claim 3 of Approximation Property of the Supremum and the Infimum in $\mathbb{R}$ provides a pair with value above $M-1/n$; by Axiom of Countable Choice choose such a pair $(\mu_{n},\nu_{n})$ for every $n$. As $1/n\le1$, $M-1<\Psi_{\delta,\alpha}(\mu_{n},\nu_{n})$, so by the first inequality of (0b) and $\delta e_{0}\le\delta\,\mathcal{E}(\nu_{n})$, $\delta\,\mathcal{E}(\mu_{n})<b-b'-\delta e_{0}-M+1$; multiplying by the positive $\delta^{-1}$ (claim 7 of Elementary Order Arithmetic in an Ordered Field) gives $\mathcal{E}(\mu_{n})\le c$ with $c=\delta^{-1}(b-b'-M+1)-e_{0}$, and in the same way $\mathcal{E}(\nu_{n})\le c$. The set $K=\{\sigma\in\mathcal{D}:\mathcal{E}(\sigma)\le c\}$ is sequentially compact by Wasserstein-Coercive Penalty Pairs §coercive. So $(\mu_{n})_{n}$ has a subsequence converging to some $\hat{\mu}\in K$; the corresponding subsequence of $(\nu_{n})_{n}$, a sequence in $K$, has in turn a subsequence converging to some $\hat{\nu}\in K$, along which the $\mu$-terms still converge to $\hat{\mu}$ (A Subsequence of a Convergent Sequence Has the Same Limit), the indices forming a subsequence of the indices of $\mathbb{N}$ (A Subsequence of a Subsequence is a Subsequence). Write $(\mu_{m_{k}},\nu_{m_{k}})_{k\in\mathbb{N}}$ for this subsequence, with $(m_{k})_{k}$ strictly increasing; then $\hat{\mu},\hat{\nu}\in\mathcal{D}$, $W(\mu_{m_{k}},\hat{\mu})\to0$, $W(\nu_{m_{k}},\hat{\nu})\to0$, and $1/m_{k}\to0$ by A Subsequence of a Convergent Sequence Has the Same Limit.

Put $a=W(\hat{\mu},\hat{\nu})$, $a_{k}=W(\mu_{m_{k}},\nu_{m_{k}})$ and $s_{k}=W(\mu_{m_{k}},\hat{\mu})+W(\nu_{m_{k}},\hat{\nu})$, all nonnegative. The triangle inequality, used twice with the symmetry of $W$, gives $a\le a_{k}+s_{k}$ and $a_{k}\le a+s_{k}$. Squares being monotone on nonnegative reals (claim 2 of Monotonicity of Squaring on the Nonnegative Elements of an Ordered Field) and by the square of a sum (claim 5 of Zero Products and Elementary Identities in a Field),

the second step by $a_{k}\le a+s_{k}$ and claim 5 of Elementary Arithmetic in an Ordered Field.

Let $\varepsilon\in\mathbb{R}$ be positive. By the upper semicontinuity of $u^{-}_{\delta}$ at $\hat{\mu}$ (Upper Semicontinuous Function on a Subset of a Metric Space) there is a positive $r_{1}$ with $u^{-}_{\delta}(\mu')<u^{-}_{\delta}(\hat{\mu})+\varepsilon$ for $\mu'\in\mathcal{D}$ with $W(\mu',\hat{\mu})<r_{1}$, and by the lower semicontinuity of $v^{+}_{\delta}$ at $\hat{\nu}$ (Lower Semicontinuous Function on a Subset of a Metric Space) a positive $r_{2}$ with $v^{+}_{\delta}(\hat{\nu})-\varepsilon<v^{+}_{\delta}(\nu')$ for $\nu'\in\mathcal{D}$ with $W(\nu',\hat{\nu})<r_{2}$. The number $\alpha(2a+3)$ is positive (claims 1 and 5 of Elementary Order Arithmetic in an Ordered Field), so it has a positive inverse (claim 7 of that lemma). Let $t$ be the least of $\tfrac12$, $r_{1}$, $r_{2}$ and $\varepsilon\,\bigl(\alpha(2a+3)\bigr)^{-1}$, obtained by applying claim 9 of Elementary Order Arithmetic in an Ordered Field three times, a positive number. Fix $k$ with $W(\mu_{m_{k}},\hat{\mu})<t$, $W(\nu_{m_{k}},\hat{\nu})<t$ and $1/m_{k}<\varepsilon$. Then $s_{k}<2t\le1$, so $s_{k}(2a+3s_{k})\le s_{k}(2a+3)<2t(2a+3)\le2\varepsilon\alpha^{-1}$, and the last display gives $\tfrac{\alpha}{2}a^{2}<\tfrac{\alpha}{2}a_{k}^{2}+\varepsilon$. Together with the two semicontinuity bounds at $\mu'=\mu_{m_{k}}$ and $\nu'=\nu_{m_{k}}$,

(claim 3 of Elementary Order Arithmetic in an Ordered Field). As $\varepsilon$ was arbitrary, $M\le\Psi_{\delta,\alpha}(\hat{\mu},\hat{\nu})$ by Comparison of Real Numbers with Arbitrary Positive Slack §slack-above; and $\Psi_{\delta,\alpha}(\hat{\mu},\hat{\nu})\le M$, $M$ being an upper bound. So $\Psi_{\delta,\alpha}(\hat{\mu},\hat{\nu})=M$.

Claim 2. Let $0\le\Psi_{\delta,\alpha}(\mu,\nu)$. By (0b), $\delta\,\mathcal{E}(\mu)\le b-b'-\delta\,\mathcal{E}(\nu)\le b-b'-\delta e_{0}\le|b|+|b'|+\delta|e_{0}|$, the last step by claim 3 of Properties of the Absolute Value in an Ordered Field applied to $b$, $-b'$ and $-\delta e_{0}$, with $|-b'|=|b'|$ and $|-\delta e_{0}|=\delta|e_{0}|$ (claims 2 and 4 of that lemma, $\delta$ being positive). Also $-(|b|+|b'|+\delta|e_{0}|)\le-\delta|e_{0}|\le\delta e_{0}\le\delta\,\mathcal{E}(\mu)$, by claims 1 and 3 of that lemma. So $|\delta\,\mathcal{E}(\mu)|\le|b|+|b'|+\delta|e_{0}|$ by claim 6 of that lemma, and $|\delta\,\mathcal{E}(\mu)|=\delta\,|\mathcal{E}(\mu)|$ by its claim 4. The bound for $\nu$ follows by exchanging the roles of $\mu$ and $\nu$.

Claim 3. Let $0<\delta'<\delta$ and let $(\hat{\mu},\hat{\nu})$ be a maximising pair of $\Psi_{\delta,\alpha}$. By The Delta-Envelopes of Bounded Functions and Their Monotonicity in the Weight, for a Wasserstein-Coercive Penalty Pair §monotone, read with $\delta'$ and $\delta$ in the roles of the smaller and the larger weight there, $u^{-}_{\delta}(\hat{\mu})+(\delta-\delta')\,\mathcal{E}(\hat{\mu})\le u^{-}_{\delta'}(\hat{\mu})$ and $v^{+}_{\delta'}(\hat{\nu})\le v^{+}_{\delta}(\hat{\nu})-(\delta-\delta')\,\mathcal{E}(\hat{\nu})$. Hence

Claim 4. Let $0<\alpha'<\alpha$ and let $(\hat{\mu},\hat{\nu})$ be a maximising pair of $\Psi_{\delta,\alpha}$. Since $\tfrac{\alpha'}{2}=\tfrac{\alpha}{2}-\tfrac{\alpha-\alpha'}{2}$ by distributivity,