Proof of Existence, Penalty Bounds and Monotonicity of the Maximum of a Two-Space Doubled Difference with a Continuous Link

Each result cited is universally quantified over the data in its own statement. For $i\in\{1,2\}$, the results on penalty pairs used below, namely The Delta-Envelopes of Bounded Functions and Their Monotonicity in the Weight, for a Wasserstein-Coercive Penalty Pair, Basic Properties of the Delta-Envelopes on the Wasserstein Space and Wasserstein-Coercive Penalty Pairs (as well as Basic Properties of a Wasserstein-Coercive Penalty Pair, through which the statement provides $e_{0}$), adopt The Intrinsic Calculus on the Wasserstein Space: Standing Notation or a setting on which it is layered, so each of them is applied to the $i$-th pair at the dimension $n_{i}$: directly when $n_{i}=d$, and at the configuration level when $n_{i}=dN$. We write $W$ for $W_{2}$ on $\mathcal{P}_{2}(\mathbb{R}^{n_{1}})$ and on $\mathcal{P}_{2}(\mathbb{R}^{n_{2}})$, a metric by The Quadratic Wasserstein Distance is a Metric on the Wasserstein Space §metric (Wasserstein Spaces, Random Vectors, Vector Fields and Symmetric Matrices in Every Dimension: Standing Notation §dimensions with $m=n_{i}$), so that it is symmetric and nonnegative (Metric Space). $1/n$ is the multiplicative inverse of the positive real attached to $n\in\mathbb{N}$ (The Real Numbers: Standing Notation and Background §numbers); it satisfies $0<1/n\le1$ (claim 2 of Properties of the Canonical Map from the Natural Numbers to an Ordered Field and Order Reversal under Reciprocals, and Summability of the Reciprocals of the Squares §reciprocal), and the sequence $(1/n)_{n\in\mathbb{N}}$ converges to $0$: given a positive $\varepsilon$, claim 3 of The Archimedean Property of the Real Numbers provides $N\in\mathbb{N}$ with $1/N<\varepsilon$, and for $n\ge N$ the real attached to $n$ is at least the one attached to $N$ (claims 3 and 6 of Properties of the Canonical Map from the Natural Numbers to an Ordered Field), so $|1/n-0|=1/n\le1/N<\varepsilon$ by Order Reversal under Reciprocals, and Summability of the Reciprocals of the Squares §reciprocal. Fix positive $\delta,\alpha\in\mathbb{R}$. Each $\mathcal{D}_{i}$ contains the nonempty $\mathcal{D}_{i,\Sigma}$ (Penalty Pairs on the Wasserstein Space: the Penalty, Its Score, and Their Domains §pair, Penalty Pairs on the Wasserstein Space: the Penalty, Its Score, and Their Domains §nonempty), so $\mathcal{D}_{1}\times\mathcal{D}_{2}$ is nonempty. The products $\kappa_{1}\delta$ and $\kappa_{2}\delta$ are positive (claim 5 of Elementary Order Arithmetic in an Ordered Field), so $\kappa_{i}\delta e_{0}\le\kappa_{i}\delta\,\mathcal{E}_{i}(\sigma)$ for $\sigma\in\mathcal{D}_{i}$ (claim 5 of Elementary Arithmetic in an Ordered Field). Adding weak inequalities is justified by claims 2 and 3 of Elementary Arithmetic in an Ordered Field throughout.

Step 0 (Envelope bounds). By The Delta-Envelopes of Bounded Functions and Their Monotonicity in the Weight, for a Wasserstein-Coercive Penalty Pair §growth, $u$ has penalty-subordinate growth from above relative to the first pair and $v$ from below relative to the second. By Basic Properties of the Delta-Envelopes on the Wasserstein Space §semicontinuity, $u^{-}_{\delta}$ is upper semicontinuous on $\mathcal{D}_{1}$ relative to $\mathcal{D}_{1}$ in $(\mathcal{P}_{2}(\mathbb{R}^{n_{1}}),W)$ and $v^{+}_{\delta}$ is lower semicontinuous on $\mathcal{D}_{2}$ relative to $\mathcal{D}_{2}$ in $(\mathcal{P}_{2}(\mathbb{R}^{n_{2}}),W)$. By The Delta-Envelopes of Bounded Functions and Their Monotonicity in the Weight, for a Wasserstein-Coercive Penalty Pair §bounded,

Let $(\mu,\nu)\in\mathcal{D}_{1}\times\mathcal{D}_{2}$. Multiplying (0a) by the positive $\kappa_{1}$ and $\kappa_{2}$ (claim 5 of Elementary Arithmetic in an Ordered Field) and reversing the sign of the second (claim 4 of Elementary Order Arithmetic in an Ordered Field) gives $\kappa_{1}u^{-}_{\delta}(\mu)\le\kappa_{1}b-\kappa_{1}\delta\,\mathcal{E}_{1}(\mu)$ and $-\kappa_{2}v^{+}_{\delta}(\nu)\le-\kappa_{2}b'-\kappa_{2}\delta\,\mathcal{E}_{2}(\nu)$; and $0=\alpha\cdot0\le\alpha L(\mu,\nu)$ (claim 5 of Elementary Arithmetic in an Ordered Field, as $0\le L(\mu,\nu)$), so $-\alpha L(\mu,\nu)\le0$. Adding, and then using $\kappa_{i}\delta e_{0}\le\kappa_{i}\delta\,\mathcal{E}_{i}$ and distributivity,

Step 1 (Claim 1: the supremum and a maximising sequence). By (0b) the nonempty set of values of $\Psi_{\delta,\alpha}$ is bounded above by $\kappa_{1}b-\kappa_{2}b'-(\kappa_{1}+\kappa_{2})\delta e_{0}$, so it has a least upper bound $M=M(\delta,\alpha)\in\mathbb{R}$. For each $n\in\mathbb{N}$, claim 3 of Approximation Property of the Supremum and the Infimum in $\mathbb{R}$ with the positive $1/n$ shows that the set of $(\mu,\nu)\in\mathcal{D}_{1}\times\mathcal{D}_{2}$ with $M-1/n<\Psi_{\delta,\alpha}(\mu,\nu)$ is nonempty; by Axiom of Countable Choice choose such a pair $(\mu_{n},\nu_{n})$ for every $n$. As $1/n\le1$, $M-1<\Psi_{\delta,\alpha}(\mu_{n},\nu_{n})$ (claims 2 and 4 of Elementary Order Arithmetic in an Ordered Field). By the first inequality of (0b) and $\kappa_{2}\delta e_{0}\le\kappa_{2}\delta\,\mathcal{E}_{2}(\nu_{n})$,

so, by claim 1 of Elementary Order Arithmetic in an Ordered Field, $\kappa_{1}\delta\,\mathcal{E}_{1}(\mu_{n})<A_{1}$ with $A_{1}=\kappa_{1}b-\kappa_{2}b'-\kappa_{2}\delta e_{0}-M+1$; multiplying by the positive $(\kappa_{1}\delta)^{-1}$ (claims 7 and 10 of that lemma) gives $\mathcal{E}_{1}(\mu_{n})<c_{1}$ with $c_{1}=(\kappa_{1}\delta)^{-1}A_{1}$. In the same way, from $\kappa_{1}\delta e_{0}\le\kappa_{1}\delta\,\mathcal{E}_{1}(\mu_{n})$, we get $\kappa_{2}\delta\,\mathcal{E}_{2}(\nu_{n})<A_{2}$ with $A_{2}=\kappa_{1}b-\kappa_{2}b'-\kappa_{1}\delta e_{0}-M+1$, and $\mathcal{E}_{2}(\nu_{n})<c_{2}$ with $c_{2}=(\kappa_{2}\delta)^{-1}A_{2}$. The constants $c_{1},c_{2}$ do not depend on $n$.

Step 2 (Claim 1: a jointly convergent subsequence). The sets $K_{1}=\{\sigma\in\mathcal{D}_{1}:\mathcal{E}_{1}(\sigma)\le c_{1}\}$ and $K_{2}=\{\sigma\in\mathcal{D}_{2}:\mathcal{E}_{2}(\sigma)\le c_{2}\}$ are sequentially compact in $(\mathcal{P}_{2}(\mathbb{R}^{n_{1}}),W)$ and in $(\mathcal{P}_{2}(\mathbb{R}^{n_{2}}),W)$ by Wasserstein-Coercive Penalty Pairs §coercive, and $\mu_{n}\in K_{1}$, $\nu_{n}\in K_{2}$ for every $n$ by Step 1. So there are $\hat{\mu}\in K_{1}$ and a strictly increasing $(n_{k})_{k\in\mathbb{N}}$ with $(\mu_{n_{k}})_{k}$ converging to $\hat{\mu}$; the sequence $(\nu_{n_{k}})_{k}$ lies in $K_{2}$, so there are $\hat{\nu}\in K_{2}$ and a strictly increasing $(k_{j})_{j\in\mathbb{N}}$ with $(\nu_{n_{k_{j}}})_{j}$ converging to $\hat{\nu}$. Put $m_{j}=n_{k_{j}}$; $(m_{j})_{j}$ is strictly increasing and $(\mu_{m_{j}})_{j}$, $(\nu_{m_{j}})_{j}$, $(1/m_{j})_{j}$ are subsequences of $(\mu_{n})_{n}$, $(\nu_{n})_{n}$, $(1/n)_{n}$ (claims 2 and 3 of A Subsequence of a Subsequence is a Subsequence), and $(\mu_{m_{j}})_{j}$ is the subsequence of $(\mu_{n_{k}})_{k}$ determined by $(k_{j})_{j}$. By A Subsequence of a Convergent Sequence Has the Same Limit, $(\mu_{m_{j}})_{j}$ converges to $\hat{\mu}$, and $(1/m_{j})_{j}$ converges to $0$ in $(\mathbb{R},d_{\mathbb{R}})$, hence in the sense of Limit of a Sequence of Real Numbers (The Real Numbers: Standing Notation and Background §sequences). Also $\hat{\mu}\in K_{1}\subseteq\mathcal{D}_{1}$ and $\hat{\nu}\in K_{2}\subseteq\mathcal{D}_{2}$.

The real sequence $(W(\mu_{m_{j}},\hat{\mu}))_{j}$ converges to $0$: given a positive $\varepsilon$, the convergence in $(\mathcal{P}_{2}(\mathbb{R}^{n_{1}}),W)$ gives $N$ with $|W(\mu_{m_{j}},\hat{\mu})-0|=W(\mu_{m_{j}},\hat{\mu})<\varepsilon$ for $j\ge N$, $W$ being nonnegative (Absolute Value in an Ordered Field); likewise $(W(\nu_{m_{j}},\hat{\nu}))_{j}$ converges to $0$. Put $L_{j}=L(\mu_{m_{j}},\nu_{m_{j}})$ and $\hat{L}=L(\hat{\mu},\hat{\nu})$. By the continuity hypothesis on $L$, $(L_{j})_{j}$ converges to $\hat{L}$; since $\bigl||L_{j}-\hat{L}|-0\bigr|=|L_{j}-\hat{L}|$ (claim 1 of Properties of the Absolute Value in an Ordered Field), $(|L_{j}-\hat{L}|)_{j}$ converges to $0$ directly by Limit of a Sequence of Real Numbers. So, by claim 1 of Arithmetic of Limits of Real Sequences applied three times, the sequence of nonnegative reals

converges to $0$, and each of its four summands is at most $a_{j}$, the other three being nonnegative.

Step 3 (Claim 1: attainment). Let $\varepsilon\in\mathbb{R}$ be positive. The choices are made in this order: first $\varepsilon_{1}$, then $r_{1}$ and $r_{2}$, then $t$, then the index $j$. Put $\varepsilon_{1}=\varepsilon\cdot4^{-1}$, where $4=2+2$ is positive; $\varepsilon_{1}$ is positive and $4\varepsilon_{1}=\varepsilon$ (claims 1, 5, 7 and 8 of Elementary Order Arithmetic in an Ordered Field). The reals $\kappa_{1}^{-1}\varepsilon_{1}$, $\kappa_{2}^{-1}\varepsilon_{1}$ and $\alpha^{-1}\varepsilon_{1}$ are positive (claims 5 and 7 of that lemma). By the upper semicontinuity of $u^{-}_{\delta}$ at $\hat{\mu}$ relative to $\mathcal{D}_{1}$ (Upper Semicontinuous Function on a Subset of a Metric Space) there is a positive $r_{1}$ such that every $\mu'\in\mathcal{D}_{1}$ with $W(\hat{\mu},\mu')<r_{1}$ satisfies $u^{-}_{\delta}(\mu')<u^{-}_{\delta}(\hat{\mu})+\kappa_{1}^{-1}\varepsilon_{1}$, hence, multiplying by $\kappa_{1}$ (claim 10 of Elementary Order Arithmetic in an Ordered Field),

by the lower semicontinuity of $v^{+}_{\delta}$ at $\hat{\nu}$ relative to $\mathcal{D}_{2}$ (Lower Semicontinuous Function on a Subset of a Metric Space) there is a positive $r_{2}$ such that every $\nu'\in\mathcal{D}_{2}$ with $W(\hat{\nu},\nu')<r_{2}$ satisfies $v^{+}_{\delta}(\hat{\nu})-\kappa_{2}^{-1}\varepsilon_{1}<v^{+}_{\delta}(\nu')$, hence $\kappa_{2}v^{+}_{\delta}(\hat{\nu})-\varepsilon_{1}<\kappa_{2}v^{+}_{\delta}(\nu')$ in the same way. Let $t$ be the least of $r_{1}$, $r_{2}$, $\alpha^{-1}\varepsilon_{1}$ and $\varepsilon_{1}$, obtained by claim 9 of Elementary Order Arithmetic in an Ordered Field applied three times; it is one of these four positive reals, hence positive. As $(a_{j})_{j}$ converges to $0$, there is $j\in\mathbb{N}$ with $a_{j}=|a_{j}-0|<t$; fix it. By Step 2 and claim 2 of Elementary Order Arithmetic in an Ordered Field, and the symmetry of $W$,

So, with $\mu'=\mu_{m_{j}}\in\mathcal{D}_{1}$ and $\nu'=\nu_{m_{j}}\in\mathcal{D}_{2}$, the two semicontinuity bounds give $\kappa_{1}u^{-}_{\delta}(\mu_{m_{j}})<\kappa_{1}u^{-}_{\delta}(\hat{\mu})+\varepsilon_{1}$ and, by sign reversal (claim 4 of Elementary Order Arithmetic in an Ordered Field), $-\kappa_{2}v^{+}_{\delta}(\nu_{m_{j}})<-\kappa_{2}v^{+}_{\delta}(\hat{\nu})+\varepsilon_{1}$; and $\hat{L}-L_{j}\le|L_{j}-\hat{L}|<\alpha^{-1}\varepsilon_{1}$ (claims 2 and 3 of Properties of the Absolute Value in an Ordered Field) gives, multiplying by $\alpha$ and reversing the sign, $-\alpha L_{j}<-\alpha\hat{L}+\varepsilon_{1}$. Adding these three strict inequalities (claim 3 of Elementary Order Arithmetic in an Ordered Field, twice),

By the choice of the pair with index $m_{j}$ in Step 1, $M-1/m_{j}<\Psi_{\delta,\alpha}(\mu_{m_{j}},\nu_{m_{j}})$, and $1/m_{j}<\varepsilon_{1}$; so $M<\Psi_{\delta,\alpha}(\hat{\mu},\hat{\nu})+4\varepsilon_{1}=\Psi_{\delta,\alpha}(\hat{\mu},\hat{\nu})+\varepsilon$ (claims 1, 2 and 3 of Elementary Order Arithmetic in an Ordered Field). As $\varepsilon$ was an arbitrary positive real, $M\le\Psi_{\delta,\alpha}(\hat{\mu},\hat{\nu})$ by Comparison of Real Numbers with Arbitrary Positive Slack §slack-above; and $\Psi_{\delta,\alpha}(\hat{\mu},\hat{\nu})\le M$, $M$ being an upper bound of the values. So $\Psi_{\delta,\alpha}(\hat{\mu},\hat{\nu})=M$, with $(\hat{\mu},\hat{\nu})\in\mathcal{D}_{1}\times\mathcal{D}_{2}$, which proves claim 1.

Step 4 (Claim 2). Let $(\mu,\nu)\in\mathcal{D}_{1}\times\mathcal{D}_{2}$ with $0\le\Psi_{\delta,\alpha}(\mu,\nu)$. By the first inequality of (0b), $0\le\kappa_{1}b-\kappa_{2}b'-\kappa_{1}\delta\,\mathcal{E}_{1}(\mu)-\kappa_{2}\delta\,\mathcal{E}_{2}(\nu)$, so by claim 3 of Elementary Arithmetic in an Ordered Field and $\kappa_{i}\delta e_{0}\le\kappa_{i}\delta\,\mathcal{E}_{i}$ (with claim 4 of Elementary Order Arithmetic in an Ordered Field),

By claims 2 and 3 of Properties of the Absolute Value in an Ordered Field, $b\le|b|$, $-b'\le|-b'|=|b'|$ and $-e_{0}\le|e_{0}|$; multiplying by the positive $\kappa_{1}$, $\kappa_{2}$ and $\kappa_{i}\delta$ (claim 5 of Elementary Arithmetic in an Ordered Field) and adding, the right-hand sides are at most $\kappa_{1}|b|+\kappa_{2}|b'|+\kappa_{2}\delta|e_{0}|$ and $\kappa_{1}|b|+\kappa_{2}|b'|+\kappa_{1}\delta|e_{0}|$ respectively. By distributivity $B_{\delta}=\kappa_{1}|b|+\kappa_{2}|b'|+\kappa_{1}\delta|e_{0}|+\kappa_{2}\delta|e_{0}|$, and each of its four summands is nonnegative (claim 1 of Properties of the Absolute Value in an Ordered Field, claim 5 of Elementary Arithmetic in an Ordered Field); so both right-hand sides are at most $B_{\delta}$, and $\kappa_{1}\delta\,\mathcal{E}_{1}(\mu)\le B_{\delta}$, $\kappa_{2}\delta\,\mathcal{E}_{2}(\nu)\le B_{\delta}$. For the lower bounds, $-|e_{0}|\le e_{0}$ (claim 3 of Properties of the Absolute Value in an Ordered Field) gives $-\kappa_{i}\delta|e_{0}|\le\kappa_{i}\delta e_{0}\le\kappa_{i}\delta\,\mathcal{E}_{i}$, and $\kappa_{i}\delta|e_{0}|\le B_{\delta}$ gives $-B_{\delta}\le-\kappa_{i}\delta|e_{0}|$ (claim 4 of Elementary Order Arithmetic in an Ordered Field); so $-B_{\delta}\le\kappa_{1}\delta\,\mathcal{E}_{1}(\mu)$ and $-B_{\delta}\le\kappa_{2}\delta\,\mathcal{E}_{2}(\nu)$. By claim 6 of Properties of the Absolute Value in an Ordered Field, $|\kappa_{1}\delta\,\mathcal{E}_{1}(\mu)|\le B_{\delta}$ and $|\kappa_{2}\delta\,\mathcal{E}_{2}(\nu)|\le B_{\delta}$, and by its claim 4, $|\kappa_{i}\delta\,\mathcal{E}_{i}|=|\kappa_{i}\delta|\,|\mathcal{E}_{i}|=\kappa_{i}\delta\,|\mathcal{E}_{i}|$, as $\kappa_{i}\delta$ is positive (Absolute Value in an Ordered Field). This is claim 2.

Step 5 (Claim 3). Let $0<\delta'<\delta$ and let $(\hat{\mu},\hat{\nu})$ be a maximising pair of $\Psi_{\delta,\alpha}$; by Step 1 applied to $(\delta',\alpha)$, $M(\delta',\alpha)$ is a real number and an upper bound of the values of $\Psi_{\delta',\alpha}$. By The Delta-Envelopes of Bounded Functions and Their Monotonicity in the Weight, for a Wasserstein-Coercive Penalty Pair §monotone, read with $\delta'$ and $\delta$ in the roles of the smaller and the larger weight there, and applied to the first pair and to the second,

Multiplying by the positive $\kappa_{1}$ and $\kappa_{2}$ (claim 5 of Elementary Arithmetic in an Ordered Field), reversing the sign of the second (claim 4 of Elementary Order Arithmetic in an Ordered Field) and adding $-\alpha L(\hat{\mu},\hat{\nu})$,

the middle step by distributivity. This is claim 3.

Step 6 (Claim 4). Let $0<\alpha'<\alpha$ and let $(\hat{\mu},\hat{\nu})$ be a maximising pair of $\Psi_{\delta,\alpha}$; by Step 1 applied to $(\delta,\alpha')$, $M(\delta,\alpha')$ is an upper bound of the values of $\Psi_{\delta,\alpha'}$. Since $-\alpha'L(\hat{\mu},\hat{\nu})=-\alpha L(\hat{\mu},\hat{\nu})+(\alpha-\alpha')L(\hat{\mu},\hat{\nu})$ by distributivity,

which is claim 4.