Proof of The Law of Large Numbers for Empirical Measures is Uniform on Wasserstein-Compact Sets

Each result cited is universally quantified over the data in its own statement.

Conventions. Fix $q\in\mathbb{N}$ as in the statement; $1\le q$ by Wasserstein Spaces, Random Vectors, Vector Fields and Symmetric Matrices in Every Dimension: Standing Notation, so $1\le qN$ for every $N\in\mathbb{N}$. For $N\in\mathbb{N}$, $\iota(N)$ denotes the image of $N$ under the canonical map $\iota:\mathbb{N}\to\mathbb{R}$ of The Canonical Map from the Natural Numbers to a Field, the real number written $N$ in the displayed formulas of the results cited; $0<\iota(N)$ and $0<\iota(N)^{-1}$ by claim 3 of Properties of the Canonical Map from the Natural Numbers to an Ordered Field. For $N\in\mathbb{N}$ and $\nu\in\mathcal{P}_{2}(\mathbb{R}^{q})$ put $\Phi^{N}_{\nu}(x)=W_{2}(\mu^{N}_{x},\nu)$ for $x\in\mathbb{R}^{qN}$. By Tensor Powers and One-Particle Marginals: Particle Laws, Product Integrals, Push-Forwards, Moments, Product Maps and Diagonal Shifts §moments, $\nu^{\otimes N}\in\mathcal{P}_{2}(\mathbb{R}^{qN})$. By Law of Large Numbers for Empirical Measures in the Wasserstein Distance §finite, $(\Phi^{N}_{\nu})^{2}$ is integrable with respect to $\nu^{\otimes N}$ and $\phi_{N}(\nu)$ is the nonnegative square root of $\int_{\mathbb{R}^{qN}}(\Phi^{N}_{\nu})^{2}\,d\nu^{\otimes N}$. For $Q\in\mathcal{P}(\mathbb{R}^{qN})$, $(\mathbb{R}^{qN},\mathcal{B}(\mathbb{R}^{qN}),Q)$ is a probability space (Probability Space, Event, and Random Variable), Borel maps $\mathbb{R}^{qN}\to\mathbb{R}$ (Probability Measures on Euclidean Space and Random Vectors: Standing Notation §borel-maps) are random variables on it, the expectation of a nonnegative one is its integral against $Q$ (Expectation, Variance, and Moments), and square-integrability and the norm $\lVert V\rVert_{2,Q}$, the nonnegative square root of $\int V^{2}\,dQ$, are those of Square-Integrable Random Variables and the Mean-Square Inner Product; for Borel $\Phi$ with $\Phi^{2}$ integrable against $Q$, the norm $\lVert\Phi\rVert_{Q}$ of Wasserstein Spaces, Random Vectors, Vector Fields and Symmetric Matrices in Every Dimension: Standing Notation §fields is the same number $\lVert\Phi\rVert_{2,Q}$, both being the nonnegative square root of $\int_{\mathbb{R}^{qN}}\Phi^{2}\,dQ$. In particular $\phi_{N}(\nu)=\lVert\Phi^{N}_{\nu}\rVert_{2,\nu^{\otimes N}}$. For nonnegative reals $a,b$ we use that $a\le b$ if and only if $a^{2}\le b^{2}$, and $a=b$ if and only if $a^{2}=b^{2}$ (claims 2 and 3 of Monotonicity of Squaring on the Nonnegative Elements of an Ordered Field); we refer to this as (S).

Step 1 (A Lipschitz constant in the configuration). Fix $N\in\mathbb{N}$ and $\nu\in\mathcal{P}_{2}(\mathbb{R}^{q})$. Let $L_{N}$ be the nonnegative square root of $\iota(N)^{-1}$ (Existence and Uniqueness of the Nonnegative Square Root), so $0\le L_{N}$ and $L_{N}^{2}=\iota(N)^{-1}$. Let $x,x'\in\mathbb{R}^{qN}$. By Basic Properties of Empirical Measures: Values, Integrals, Push-Forwards, Second Moment, and Lipschitz Dependence on the Configuration §distance, $|\Phi^{N}_{\nu}(x)-\Phi^{N}_{\nu}(x')|\le W_{2}(\mu^{N}_{x},\mu^{N}_{x'})$. By Basic Properties of Empirical Measures: Values, Integrals, Push-Forwards, Second Moment, and Lipschitz Dependence on the Configuration §lipschitz, $\iota(N)W_{2}(\mu^{N}_{x},\mu^{N}_{x'})^{2}\le\lVert x-x'\rVert^{2}$; multiplying by $\iota(N)^{-1}\ge0$ (claim 5 of Elementary Arithmetic in an Ordered Field) gives $W_{2}(\mu^{N}_{x},\mu^{N}_{x'})^{2}\le(L_{N}\lVert x-x'\rVert)^{2}$, hence $W_{2}(\mu^{N}_{x},\mu^{N}_{x'})\le L_{N}\lVert x-x'\rVert$ by (S), both sides being nonnegative (the right one by the second axiom of Ordered Field). Therefore $|\Phi^{N}_{\nu}(x)-\Phi^{N}_{\nu}(x')|\le L_{N}\lVert x-x'\rVert$. Since $\lVert x-x'\rVert$ is the Euclidean distance of Euclidean Distance on $\mathbb{R}^n$ and $|s-t|$ is the metric of The Absolute Value Metric on the Real Line, $\Phi^{N}_{\nu}$ is Lipschitz with constant $L_{N}$ in the sense of Lipschitz Map Between Metric Spaces.

Step 2 (Clause 1). Fix $N\in\mathbb{N}$ and $\nu,\nu'\in\mathcal{P}_{2}(\mathbb{R}^{q})$; write $w=W_{2}(\nu,\nu')$, $P=\nu^{\otimes N}$, $P'=\nu'^{\otimes N}$, $\Phi=\Phi^{N}_{\nu}$ and $\Phi'=\Phi^{N}_{\nu'}$. Both $P,P'$ lie in $\mathcal{P}_{2}(\mathbb{R}^{qN})$ by the conventions.

First term. By Step 1, The Mean-Square Norm of a Lipschitz Function Depends Lipschitz-Continuously on the Measure for the Wasserstein Distance §bound applies with $n=qN$, $L=L_{N}$, the function $\Phi$ and the measures $P,P'$: $\Phi$ is Borel, $\Phi^{2}$ is integrable against $P$ and $P'$, and $\bigl|\lVert\Phi\rVert_{P}-\lVert\Phi\rVert_{P'}\bigr|\le L_{N}W_{2}(P,P')$. By Tensor Powers Scale the Wasserstein Distance by the Square Root of N, and the One-Particle Marginal is Lipschitz with Constant N^{-1/2} §tensor, $W_{2}(P,P')^{2}=\iota(N)w^{2}$, so $(L_{N}W_{2}(P,P'))^{2}=\iota(N)^{-1}\iota(N)w^{2}=w^{2}$; both $L_{N}W_{2}(P,P')$ and $w$ are nonnegative, so $L_{N}W_{2}(P,P')=w$ by (S). Since $\lVert\Phi\rVert_{P}=\phi_{N}(\nu)$ by the conventions,

Second term. Let $x\in\mathbb{R}^{qN}$. By Basic Properties of Empirical Measures: Values, Integrals, Push-Forwards, Second Moment, and Lipschitz Dependence on the Configuration §moment, $\mu^{N}_{x}\in\mathcal{P}_{2}(\mathbb{R}^{q})$, so the triangle inequality and symmetry of The Quadratic Wasserstein Distance is a Metric on the Wasserstein Space §triangle and The Quadratic Wasserstein Distance is a Metric on the Wasserstein Space §symmetry give $\Phi(x)\le\Phi'(x)+W_{2}(\nu',\nu)=\Phi'(x)+w$ and $\Phi'(x)\le\Phi(x)+w$. Hence $-w\le\Phi(x)-\Phi'(x)\le w$, i.e. $|\Phi(x)-\Phi'(x)|\le w$ by claim 6 of Properties of the Absolute Value in an Ordered Field, and $(\Phi(x)-\Phi'(x))^{2}\le w^{2}$ by (S), since $(\Phi(x)-\Phi'(x))^{2}=|\Phi(x)-\Phi'(x)|^{2}$ (claim 1 of Properties of the Absolute Value in an Ordered Field and claim 2 of Zero Products and Elementary Identities in a Field). On the probability space $(\mathbb{R}^{qN},\mathcal{B}(\mathbb{R}^{qN}),P')$ the random variables $\Phi$ and $\Phi'$ are square-integrable (by The Mean-Square Norm of a Lipschitz Function Depends Lipschitz-Continuously on the Measure for the Wasserstein Distance §bound as just applied, and by Law of Large Numbers for Empirical Measures in the Wasserstein Distance §finite for $\nu'$), so $\Phi-\Phi'$ is square-integrable by Square-Integrable Random Variables and the Mean-Square Inner Product. The constant $w^{2}$ is a nonnegative simple function with integral $w^{2}P'(\mathbb{R}^{qN})=w^{2}$ (Simple Function and Its Integral, and the last sentence of Lebesgue Integral of a Nonnegative Measurable Function); by monotonicity of the integral (claim 1 of Linearity and Monotonicity of the Lebesgue Integral), $\lVert\Phi-\Phi'\rVert_{2,P'}^{2}\le w^{2}$, so $\lVert\Phi-\Phi'\rVert_{2,P'}\le w$ by (S). By claim 2 of Cauchy-Schwarz and Triangle Inequalities for the Mean-Square Norm, applied to $(\Phi-\Phi')+\Phi'$ and to $(\Phi'-\Phi)+\Phi$ (with $\lVert\Phi'-\Phi\rVert_{2,P'}=\lVert\Phi-\Phi'\rVert_{2,P'}$, as $(\Phi'-\Phi)^{2}=(\Phi-\Phi')^{2}$ pointwise by claim 2 of Zero Products and Elementary Identities in a Field),

so $\bigl|\lVert\Phi\rVert_{2,P'}-\lVert\Phi'\rVert_{2,P'}\bigr|\le w$ by claim 6 of Properties of the Absolute Value in an Ordered Field. As $\lVert\Phi'\rVert_{2,P'}=\phi_{N}(\nu')$ by the conventions, $\bigl|\lVert\Phi\rVert_{2,P'}-\phi_{N}(\nu')\bigr|\le w$.

Conclusion. By the triangle inequality (claim 5 of Properties of the Absolute Value in an Ordered Field) applied to $\phi_{N}(\nu)-\phi_{N}(\nu')=\bigl(\phi_{N}(\nu)-\lVert\Phi\rVert_{2,P'}\bigr)+\bigl(\lVert\Phi\rVert_{2,P'}-\phi_{N}(\nu')\bigr)$ and the first axiom of Ordered Field (used twice to add the two bounds),

This proves clause 1.

Step 3 (Clause 2: a finite net). Let $K$ and $\varepsilon$ be as in clause 2. The objects below are chosen in this order: $\delta$, then the finite set $J$ and its enumeration $\nu_{1},\dots,\nu_{n}$, then $N_{1},\dots,N_{n}$, then $N_{0}$. Put $\delta=(\varepsilon\cdot2^{-1})\cdot2^{-1}$; by claim 8 of Elementary Order Arithmetic in an Ordered Field, applied to $\varepsilon$ and to $\varepsilon\cdot2^{-1}$, we have $0<\delta$, $\delta+\delta=\varepsilon\cdot2^{-1}$ and $\varepsilon\cdot2^{-1}+\varepsilon\cdot2^{-1}=\varepsilon$.

By The Quadratic Wasserstein Distance is a Metric on the Wasserstein Space §metric, $W_{2}$ is a metric on $\mathcal{P}_{2}(\mathbb{R}^{q})$; let $\mathcal{T}$ be the collection of its open sets, a topology by Metric Open Sets Form a Topology, and let $\mathcal{T}_{K}$ be the subspace topology on $K$ (Subspace Topology); by Compact Topological Space and Compact Subset, $(K,\mathcal{T}_{K})$ is compact. For $\lambda\in K$ let $B(\lambda)=\{\rho\in\mathcal{P}_{2}(\mathbb{R}^{q}):W_{2}(\lambda,\rho)<\delta\}$ be the open ball of Open Ball in a Metric Space; it belongs to $\mathcal{T}$ by Open Ball in a Metric Space is Open, so $U_{\lambda}=K\cap B(\lambda)\in\mathcal{T}_{K}$. Since $W_{2}(\lambda,\lambda)=0<\delta$ by The Quadratic Wasserstein Distance is a Metric on the Wasserstein Space §separation, $\lambda\in U_{\lambda}$, so the family $(U_{\lambda})_{\lambda\in K}$ covers $K$. By compactness there is a finite $J\subseteq K$ with $K\subseteq\bigcup_{\lambda\in J}U_{\lambda}$.

If $J=\emptyset$, the union is empty, so $K=\emptyset$ and $N_{0}=1$ satisfies clause 2 vacuously. Otherwise, by Finite Set and Number of Elements of a Set there are $n\in\mathbb{N}$ and a bijection $[n]\to J$, $j\mapsto\nu_{j}$. For each $j\in[n]$, Law of Large Numbers for Empirical Measures in the Wasserstein Distance §convergence, applied to $\nu_{j}\in K\subseteq\mathcal{P}_{2}(\mathbb{R}^{q})$, and Limit of a Sequence of Real Numbers with the positive number $\delta+\delta$ give $N_{j}\in\mathbb{N}$ such that $|\phi_{N}(\nu_{j})-0|<\delta+\delta$ for every $N\in\mathbb{N}$ with $N_{j}\le N$. Define $m_{1}=N_{1}$ and $m_{j+1}=\max\{m_{j},N_{j+1}\}$ for $j\in[n]$ with $j+1\in[n]$, the maximum for the total order of $\mathbb{N}$ (Order on the Natural Numbers, claim 3 of Properties of the Order on the Natural Numbers), and put $N_{0}=m_{n}$. By claim 1 of Elementary Properties of the Maximum of Two Elements, $m_{j}\le m_{j+1}$ and $N_{j+1}\le m_{j+1}$, so by induction on $j$ with transitivity (claim 1 of Properties of the Order on the Natural Numbers) $N_{i}\le m_{j}$ for all $i\le j$ in $[n]$; in particular $N_{j}\le N_{0}$ for every $j\in[n]$.

Step 4 (Clause 2: conclusion). Let $N\in\mathbb{N}$ with $N_{0}\le N$ and let $\nu\in K$. Choose $j\in[n]$ with $\nu\in U_{\nu_{j}}$, so $W_{2}(\nu_{j},\nu)<\delta$, and $W_{2}(\nu,\nu_{j})<\delta$ by The Quadratic Wasserstein Distance is a Metric on the Wasserstein Space §symmetry. Since $N_{j}\le N_{0}\le N$, $N_{j}\le N$ (claim 1 of Properties of the Order on the Natural Numbers), so $|\phi_{N}(\nu_{j})-0|<\delta+\delta$, and $\phi_{N}(\nu_{j})\le|\phi_{N}(\nu_{j})|$ (claim 3 of Properties of the Absolute Value in an Ordered Field) gives $\phi_{N}(\nu_{j})<\delta+\delta$ (claim 2 of Elementary Order Arithmetic in an Ordered Field). By Step 2 with $\nu'=\nu_{j}$, and claim 3 of Properties of the Absolute Value in an Ordered Field,

the strict inequality by claim 3 of Elementary Order Arithmetic in an Ordered Field (from $W_{2}(\nu,\nu_{j})<\delta$ and $W_{2}(\nu,\nu_{j})\le\delta$), combined with claim 2 there. Adding the two strict bounds (claim 3 of Elementary Order Arithmetic in an Ordered Field),

so $\phi_{N}(\nu)\le\varepsilon$. As $N\ge N_{0}$ and $\nu\in K$ were arbitrary, and $N_{0}$ depends only on $K$ and $\varepsilon$ (through $\delta$ and the finite set $J$), clause 2 holds.