Proof of Implications between Modes of Convergence of Random Elements

Throughout, $d(X_{n},X)$, $d(X_{n},Z)$ and $d(X,Z)$ are random variables (The Distance of Two Random Elements of a Separable Metric Space Is a Random Variable (i)), so the sets below defined by them are events; and $d$ is symmetric, satisfies the triangle inequality and vanishes exactly on the diagonal (Metric Space (iv), Metric Space (v), Metric Space (iii)).

Thresholds. Define real numbers $r_{k}$ ($k\in\mathbb{N}$) by recursion (Recursion on the Natural Numbers Starting at One (i)):

$$r_{1}=1,\qquad r_{k+1}=\tfrac{1}{2}r_{k}.$$

By induction $0<r_{k+1}<r_{k}$ for every $k$. For every real $D>0$ there is $k_{0}$ with $r_{k}<D$ for all $k\ge k_{0}$: the infimum $\rho\ge0$ of the $r_{k}$ (Infima, the Archimedean Property, Density of the Rationals and Rational Approximation from Below in an Ordered Field Whose Nonempty Sets Bounded Above Have Suprema (i), applicable to $\mathbb{R}$ by The Real Numbers, with the Natural Numbers, Integers and Rationals Identified with Subsets of the Reals, and Completeness (vii)) is $0$, since otherwise some $r_{k}<2\rho$ and then $r_{k+1}<\rho$; so some $r_{k_{0}}<D$, and $r_{k}\le r_{k_{0}}$ for $k\ge k_{0}$.

Step 1 (proof of (i)). Let $A$ be an event with $P(A)=1$ on which $X_{n}(\omega)\to X(\omega)$ (Almost Sure Convergence of Random Elements of a Metric Space (i)), and fix $\varepsilon>0$. The strict tails

$$B_{n}=\bigcup_{m>n}\{d(X_{m},X)>\varepsilon\}\qquad(n\in\mathbb{N})$$

satisfy $B_{n+1}\subseteq B_{n}$ and $P(B_{1})\le1$, and $B=\bigcap_{n}B_{n}$ is an event (Unions and Intersections over a Countable Index Set in a σ-Algebra (ii)). For $\omega\in A$ we have $d(X_{m}(\omega),X(\omega))\le\varepsilon$ for all large $m$ (Convergent Sequence in a Metric Space (i)), so $\omega\notin B$; hence $B\subseteq\Omega\setminus A$, and $P(B)\le1-P(A)=0$ (Basic Properties of a Measure (iii), Basic Properties of a Measure (v)). Continuity from above, Basic Properties of a Measure (ix), gives

$$\inf_{n}P(B_{n})=P(B)=0 .$$

Now let $\eta>0$. As $\eta$ is not a lower bound of the $P(B_{n})$, there is $N$ with $P(B_{N})<\eta$. For $n\ge N+1$ we have $\{d(X_{n},X)>\varepsilon\}\subseteq B_{N}$, so by monotonicity

$$0\le P\bigl(d(X_{n},X)>\varepsilon\bigr)\le P(B_{N})<\eta .$$

So $P(d(X_{n},X)>\varepsilon)\to0$ (Convergent Sequences of Real Numbers (i)), and $X_{n}\xrightarrow{P}X$ by Convergence in Probability (i).

Step 2 (proof of (ii)). Let $W_{n}=\lVert Y_{n}-Y\rVert$, so that $d_{E}(Y_{n},Y)=W_{n}$ (Properties of the Euclidean Norm on $\mathbb{R}^n$: Square, Distance, Vanishing, Coordinate Bounds, Homogeneity, Triangle Inequalities and Expansion of a Square (ii)) and $W_{n}^{p}\in\mathcal{M}^{+}(\Omega,\mathcal{F})$ as $W_{n}\in\mathcal{L}^{p}(P)$. Let $\varepsilon>0$; then $\varepsilon^{p}>0$ (Properties of Real Powers (ii)), and since $t\mapsto t^{p}$ is strictly increasing on $[0,\infty)$ (Properties of Real Powers (vi)), $\{W_{n}>\varepsilon\}\subseteq\{W_{n}^{p}\ge\varepsilon^{p}\}$. Markov's inequality, Markov's and Chebyshev's Inequalities (i), with $a=\varepsilon^{p}$, and monotonicity give

$$P(W_{n}>\varepsilon)\,\varepsilon^{p}\le\mathbb{E}W_{n}^{p}\qquad(n\in\mathbb{N}).$$

Let $\eta>0$. Since $\eta\varepsilon^{p}>0$ and $\mathbb{E}W_{n}^{p}\to0$ (Convergence in Lᵖ of Random Vectors (i)), there is $N$ with $\mathbb{E}W_{n}^{p}<\eta\varepsilon^{p}$ for all $n\ge N$; for these $n$, $P(W_{n}>\varepsilon)\,\varepsilon^{p}<\eta\varepsilon^{p}$, so $0\le P(W_{n}>\varepsilon)<\eta$. Hence $P(W_{n}>\varepsilon)\to0$ (Convergent Sequences of Real Numbers (i)), and $Y_{n}\xrightarrow{P}Y$ by Convergence in Probability (i).

Step 3 (proof of (iii)). For $k\in\mathbb{N}$ let $N_{k}$ be the least $n$ (Arithmetic and Order of the Natural Numbers (xv)) with $P(d(X_{m},X)>r_{k})<r_{k}$ for every $m\ge n$, which exists by Convergence in Probability (i). Define $\sigma:\mathbb{N}\to\mathbb{N}$ recursively (Recursion on the Natural Numbers Starting at One (i)) by

$$\sigma(1)=N_{1},\qquad\sigma(k+1)=\sigma(k)+N_{k+1}.$$

Then $\sigma$ is strictly increasing and $\sigma(k)\ge N_{k}$ for every $k$, so $(X_{n_{k}})$ with $n_{k}=\sigma(k)$ is a subsequence (Subsequences (i)), and the events $A_{k}=\{d(X_{n_{k}},X)>r_{k}\}$ satisfy $P(A_{k})<r_{k}$. Let $s_{K}=\sum_{k=1}^{K}r_{k}$. By induction on $K$,

$$s_{K}+r_{K}=2\qquad(K\in\mathbb{N}),$$

since $s_{1}+r_{1}=2$ and $s_{K+1}+r_{K+1}=s_{K}+2r_{K+1}=s_{K}+r_{K}$; so $s_{K}=2-r_{K}<2$. Hence $\sum_{k=1}^{K}P(A_{k})\le s_{K}\le2$ for every $K$, so the sum $\sum_{k\in\mathbb{N}}P(A_{k})$ in $[0,\infty]$ is at most $2$ (Sums of Series: a Convergent Series Has a Real Sum, and a Bound on All Partial Sums Bounds the Sum in [0,∞] (ii)), in particular finite. The first Borel–Cantelli lemma, Borel-Cantelli Lemmas (i), gives $P(\{A_{k}\text{ infinitely often}\})=0$. Its complement $A$ is an event with $P(A)=1$ (Basic Properties of a Measure (v)). For $\omega\in A$ there is $K$ with $d(X_{n_{k}}(\omega),X(\omega))\le r_{k}$ for all $k>K$. Given $\varepsilon>0$, choose $k_{0}$ with $r_{k}<\varepsilon$ for $k\ge k_{0}$ (Thresholds); then $d(X_{n_{k}}(\omega),X(\omega))<\varepsilon$ for all $k>\max(K,k_{0})$. So $X_{n_{k}}(\omega)\to X(\omega)$ (Convergent Sequence in a Metric Space (i)) for every $\omega\in A$, which is Almost Sure Convergence of Random Elements of a Metric Space (i).

Step 4 (proof of (iv)). By Convergence in Distribution through Expectations of Bounded Continuous and of Bounded Lipschitz Functions (ii) it suffices to show $\mathbb{E}f(X_{n})\to\mathbb{E}f(X)$ for every bounded Lipschitz $f:S\to\mathbb{R}$. Let $|f|\le M$ and let $L\ge0$ be a Lipschitz constant. The maps $f\circ X_{n}$ and $f\circ X$ are measurable (A Lipschitz Map is Uniformly Continuous and Continuous (ii), Measurable Maps: Generators, Composition, Continuous Maps, Restrictions, and Criteria for Real- and [0,∞]-Valued Maps (iv), Measurable Maps: Generators, Composition, Continuous Maps, Restrictions, and Criteria for Real- and [0,∞]-Valued Maps (ii)) and bounded by $M$, hence integrable (Bounded Measurable Functions on a Finite Measure Space Are Integrable (i)), and $n\mapsto\mathbb{E}f(X_{n})$ is a real sequence (The Simple, Nonnegative, Integrable and Signed Integrals Are Predicative in the Measure and the Function, at Any Variable Positions (iii)). Let $\eta>0$, put $h=\eta/2$ and $c=1+L$, which is positive even when $L=0$, and choose, in this order, $\varepsilon>0$ and $\delta>0$ with

$$c\,\varepsilon=h,\qquad(1+2M)\,\delta=h .$$

For $D_{n}=\{d(X_{n},X)>\varepsilon\}$, whose indicator $\mathbf{1}_{D_{n}}$ is bounded and measurable, hence integrable (Bounded Measurable Functions on a Finite Measure Space Are Integrable (i)),

$$|f(X_{n})-f(X)|\le L\varepsilon+2M\,\mathbf{1}_{D_{n}},$$

so, as $L\varepsilon\le c\,\varepsilon=h$,

$$|\mathbb{E}f(X_{n})-\mathbb{E}f(X)|\le h+2M\,P(D_{n})$$

by Rules for Expectations, Variances and Covariances (v), Rules for Expectations, Variances and Covariances (iv), Rules for Expectations, Variances and Covariances (iii), Rules for Expectations, Variances and Covariances (ii) and Rules for Expectations, Variances and Covariances (i). Since $P(D_{n})\to0$ (Convergence in Probability (i)), there is $N$ with $P(D_{n})<\delta$ for $n\ge N$, and then $2M\,P(D_{n})\le2M\delta<(1+2M)\delta=h$, so $|\mathbb{E}f(X_{n})-\mathbb{E}f(X)|<2h=\eta$. Hence $\mathbb{E}f(X_{n})\to\mathbb{E}f(X)$ (Convergent Sequences of Real Numbers (i)).

Step 5 (proof of (v)). Let $t>0$. The triangle inequality and symmetry give $\{d(X,Z)>t\}\subseteq\{d(X_{n},X)>t/2\}\cup\{d(X_{n},Z)>t/2\}$, so by Basic Properties of a Measure (iii) and Basic Properties of a Measure (vii)

$$P\bigl(d(X,Z)>t\bigr)\le P\bigl(d(X_{n},X)>t/2\bigr)+P\bigl(d(X_{n},Z)>t/2\bigr)\qquad(n\in\mathbb{N}).$$

Suppose $P(d(X,Z)>t)=2\theta$ with $\theta>0$. Both tail probabilities on the right tend to $0$ (Convergence in Probability (i)), so for some $N$ each is $<\theta$ at $n=N$, and the display gives $2\theta<2\theta$, a contradiction. Hence $P(d(X,Z)>t)=0$ for every $t>0$. Now let $G_{n}=\{d(X,Z)>r_{n}\}$. As $r_{n+1}<r_{n}$, $G_{n}\subseteq G_{n+1}$. If $X(\omega)\neq Z(\omega)$, then $d(X(\omega),Z(\omega))>0$, and by Thresholds some $r_{n}<d(X(\omega),Z(\omega))$; so $\{X\neq Z\}=\bigcup_{n\in\mathbb{N}}G_{n}$. Continuity from below, Basic Properties of a Measure (viii), gives

$$P(X\neq Z)=\sup_{n}P(G_{n})=0,$$

that is, $X=Z$ almost surely.