Proof of Convergence in Distribution of Real Random Variables through Distribution Functions

Throughout, $\mu_{n}=P_{X_{n}}$ and $\mu=P_{X}$ are the laws, Borel probability measures on $(\mathbb{R},d_{\mathbb{R}})$ (The Law of a Random Element (i)), and $F_{n}=F_{X_{n}}$, $F=F_{X}$, so that $F_{n}(t)=\mu_{n}((-\infty,t])$ and $F(t)=\mu((-\infty,t])$ (Distribution Functions (ii), Distribution Functions (i)). The distance on $\mathbb{R}$ is $d_{\mathbb{R}}(x,y)=|x-y|$ (The Distance on the Real Numbers Is a Map with Values the Absolute Differences (i)). So continuity of $F$ at $t$ (Continuous Map Between Metric Spaces (i)) is the usual $\varepsilon$-$\delta$ condition with $|u-t|<\delta$; for $t\in\mathbb{R}$ the ray $(-\infty,t)$ is open, as it contains the ball $B_{d_{\mathbb{R}}}(x,t-x)$ about each of its points $x$; and $(-\infty,t]$ is closed, its complement $(t,\infty)$ containing $B_{d_{\mathbb{R}}}(x,x-t)$ about each of its points $x$.

Step 1 (proof of (i): upper bound). By Convergence in Distribution of Random Elements of a Metric Space (i), $\mu_{n}\Rightarrow\mu$. Let $t$ be a continuity point of $F$. The ray $(-\infty,t]$ is closed, so Portmanteau Theorem on a Metric Space (iii) (from Portmanteau Theorem on a Metric Space (i)) gives $\limsup_{n}F_{n}(t)\le F(t)$.

Step 2 (proof of (i): lower bound). Since $\mu_{n}((-\infty,t))\le F_{n}(t)$ for every $n$ (Basic Properties of a Measure (iii)) and the ray $(-\infty,t)$ is open, Properties of the Limit Superior and the Limit Inferior: Tail Formula, Order, Comparison, Bounds, Eventual and Frequent Inequalities, Subsequences, Negation and Convergence (iii) and Portmanteau Theorem on a Metric Space (iv) give

$$\liminf_{n}F_{n}(t)\ge\liminf_{n}\mu_{n}\bigl((-\infty,t)\bigr)\ge\mu\bigl((-\infty,t)\bigr).$$

We claim that $\mu((-\infty,t))\ge F(t)$. For every $s<t$ the ray $(-\infty,t)$ contains $(-\infty,s]$, so $F(s)\le\mu((-\infty,t))$. Suppose $\mu((-\infty,t))<F(t)$, and let $\varepsilon=F(t)-\mu((-\infty,t))>0$. By continuity of $F$ at $t$ there is $\delta>0$ with $|F(u)-F(t)|<\varepsilon$ whenever $|u-t|<\delta$. For $s=t-\theta$ with $0<\theta<\delta$ this gives $F(s)>F(t)-\varepsilon=\mu((-\infty,t))$, a contradiction. Hence $\liminf_{n}F_{n}(t)\ge F(t)$, and with Step 1 and Properties of the Limit Superior and the Limit Inferior: Tail Formula, Order, Comparison, Bounds, Eventual and Frequent Inequalities, Subsequences, Negation and Convergence (ii),

$$F(t)\le\liminf_{n}F_{n}(t)\le\limsup_{n}F_{n}(t)\le F(t),$$

so $F_{n}(t)\to F(t)$ by Properties of the Limit Superior and the Limit Inferior: Tail Formula, Order, Comparison, Bounds, Eventual and Frequent Inequalities, Subsequences, Negation and Convergence (xi).

Step 3 (null points are dense continuity points). Call $t\in\mathbb{R}$ a null point if $\mu(\{t\})=0$. Let $a<b$. By A Measurable Real Function Has Null Level Sets at Levels Arbitrarily Close to Any Point, for a Finite Measure (i), applied to the finite measure $P$ and the measurable function $X$, there is $t$ with $a<t<b$ and $\mu(\{t\})=P(X=t)=0$. So every interval $(a,b)$ with $a<b$ contains a null point.

Now let $t$ be a null point; we show that $F$ is continuous at $t$. Since $(-\infty,t)=(-\infty,t]\setminus\{t\}$, we have $\mu((-\infty,t))=F(t)-\mu(\{t\})=F(t)$ (Basic Properties of a Measure (iv)). To control $\mu((-\infty,s))$ for $s$ just below $t$, we reflect. The map $\varrho:\mathbb{R}\to\mathbb{R}$, $\varrho(u)=-u$, preserves distances, so it is continuous and hence Borel measurable (Measurable Maps: Generators, Composition, Continuous Maps, Restrictions, and Criteria for Real- and [0,∞]-Valued Maps (iv)), and the image measure $\nu=\mu\circ\varrho^{-1}$ is a Borel probability measure on $\mathbb{R}$ (Image Measures and Measures with a Density: Measures, Integrals and Change of Variables (i)). Its distribution function is

$$F_{\nu}(y)=\mu\bigl(\varrho^{-1}((-\infty,y])\bigr)=\mu\bigl([-y,\infty)\bigr)=1-\mu\bigl((-\infty,-y)\bigr)\qquad(y\in\mathbb{R}).$$

Let $\varepsilon>0$. Right-continuity of $F_{\nu}$ at $-t$ (Properties of Distribution Functions (ii), applied to $\nu$) gives $\delta_{1}>0$ with $F_{\nu}(y)<F_{\nu}(-t)+\varepsilon$ for $-t\le y<-t+\delta_{1}$. Writing $s=-y$, this says

$$\mu\bigl((-\infty,s)\bigr)>\mu\bigl((-\infty,t)\bigr)-\varepsilon=F(t)-\varepsilon\qquad(t-\delta_{1}<s\le t),$$

and since $\mu((-\infty,s))\le F(s)\le F(t)$, we get $|F(s)-F(t)|<\varepsilon$ for $t-\delta_{1}<s\le t$. Right-continuity of $F$ at $t$ (the same lemma, applied to $\mu$) gives $\delta_{2}>0$ with $F(t)\le F(y)<F(t)+\varepsilon$ for $t\le y<t+\delta_{2}$. Hence $|F(u)-F(t)|<\varepsilon$ whenever $|u-t|<\min(\delta_{1},\delta_{2})$, and $F$ is continuous at $t$.

Step 4 (proof of (ii): reduction). By Convergence in Distribution of Random Elements of a Metric Space (i), $X_{n}\xrightarrow{d}X$ means $\mu_{n}\Rightarrow\mu$, so by Portmanteau Theorem on a Metric Space (ii) (equivalent to Portmanteau Theorem on a Metric Space (i)) it suffices to show

$$\int_{\mathbb{R}}f\,d\mu_{n}\to\int_{\mathbb{R}}f\,d\mu$$

for every bounded Lipschitz function $f:\mathbb{R}\to\mathbb{R}$. Fix such an $f$, a bound $M\ge0$ with $|f|\le M$, and a Lipschitz constant $L\ge0$, so that $|f(u)-f(v)|\le L|u-v|$. As $f$ is continuous, hence measurable (A Lipschitz Map is Uniformly Continuous and Continuous (ii), Measurable Maps: Generators, Composition, Continuous Maps, Restrictions, and Criteria for Real- and [0,∞]-Valued Maps (iv)), the functions $f$ and $f\mathbf{1}_{B}$, for Borel sets $B$, are bounded and measurable; so they lie in $\mathcal{L}^{1}(\rho)$ for every Borel probability measure $\rho$ on $\mathbb{R}$ (Bounded Measurable Functions on a Finite Measure Space Are Integrable (i)), and $n\mapsto\int f\mathbf{1}_{B}\,d\mu_{n}$ is a real sequence (The Simple, Nonnegative, Integrable and Signed Integrals Are Predicative in the Measure and the Function, at Any Variable Positions (iii)). Integrals against such a $\rho$ are expectations on the probability space $(\mathbb{R},\mathcal{B}(\mathbb{R}),\rho)$, so they obey Rules for Expectations, Variances and Covariances (i), Rules for Expectations, Variances and Covariances (iii), Rules for Expectations, Variances and Covariances (iv) and Rules for Expectations, Variances and Covariances (v); these rules are used without further mention below.

Step 5 (the choices). Fix $\varepsilon>0$. By Properties of Distribution Functions (iii) and Properties of Distribution Functions (iv) there are $M_{1},M_{2}>0$ with $F(x)<\varepsilon/2$ for $x\le-M_{1}$ and $F(x)>1-\varepsilon/2$ for $x\ge M_{2}$, and by Step 3 there are null points $a\in(-M_{1}-1,-M_{1})$ and $b\in(M_{2},M_{2}+1)$. Then $a<b$ and

$$\mu\bigl((a,b]\bigr)=F(b)-F(a)>1-\varepsilon$$

(Basic Properties of a Measure (iv)). With $a$ and $b$ fixed, put

$$h=\frac{\varepsilon}{2L+1},\qquad\kappa=\frac{\varepsilon}{b-a},$$

so that $2Lh<\varepsilon$ and $\kappa>0$. For $c\in[a,b]$ and a Borel probability measure $\rho$ on $\mathbb{R}$ let $I_{\rho}(c)=\int f\mathbf{1}_{(a,c]}\,d\rho$, and let

$$\beta(c)=2Lh\bigl(F(c)-F(a)\bigr)+\kappa(c-a).$$

Call $c\in[a,b]$ good if $c$ is a null point and there is $N\in\mathbb{N}$ with

$$\bigl|I_{\mu_{n}}(c)-I_{\mu}(c)\bigr|\le\beta(c)\qquad\text{for all }n\ge N.$$

The point $a$ is good: $(a,a]=\emptyset$, so $I_{\mu_{n}}(a)=I_{\mu}(a)=0=\beta(a)$.

Step 6 (one step to the right). Let $c$ be good, with $N$ as above, and let $c'$ be a null point with $c<c'\le\min(c+h,b)$. We show that $c'$ is good. Let $\rho$ be $\mu_{n}$ or $\mu$. Since $(a,c']$ is the disjoint union of $(a,c]$ and $(c,c']$, linearity gives $I_{\rho}(c')=I_{\rho}(c)+\int f\mathbf{1}_{(c,c']}\,d\rho$. For $u\in(c,c']$ we have $|u-c'|\le h$, hence $|f(u)-f(c')|\le Lh$, and therefore

$$\Bigl|\int f\mathbf{1}_{(c,c']}\,d\rho-f(c')\,\rho\bigl((c,c']\bigr)\Bigr|\le Lh\,\rho\bigl((c,c']\bigr).$$

By Step 3, $c$ and $c'$ are continuity points of $F$, so $F_{n}(c)\to F(c)$ and $F_{n}(c')\to F(c')$ by hypothesis. Let

$$\eta=\frac{\kappa(c'-c)}{M+Lh+1}>0,$$

and choose $N'\ge N$ with $|F_{n}(c)-F(c)|<\eta/2$ and $|F_{n}(c')-F(c')|<\eta/2$ for all $n\ge N'$. As $\mu_{n}((c,c'])=F_{n}(c')-F_{n}(c)$ and $\mu((c,c'])=F(c')-F(c)$ (Basic Properties of a Measure (iv)), this gives $|\mu_{n}((c,c'])-\mu((c,c'])|<\eta$ for $n\ge N'$. Combining these estimates with $|f(c')|\le M$, for every $n\ge N'$

$$\bigl|I_{\mu_{n}}(c')-I_{\mu}(c')\bigr|\le\beta(c)+M\eta+Lh\bigl(\mu((c,c'])+\eta\bigr)+Lh\,\mu\bigl((c,c']\bigr)\le\beta(c)+2Lh\bigl(F(c')-F(c)\bigr)+\kappa(c'-c)=\beta(c'),$$

since $(M+Lh)\eta\le\kappa(c'-c)$. So $c'$ is good.

Step 7 (the supremum of the good points). The set $G$ of good points contains $a$ and is bounded above by $b$, so it has a supremum $s\in[a,b]$ (The Real Numbers Form an Ordered Field in Which Every Nonempty Set Bounded Above Has a Supremum (ii)). As $s-h<s$, the number $s-h$ is not an upper bound of $G$, so there is a good point $c>s-h$. Suppose $c+h<b$. Then $s<c+h$, and by Step 3 there is a null point $c'$ with $s<c'<c+h$; since $c\le s<c'\le c+h<b$, Step 6 shows that $c'$ is good, contradicting $c'>s$. Hence $b\le c+h$. If $c=b$, then $b$ is good; otherwise $c<b\le c+h$, and $b$, a null point, is good by Step 6. In either case $b$ is good, and $s=b$.

Step 8 (passage to the limit). Since $b$ is good and $F(b)-F(a)\le1$, there is $N_{0}$ with

$$\bigl|I_{\mu_{n}}(b)-I_{\mu}(b)\bigr|\le\beta(b)\le2Lh+\kappa(b-a)<2\varepsilon\qquad(n\ge N_{0}).$$

For $\rho$ equal to $\mu_{n}$ or $\mu$, $\int f\,d\rho=I_{\rho}(b)+\int f\mathbf{1}_{\mathbb{R}\setminus(a,b]}\,d\rho$, and

$$\Bigl|\int f\mathbf{1}_{\mathbb{R}\setminus(a,b]}\,d\rho\Bigr|\le M\rho\bigl(\mathbb{R}\setminus(a,b]\bigr)=M\bigl(1-\rho((a,b])\bigr)$$

(Basic Properties of a Measure (iv)). As $a$ and $b$ are continuity points of $F$, choosing $N_{1}$ as $N'$ was chosen in Step 6, now with $\eta=\mu((a,b])-(1-\varepsilon)>0$, gives $\mu_{n}((a,b])=F_{n}(b)-F_{n}(a)>1-\varepsilon$ for $n\ge N_{1}$. For $n\ge\max(N_{0},N_{1})$ adding the estimates above gives

$$\Bigl|\int f\,d\mu_{n}-\int f\,d\mu\Bigr|<2\varepsilon+M\varepsilon+M\varepsilon=2(1+M)\varepsilon .$$

Given $\varepsilon'>0$, Steps 5 to 8 with $\varepsilon=\varepsilon'/(2(1+M))$ yield $N$ with $|\int f\,d\mu_{n}-\int f\,d\mu|<\varepsilon'$ for all $n\ge N$. Hence $\int f\,d\mu_{n}\to\int f\,d\mu$, and Step 4 gives $X_{n}\xrightarrow{d}X$.