Proof of Convex Combinations of Probability Measures on Euclidean Space, and Finitely Supported Probability Measures as Combinations of Dirac Measures

Each result cited is universally quantified over the data in its own statement. Integrals against measures on $(\mathbb{R}^{n},\mathcal{B}(\mathbb{R}^{n}))$ are those of Measure Spaces and the Lebesgue Integral: Standing Notation §integral, as fixed for probability measures in Probability Measures on Euclidean Space and Random Vectors: Standing Notation §measures, and a Borel map into $[0,\infty]$ is measurable in the sense of Measure Spaces and the Lebesgue Integral: Standing Notation §measurable (Probability Measures on Euclidean Space and Random Vectors: Standing Notation §borel-maps); so Nonnegative Combinations of Two Finite Measures, and the Average of Two Couplings §combination applies with $E=\mathbb{R}^{n}$ and $\mathcal{E}=\mathcal{B}(\mathbb{R}^{n})$. For $\rho'\in\mathcal{P}(\mathbb{R}^{n})$ and $B\in\mathcal{B}(\mathbb{R}^{n})$, $\rho'(B)\le\rho'(\mathbb{R}^{n})=1$ by claim 2 of Basic Properties of a Measure, so $\rho'(B)$ is a real number in $[0,1]$; in particular all sums below are finite sums of real numbers.

Step 1 (Induction for clause 1). For $r\in[M]$ let $\gamma_{r}:\mathcal{B}(\mathbb{R}^{n})\to\mathbb{R}$ be given by $\gamma_{r}(B)=\sum_{i=1}^{r}c_{i}\rho_{i}(B)$. We prove by induction on $r\in[M]$ the statement $H(r)$: (a) $\gamma_{r}$ is a measure on $(\mathbb{R}^{n},\mathcal{B}(\mathbb{R}^{n}))$ with $\gamma_{r}(\mathbb{R}^{n})<\infty$; (b) for every Borel $f:\mathbb{R}^{n}\to[0,\infty]$, if $\int f\,d\rho_{i}<\infty$ for every $i\in[r]$ then $\int f\,d\gamma_{r}=\sum_{i=1}^{r}c_{i}\int f\,d\rho_{i}$, and if $\int f\,d\rho_{i}=\infty$ for some $i\in[r]$ with $0<c_{i}$ then $\int f\,d\gamma_{r}=\infty$; (c) every Borel $f:\mathbb{R}^{n}\to\mathbb{R}$ that is integrable with respect to every $\rho_{i}$, $i\in[r]$, is integrable with respect to $\gamma_{r}$, and $\int f\,d\gamma_{r}=\sum_{i=1}^{r}c_{i}\int f\,d\rho_{i}$.

Base case $r=1$. By claim 1 of Properties of Finite Sums, $\gamma_{1}(B)=c_{1}\rho_{1}(B)=c_{1}\rho_{1}(B)+0\cdot\rho_{1}(B)$. Apply Nonnegative Combinations of Two Finite Measures, and the Average of Two Couplings §combination with $\alpha=\beta=\rho_{1}$ (finite, as $\rho_{1}(\mathbb{R}^{n})=1$), $s=c_{1}$ and $t=0$: $\gamma_{1}$ is a measure with $\gamma_{1}(\mathbb{R}^{n})<\infty$, giving (a). For Borel $f\ge0$ the same clause gives $\int f\,d\gamma_{1}=c_{1}\int f\,d\rho_{1}+0\cdot\int f\,d\rho_{1}$ in $[0,\infty]$, where the second term is $0$ even when $\int f\,d\rho_{1}=\infty$, by the convention $0\cdot\infty=0$ of Measure, Measure Space, and Probability Measure; so $\int f\,d\gamma_{1}=c_{1}\int f\,d\rho_{1}$. If $\int f\,d\rho_{1}<\infty$ this is $\sum_{i=1}^{1}c_{i}\int f\,d\rho_{i}$ by claim 1 of Properties of Finite Sums; if $\int f\,d\rho_{1}=\infty$ and $0<c_{1}$ it is $c_{1}\cdot\infty=\infty$ by Measure Spaces and the Lebesgue Integral: Standing Notation §extended. This is (b). For (c), the last assertion of Nonnegative Combinations of Two Finite Measures, and the Average of Two Couplings §combination (with the same data) shows that $f$ is integrable with respect to $\gamma_{1}$ with $\int f\,d\gamma_{1}=c_{1}\int f\,d\rho_{1}+0=\sum_{i=1}^{1}c_{i}\int f\,d\rho_{i}$.

Inductive step. Let $r\in[M]$ with $r+1\in[M]$, and assume $H(r)$. By claim 1 of Properties of Finite Sums, $\gamma_{r+1}(B)=\gamma_{r}(B)+c_{r+1}\rho_{r+1}(B)=1\cdot\gamma_{r}(B)+c_{r+1}\rho_{r+1}(B)$. Apply Nonnegative Combinations of Two Finite Measures, and the Average of Two Couplings §combination with $\alpha=\gamma_{r}$ (a measure with $\gamma_{r}(\mathbb{R}^{n})<\infty$ by $H(r)$(a)), $\beta=\rho_{r+1}$, $s=1$ and $t=c_{r+1}$. It gives (a) for $r+1$, and for Borel $f\ge0$

If $\int f\,d\rho_{i}<\infty$ for every $i\in[r+1]$, then $\int f\,d\gamma_{r}=\sum_{i=1}^{r}c_{i}\int f\,d\rho_{i}$ is real by $H(r)$(b), and the right side equals $\sum_{i=1}^{r+1}c_{i}\int f\,d\rho_{i}$ by claim 1 of Properties of Finite Sums. If $\int f\,d\rho_{i}=\infty$ for some $i\in[r+1]$ with $0<c_{i}$, then either $i\in[r]$, in which case $\int f\,d\gamma_{r}=\infty$ by $H(r)$(b) and the first term is $1\cdot\infty=\infty$, or $i=r+1$, in which case the second term is $c_{r+1}\cdot\infty=\infty$; in both cases the sum is $\infty$ by the conventions of Measure Spaces and the Lebesgue Integral: Standing Notation §extended. This is (b) for $r+1$. For (c), if $f$ is integrable with respect to every $\rho_{i}$ with $i\in[r+1]$, then it is integrable with respect to $\gamma_{r}$ by $H(r)$(c) and with respect to $\rho_{r+1}$, so the last assertion of Nonnegative Combinations of Two Finite Measures, and the Average of Two Couplings §combination shows that $f$ is integrable with respect to $\gamma_{r+1}$ with $\int f\,d\gamma_{r+1}=\int f\,d\gamma_{r}+c_{r+1}\int f\,d\rho_{r+1}=\sum_{i=1}^{r+1}c_{i}\int f\,d\rho_{i}$, by $H(r)$(c) and claim 1 of Properties of Finite Sums.

Step 2 (Conclusion of clause 1). By Step 1, $H(M)$ holds, and $\gamma_{M}$ is the function $\sum_{i=1}^{M}c_{i}\rho_{i}$ of the statement. It is a measure by $H(M)$(a), and $\gamma_{M}(\mathbb{R}^{n})=\sum_{i=1}^{M}c_{i}\rho_{i}(\mathbb{R}^{n})=\sum_{i=1}^{M}c_{i}=1$, so it is a probability measure. The two assertions about Borel $f:\mathbb{R}^{n}\to[0,\infty]$ are $H(M)$(b), and the assertion about real-valued $f$ is $H(M)$(c).

Step 3 (Clause 2: the weights). The set $F$ is finite, so by Pairs of Euclidean Points: Coordinate Projections, Pairings, the Product Measure on a Euclidean Space, Borel Norm Functions and Finite Sets §finite-sets the sets $F$, $\mathbb{R}^{n}\setminus F$ and $\{z_{i}\}$ for $i\in[M]$ belong to $\mathcal{B}(\mathbb{R}^{n})$. The points $z_{i}$ being pairwise distinct, the sets $\{z_{1}\},\dots,\{z_{M}\}$ are pairwise disjoint with union $F$, so claim 1 of Basic Properties of a Measure gives $\rho(F)=\sum_{i=1}^{M}\rho(\{z_{i}\})$, a sum of real numbers. By claim 3 of the same lemma (differences, $\rho$ being finite), $\rho(F)=\rho(\mathbb{R}^{n}\setminus(\mathbb{R}^{n}\setminus F))=\rho(\mathbb{R}^{n})-\rho(\mathbb{R}^{n}\setminus F)=1-0=1$. Hence $\sum_{i=1}^{M}\rho(\{z_{i}\})=1$. Put $w_{i}=\rho(\{z_{i}\})$, a nonnegative real number. Each $\delta_{z_{i}}$ belongs to $\mathcal{P}(\mathbb{R}^{n})$ by Dirac Measures on Euclidean Space: Probability Measure, Integrals, Push-Forwards and the Coupling of Two Dirac Measures §measure, so by clause 1 the function $\sigma=\sum_{i=1}^{M}w_{i}\delta_{z_{i}}$ is a probability measure on $\mathbb{R}^{n}$.

Step 4 (Clause 2: the representation). Let $B\in\mathcal{B}(\mathbb{R}^{n})$. The sets $B\cap F$ and $B\setminus F$ are Borel, disjoint, with union $B$, and $\rho(B\setminus F)\le\rho(\mathbb{R}^{n}\setminus F)=0$ by claim 2 of Basic Properties of a Measure, so $\rho(B\setminus F)=0$ and, by claim 1 of that lemma, $\rho(B)=\rho(B\cap F)$. For $i\in[M]$ let $C_{i}=\{z_{i}\}\cap B$, a Borel set; the sets $C_{1},\dots,C_{M}$ are pairwise disjoint with union $B\cap F$, so claim 1 of Basic Properties of a Measure gives $\rho(B\cap F)=\sum_{i=1}^{M}\rho(C_{i})$. If $z_{i}\in B$, then $C_{i}=\{z_{i}\}$ and $\rho(C_{i})=w_{i}=w_{i}\,\delta_{z_{i}}(B)$; if $z_{i}\notin B$, then $C_{i}=\varnothing$ and $\rho(C_{i})=0=w_{i}\,\delta_{z_{i}}(B)$, by the definition of $\delta_{z_{i}}$ in The Dirac Measure at a Point of Euclidean Space §dirac. Therefore

for every $B\in\mathcal{B}(\mathbb{R}^{n})$, that is, $\rho=\sum_{i=1}^{M}\rho(\{z_{i}\})\,\delta_{z_{i}}$.