Proof of Entropy of Products, Subadditivity over Two Marginals, and Convexity of the Entropy

Each result cited is universally quantified over the data in its own statement. Write $\iota=\iota^{q,p}$, $\mathrm{pr}_{1}=\mathrm{pr}^{q,p}_{1}$ and $\mathrm{pr}_{2}=\mathrm{pr}^{q,p}_{2}$; by Pairs of Euclidean Points: Coordinate Projections, Pairings, the Product Measure on a Euclidean Space, Borel Norm Functions and Finite Sets §projections, $\mathrm{pr}_{1}(\iota(u,v))=u$, $\mathrm{pr}_{2}(\iota(u,v))=v$ and $z=\iota(\mathrm{pr}_{1}(z),\mathrm{pr}_{2}(z))$ for all $u,v,z$, and both projections are Borel. For $Q\in\mathcal{P}(\mathbb{R}^{q+p})$ write $Q_{i}=(\mathrm{pr}_{i})_{\#}Q$. Probability measures are $\sigma$-finite (take every member of the sequence in Measure, Measure Space, and Probability Measure equal to the whole space). For a nonnegative integrable function, its integral in the sense of Integrable Function and the Lebesgue Integral equals its integral as a $[0,\infty]$-valued function, its negative part being $0$; both readings are used below without further comment. Densities are those of The Radon-Nikodym Theorem for a Finite Measure and a Sigma-Finite Measure, and Uniqueness of Densities; a density $\rho$ of $\mu\in\mathcal{P}(\mathbb{R}^{m})$ with respect to $\lambda_{m}$ satisfies $\int_{\mathbb{R}^{m}}\rho\,d\lambda_{m}=1$ by The Radon-Nikodym Theorem for a Finite Measure and a Sigma-Finite Measure, and Uniqueness of Densities §uniqueness.

For a dimension $m\in\mathbb{N}$ let $\psi_{m}(z)=\exp(-\tfrac12\lVert z\rVert^{2})$ for $z\in\mathbb{R}^{m}$, and let $c_{m}$ and $g_{m}=c_{m}\psi_{m}$ be the constant $c_{\eta}$ and the Gaussian smoothing weight $\varphi_{\eta}$ of claim 1 of The Gaussian Smoothing Weight: Normalization, Derivatives, Exponential Tilting, Moments, and First-Order Remainder in dimension $m$ with $\eta=1$; by that claim $\psi_{m}$ and $g_{m}$ are Borel, $0<\psi_{m}(z)$ and $0<g_{m}(z)$ for every $z$, and $c_{m}$ is the only positive real number with $\int_{\mathbb{R}^{m}}c_{m}\psi_{m}\,d\lambda_{m}=1$. Let $\gamma_{m}$ be the measure with density $g_{m}$ with respect to $\lambda_{m}$ of claim 3 of Image Measures, Measures with Densities, and Change of Variables; thus $\gamma_{m}(A)=\int_{\mathbb{R}^{m}}\mathbf{1}_{A}g_{m}\,d\lambda_{m}$ for $A\in\mathcal{B}(\mathbb{R}^{m})$, $\gamma_{m}(\mathbb{R}^{m})=\int_{\mathbb{R}^{m}}g_{m}\,d\lambda_{m}=1$, so $\gamma_{m}\in\mathcal{P}(\mathbb{R}^{m})$, and $g_{m}$ is a density of $\gamma_{m}$ with respect to $\lambda_{m}$. For bounded Borel $h:\mathbb{R}^{m}\to\mathbb{R}$ and $\nu\in\mathcal{P}(\mathbb{R}^{m})$ let $\Lambda^{m}_{h}(\nu)$ be the quantity $\Lambda_{h}(\nu)$ of Relative Entropy on Euclidean Space: the Gibbs Inequality, a Variational Criterion for Finite Relative Entropy, and Closed Sublevel Sets under Weak Convergence for $\gamma=\gamma_{m}$. Step 0 below proves, for every dimension $m$:

(E) a measure $\mu\in\mathcal{P}_{2}(\mathbb{R}^{m})$ has finite entropy if and only if it has finite relative entropy with respect to $\gamma_{m}$, and then $\mathrm{Ent}(\mu)=H(\mu\,|\,\gamma_{m})+\log c_{m}-\tfrac12M_{2}(\mu)$.

A bounded Lipschitz $h:\mathbb{R}^{m}\to\mathbb{R}$ is continuous by A Lipschitz Map is Uniformly Continuous, hence Borel by Probability Measures on Euclidean Space and Random Vectors: Standing Notation §borel-maps; and the constant map $0$ is bounded and Lipschitz with constant $0$ (Bounded Real-Valued Function on a Set, Lipschitz Map Between Metric Spaces).

Step 0 (Proof of (E)). Fix $m$ and $\mu\in\mathcal{P}_{2}(\mathbb{R}^{m})$. (a) If $f$ is a density of $\mu$ with respect to $\gamma_{m}$, then $\rho=fg_{m}$ is a density of $\mu$ with respect to $\lambda_{m}$: $\rho$ is Borel by claim 3 of Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions, nonnegative by claim 5 of Elementary Arithmetic in an Ordered Field, and for $A\in\mathcal{B}(\mathbb{R}^{m})$ claim 3 of Image Measures, Measures with Densities, and Change of Variables, applied to $\mathbf{1}_{A}f$, gives $\mu(A)=\int\mathbf{1}_{A}f\,d\gamma_{m}=\int\mathbf{1}_{A}fg_{m}\,d\lambda_{m}$. (b) If $\mu$ has a density $\rho_{0}$ with respect to $\lambda_{m}$, it has a density with respect to $\gamma_{m}$. Indeed, let $A\in\mathcal{B}(\mathbb{R}^{m})$ with $0=\gamma_{m}(A)=\int\mathbf{1}_{A}g_{m}\,d\lambda_{m}$. By The Lebesgue Integral and Null Sets: Almost-Everywhere Comparison, Markov's Inequality, and Dominated Convergence Almost Everywhere §vanishing, $\mathbf{1}_{A}g_{m}$ vanishes $\lambda_{m}$-almost everywhere; since $g_{m}>0$, the set where it does not vanish is $A$, so $A$ lies in a $\lambda_{m}$-null set and $\lambda_{m}(A)=0$ by The Lebesgue Integral and Null Sets: Almost-Everywhere Comparison, Markov's Inequality, and Dominated Convergence Almost Everywhere §null-union. As $\mathbf{1}_{A}\rho_{0}$ vanishes off $A$, $\mu(A)=\int\mathbf{1}_{A}\rho_{0}\,d\lambda_{m}=0$ by The Lebesgue Integral and Null Sets: Almost-Everywhere Comparison, Markov's Inequality, and Dominated Convergence Almost Everywhere §null-integral. The probability measure $\gamma_{m}$ is $\sigma$-finite, so The Radon-Nikodym Theorem for a Finite Measure and a Sigma-Finite Measure, and Uniqueness of Densities §existence gives a density of $\mu$ with respect to $\gamma_{m}$. (c) Let $f$ and $\rho=fg_{m}$ be as in (a). For every $z$,

if $f(z)=0$ then $\rho(z)=0$ and both sides vanish since $\phi(0)=0$; otherwise $f(z)$, $g_{m}(z)$, $\rho(z)$ are positive (claim 5 of Elementary Order Arithmetic in an Ordered Field) and $\log\rho(z)=\log f(z)+\log c_{m}+\log\psi_{m}(z)=\log f(z)+\log c_{m}-\tfrac12\lVert z\rVert^{2}$ by The Natural Logarithm, so $\phi(\rho(z))=\rho(z)\log\rho(z)=\phi(f(z))g_{m}(z)+R(z)$. Now $\rho$ is $\lambda_{m}$-integrable with $\int\rho\,d\lambda_{m}=1$ (The Radon-Nikodym Theorem for a Finite Measure and a Sigma-Finite Measure, and Uniqueness of Densities §uniqueness); the nonnegative Borel function $x\mapsto\lVert x\rVert^{2}$ (Pairs of Euclidean Points: Coordinate Projections, Pairings, the Product Measure on a Euclidean Space, Borel Norm Functions and Finite Sets §functions) has $\int\lVert x\rVert^{2}\,\mu(dx)=M_{2}(\mu)<\infty$ (The Second Moment of a Probability Measure on Euclidean Space and the Probability Measures with Finite Second Moment §moment), so it is $\mu$-integrable (Measure Spaces and the Lebesgue Integral: Standing Notation §integral), and, $\mu$ being the measure with density $\rho$, claim 3 of Image Measures, Measures with Densities, and Change of Variables shows that $x\mapsto\lVert x\rVert^{2}\rho(x)$ is $\lambda_{m}$-integrable with integral $M_{2}(\mu)$. By claim 2 of Linearity and Monotonicity of the Lebesgue Integral, $R$ is $\lambda_{m}$-integrable with $\int R\,d\lambda_{m}=\log c_{m}-\tfrac12M_{2}(\mu)$. The function $\phi\circ f$ is Borel (The Function $s\log s$: Continuity, Young's Inequality and Lower Bounds, with the Elementary Bounds for the Exponential and the Logarithm §continuous), and by claim 3 of Image Measures, Measures with Densities, and Change of Variables for $\gamma_{m}$ it is $\gamma_{m}$-integrable if and only if $(\phi\circ f)g_{m}$ is $\lambda_{m}$-integrable, with $\int\phi\circ f\,d\gamma_{m}=\int(\phi\circ f)g_{m}\,d\lambda_{m}$. As $\phi\circ\rho=(\phi\circ f)g_{m}+R$, claim 2 of Linearity and Monotonicity of the Lebesgue Integral shows that $\phi\circ f$ is $\gamma_{m}$-integrable if and only if $\phi\circ\rho$ is $\lambda_{m}$-integrable, and then

(d) If $\mu$ has finite relative entropy with respect to $\gamma_{m}$, witnessed by a density $f$ (Relative Entropy of Probability Measures §relative-entropy), then by (a) and (c) $\rho=fg_{m}$ is a density of $\mu$ with respect to $\lambda_{m}$ with $\phi\circ\rho$ integrable, so $\mu$ has finite entropy and (G) is the identity in (E) (The Entropy of a Probability Measure on Euclidean Space §entropy). Conversely, if $\mu$ has finite entropy, witnessed by a density $\rho_{0}$ with $\phi\circ\rho_{0}$ integrable, let $f$ be a density of $\mu$ with respect to $\gamma_{m}$ by (b) and $\rho=fg_{m}$. By (a) and The Radon-Nikodym Theorem for a Finite Measure and a Sigma-Finite Measure, and Uniqueness of Densities §uniqueness, $\rho=\rho_{0}$, hence $\phi\circ\rho=\phi\circ\rho_{0}$, outside a $\lambda_{m}$-null set; $\phi\circ\rho$ is Borel (The Function $s\log s$: Continuity, Young's Inequality and Lower Bounds, with the Elementary Bounds for the Exponential and the Logarithm §continuous), hence integrable by The Lebesgue Integral and Null Sets: Almost-Everywhere Comparison, Markov's Inequality, and Dominated Convergence Almost Everywhere §comparison, and by (c) $\phi\circ f$ is $\gamma_{m}$-integrable. So $\mu$ has finite relative entropy with respect to $\gamma_{m}$, and (G) again gives the identity in (E).

Step 1 (Density of a product). Let $\mu\in\mathcal{P}(\mathbb{R}^{q})$ and $\nu\in\mathcal{P}(\mathbb{R}^{p})$ have densities $\rho_{1}$ and $\rho_{2}$ with respect to $\lambda_{q}$ and $\lambda_{p}$, and let $r=(\rho_{1}\circ\mathrm{pr}_{1})(\rho_{2}\circ\mathrm{pr}_{2})$. Then $r$ is a density of $\mu\boxtimes\nu$ with respect to $\lambda_{q+p}$. Indeed, $r$ is Borel by Probability Measures on Euclidean Space and Random Vectors: Standing Notation §borel-maps and claim 3 of Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions, nonnegative by claim 5 of Elementary Arithmetic in an Ordered Field, and $r(\iota(u,v))=\rho_{1}(u)\rho_{2}(v)$. Let $A\in\mathcal{B}(\mathbb{R}^{q+p})$. By The Integral of an Indicator Function is the Measure of the Set, Pairs of Euclidean Points: Coordinate Projections, Pairings, the Product Measure on a Euclidean Space, Borel Norm Functions and Finite Sets §product, the Sections and Tonelli parts of Tonelli and Fubini Theorems for $\mu\otimes\nu$, claim 3 of Image Measures, Measures with Densities, and Change of Variables applied to the inner and then to the outer integral ($\nu$ and $\mu$ being the measures with densities $\rho_{2}$ and $\rho_{1}$), and claim 1 of Linearity and Monotonicity of the Lebesgue Integral with the constant $\rho_{1}(u)$,

and the last expression is $\int_{\mathbb{R}^{q+p}}\mathbf{1}_{A}r\,d\lambda_{q+p}$ by Lebesgue Measure on a Concatenated Euclidean Space: Iterated Integration over the Two Factors §iterated applied to the Borel function $\mathbf{1}_{A}r$ (claims 1 and 3 of Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions).

Step 2 (Second moments). Let $Q\in\mathcal{P}(\mathbb{R}^{q+p})$. Since $z=\iota(\mathrm{pr}_{1}(z),\mathrm{pr}_{2}(z))$, claim 3 of Concatenation Identifies a Product of Euclidean Spaces with a Euclidean Space gives $\lVert z\rVert^{2}=\lVert\mathrm{pr}_{1}(z)\rVert^{2}+\lVert\mathrm{pr}_{2}(z)\rVert^{2}$. These functions are Borel by Pairs of Euclidean Points: Coordinate Projections, Pairings, the Product Measure on a Euclidean Space, Borel Norm Functions and Finite Sets §functions and Probability Measures on Euclidean Space and Random Vectors: Standing Notation §borel-maps, so claim 1 of Linearity and Monotonicity of the Lebesgue Integral and the change of variables of Probability Measures on Euclidean Space and Random Vectors: Standing Notation §pushforward give $M_{2}(Q)=M_{2}(Q_{1})+M_{2}(Q_{2})$ in $[0,\infty]$ (The Second Moment of a Probability Measure on Euclidean Space and the Probability Measures with Finite Second Moment §moment). Since $a+\infty=\infty$ in $[0,\infty]$, $Q\in\mathcal{P}_{2}(\mathbb{R}^{q+p})$ if and only if $Q_{1}\in\mathcal{P}_{2}(\mathbb{R}^{q})$ and $Q_{2}\in\mathcal{P}_{2}(\mathbb{R}^{p})$. For $Q=\mu\boxtimes\nu$ the marginals are $\mu$ and $\nu$ by Pairs of Euclidean Points: Coordinate Projections, Pairings, the Product Measure on a Euclidean Space, Borel Norm Functions and Finite Sets §product, so $M_{2}(\mu\boxtimes\nu)=M_{2}(\mu)+M_{2}(\nu)$.

Step 3 (Integrals against a product density). Let $\rho_{1},\rho_{2}$ be as in Step 1 and let $F:\mathbb{R}^{q}\to\mathbb{R}$ and $G:\mathbb{R}^{p}\to\mathbb{R}$ be Borel and integrable with respect to $\lambda_{q}$ and $\lambda_{p}$. Then $(F\circ\mathrm{pr}_{1})(\rho_{2}\circ\mathrm{pr}_{2})$ and $(\rho_{1}\circ\mathrm{pr}_{1})(G\circ\mathrm{pr}_{2})$ are integrable with respect to $\lambda_{q+p}$, with integrals $\int_{\mathbb{R}^{q}}F\,d\lambda_{q}$ and $\int_{\mathbb{R}^{p}}G\,d\lambda_{p}$. Suppose first that $F$ and $G$ are nonnegative. By Lebesgue Measure on a Concatenated Euclidean Space: Iterated Integration over the Two Factors §iterated and claim 1 of Linearity and Monotonicity of the Lebesgue Integral, with the constants $F(u)$, $\rho_{1}(u)$ and the real number $\int_{\mathbb{R}^{p}}G\,d\lambda_{p}$, and with $\int\rho_{1}\,d\lambda_{q}=\int\rho_{2}\,d\lambda_{p}=1$,

both are finite, which gives integrability by Measure Spaces and the Lebesgue Integral: Standing Notation §integral. In general, $F=F^{+}-F^{-}$ and $G=G^{+}-G^{-}$ with nonnegative Borel integrable parts (Integrable Function and the Lebesgue Integral); the two products split accordingly, and claim 2 of Linearity and Monotonicity of the Lebesgue Integral together with the nonnegative case gives the assertion.

Step 4 (Claim 1). Let $\mu\in\mathcal{P}_{2}^{\mathrm{Ent}}(\mathbb{R}^{q})$ and $\nu\in\mathcal{P}_{2}^{\mathrm{Ent}}(\mathbb{R}^{p})$ have densities $\rho_{1},\rho_{2}$ with $\phi\circ\rho_{1}$, $\phi\circ\rho_{2}$ integrable, so that $\mathrm{Ent}(\mu)=\int\phi\circ\rho_{1}\,d\lambda_{q}$ and $\mathrm{Ent}(\nu)=\int\phi\circ\rho_{2}\,d\lambda_{p}$ (The Entropy of a Probability Measure on Euclidean Space §entropy). By Step 1, $r=(\rho_{1}\circ\mathrm{pr}_{1})(\rho_{2}\circ\mathrm{pr}_{2})$ is a density of $\mu\boxtimes\nu$. For nonnegative reals $s,t$ we have $\phi(st)=\phi(s)t+s\phi(t)$: if $s=0$ or $t=0$ then $st=0$ and both sides vanish since $\phi(0)=0$; otherwise $0<st$ by claim 5 of Elementary Order Arithmetic in an Ordered Field and $\phi(st)=st\log(st)=st(\log s+\log t)=(s\log s)t+s(t\log t)$ by The Natural Logarithm. Hence

The functions $\phi\circ\rho_{1}$, $\phi\circ\rho_{2}$ are Borel (The Function $s\log s$: Continuity, Young's Inequality and Lower Bounds, with the Elementary Bounds for the Exponential and the Logarithm §continuous) and integrable, so by Step 3 and claim 2 of Linearity and Monotonicity of the Lebesgue Integral, $\phi\circ r$ is integrable with $\int\phi\circ r\,d\lambda_{q+p}=\mathrm{Ent}(\mu)+\mathrm{Ent}(\nu)$. Thus $\mu\boxtimes\nu$ has finite entropy and $\mathrm{Ent}(\mu\boxtimes\nu)=\mathrm{Ent}(\mu)+\mathrm{Ent}(\nu)$; by Step 2, $M_{2}(\mu\boxtimes\nu)=M_{2}(\mu)+M_{2}(\nu)<\infty$, so $\mu\boxtimes\nu\in\mathcal{P}_{2}^{\mathrm{Ent}}(\mathbb{R}^{q+p})$.

Step 5 (Gaussian products). We show $\gamma_{q}\boxtimes\gamma_{p}=\gamma_{q+p}$ and $\log c_{q+p}=\log c_{q}+\log c_{p}$. By Step 2 and claim 1 of Basic Properties of the Exponential Function, $\psi_{q}(\mathrm{pr}_{1}(z))\psi_{p}(\mathrm{pr}_{2}(z))=\exp\bigl(-\tfrac12\lVert\mathrm{pr}_{1}(z)\rVert^{2}-\tfrac12\lVert\mathrm{pr}_{2}(z)\rVert^{2}\bigr)=\psi_{q+p}(z)$, so by Step 1 the function $(g_{q}\circ\mathrm{pr}_{1})(g_{p}\circ\mathrm{pr}_{2})=c_{q}c_{p}\psi_{q+p}$ is a density of the probability measure $\gamma_{q}\boxtimes\gamma_{p}$, and $\int_{\mathbb{R}^{q+p}}c_{q}c_{p}\psi_{q+p}\,d\lambda_{q+p}=(\gamma_{q}\boxtimes\gamma_{p})(\mathbb{R}^{q+p})=1$. As $c_{q}c_{p}$ is positive (claim 5 of Elementary Order Arithmetic in an Ordered Field), the uniqueness in claim 1 of The Gaussian Smoothing Weight: Normalization, Derivatives, Exponential Tilting, Moments, and First-Order Remainder gives $c_{q}c_{p}=c_{q+p}$. Hence $g_{q+p}$ is a density of $\gamma_{q}\boxtimes\gamma_{p}$ as well as of $\gamma_{q+p}$, so both measures equal $A\mapsto\int\mathbf{1}_{A}g_{q+p}\,d\lambda_{q+p}$; and $\log c_{q+p}=\log c_{q}+\log c_{p}$ by The Natural Logarithm.

Step 6 (Splitting the variational functional). Let $h:\mathbb{R}^{q}\to\mathbb{R}$ and $k:\mathbb{R}^{p}\to\mathbb{R}$ be bounded Borel with bounds $M_{h},M_{k}$, and let $F=h\circ\mathrm{pr}_{1}+k\circ\mathrm{pr}_{2}$. Then $F$ is Borel (Probability Measures on Euclidean Space and Random Vectors: Standing Notation §borel-maps, claim 2 of Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions) and bounded by $M_{h}+M_{k}$ (claim 5 of Properties of the Absolute Value in an Ordered Field and the compatibility of the order with addition in Ordered Field). For $Q\in\mathcal{P}(\mathbb{R}^{q+p})$ we claim $\Lambda^{q+p}_{F}(Q)=\Lambda^{q}_{h}(Q_{1})+\Lambda^{p}_{k}(Q_{2})$. Bounded Borel functions are integrable against probability measures (Probability Measures on Euclidean Space and Random Vectors: Standing Notation §measures), so claim 2 of Linearity and Monotonicity of the Lebesgue Integral and Probability Measures on Euclidean Space and Random Vectors: Standing Notation §pushforward give $\int F\,dQ=\int h\,dQ_{1}+\int k\,dQ_{2}$. By Relative Entropy on Euclidean Space: the Gibbs Inequality, a Variational Criterion for Finite Relative Entropy, and Closed Sublevel Sets under Weak Convergence §functional, $\exp\circ F$, $\exp\circ h$ and $\exp\circ k$ are Borel and bounded, and their integrals against $\gamma_{q+p},\gamma_{q},\gamma_{p}$ are positive reals $a,b,b'$. By claim 1 of Basic Properties of the Exponential Function, $(\exp\circ F)(\iota(u,v))=\exp(h(u))\exp(k(v))$. By Step 5, Pairs of Euclidean Points: Coordinate Projections, Pairings, the Product Measure on a Euclidean Space, Borel Norm Functions and Finite Sets §product, the Tonelli part of Tonelli and Fubini Theorems and claim 1 of Linearity and Monotonicity of the Lebesgue Integral (constants $\exp(h(u))$ and $b'$),

Hence $\log a=\log b+\log b'$ by The Natural Logarithm, and subtracting from $\int F\,dQ=\int h\,dQ_{1}+\int k\,dQ_{2}$ gives the claim.

Step 7 (Claim 2). Let $Q\in\mathcal{P}_{2}^{\mathrm{Ent}}(\mathbb{R}^{q+p})$. By (E) in dimension $q+p$, $Q$ has finite relative entropy with respect to $\gamma_{q+p}$; put $H=H(Q\,|\,\gamma_{q+p})$, so $\mathrm{Ent}(Q)=H+\log c_{q+p}-\tfrac12M_{2}(Q)$. Let $h:\mathbb{R}^{q}\to\mathbb{R}$ and $k:\mathbb{R}^{p}\to\mathbb{R}$ be bounded Lipschitz, hence bounded Borel. By Step 6 and Relative Entropy on Euclidean Space: the Gibbs Inequality, a Variational Criterion for Finite Relative Entropy, and Closed Sublevel Sets under Weak Convergence §gibbs (for $m=q+p$, $\gamma=\gamma_{q+p}$, and the bounded Borel $F$ of Step 6),

Fix $k$ (for instance $k=0$). Since (5) holds for every bounded Lipschitz $h$, Relative Entropy on Euclidean Space: the Gibbs Inequality, a Variational Criterion for Finite Relative Entropy, and Closed Sublevel Sets under Weak Convergence §criterion (for $m=q$, $\gamma=\gamma_{q}$) shows that $Q_{1}$ has finite relative entropy with respect to $\gamma_{q}$ and $H(Q_{1}\,|\,\gamma_{q})\le H-\Lambda^{p}_{k}(Q_{2})$. Hence $\Lambda^{p}_{k}(Q_{2})\le H-H(Q_{1}\,|\,\gamma_{q})$ for every bounded Lipschitz $k$, and Relative Entropy on Euclidean Space: the Gibbs Inequality, a Variational Criterion for Finite Relative Entropy, and Closed Sublevel Sets under Weak Convergence §criterion (for $m=p$, $\gamma=\gamma_{p}$) shows that $Q_{2}$ has finite relative entropy with respect to $\gamma_{p}$ and $H(Q_{1}\,|\,\gamma_{q})+H(Q_{2}\,|\,\gamma_{p})\le H$. By Step 2, $Q_{1}\in\mathcal{P}_{2}(\mathbb{R}^{q})$, $Q_{2}\in\mathcal{P}_{2}(\mathbb{R}^{p})$ and $M_{2}(Q)=M_{2}(Q_{1})+M_{2}(Q_{2})$; so by (E) in dimensions $q$ and $p$ both marginals have finite entropy, belong to $\mathcal{P}_{2}^{\mathrm{Ent}}$, and, with Step 5,

Step 8 (Mixtures). Let $\mu_{j}$ and $t_{j}$ be as in claim 3, and let $\rho_{j}$ be a density of $\mu_{j}$ with respect to $\lambda_{q}$ (The Entropy of a Probability Measure on Euclidean Space §entropy). Put $\bar\rho=\sum_{j=1}^{n}t_{j}\rho_{j}$, Borel by claim 2 of Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions and nonnegative by claim 5 of Elementary Arithmetic in an Ordered Field and claim 5 of Properties of Finite Sums; let $\nu_{\bar\rho}$ be the measure with density $\bar\rho$ with respect to $\lambda_{q}$ (claim 3 of Image Measures, Measures with Densities, and Change of Variables). Let $f:\mathbb{R}^{q}\to\mathbb{R}$ be Borel and integrable with respect to every $\mu_{j}$. By claim 3 of Image Measures, Measures with Densities, and Change of Variables, each $f\rho_{j}$ is integrable with respect to $\lambda_{q}$ with $\int f\rho_{j}\,d\lambda_{q}=\int f\,d\mu_{j}$; since $f\bar\rho=\sum_{j=1}^{n}t_{j}f\rho_{j}$ pointwise (claim 3 of Properties of Finite Sums), Linearity of the Lebesgue Integral over a Finite Sum of Integrable Functions §integrable and Linearity of the Lebesgue Integral over a Finite Sum of Integrable Functions §linear show that $f\bar\rho$ is integrable with integral $\sum_{j=1}^{n}t_{j}\int f\,d\mu_{j}$; and claim 3 of Image Measures, Measures with Densities, and Change of Variables again gives

For $B\in\mathcal{B}(\mathbb{R}^{q})$, (6) with the bounded Borel $f=\mathbf{1}_{B}$, integrable with respect to every $\mu_{j}$ by Probability Measures on Euclidean Space and Random Vectors: Standing Notation §measures, and The Integral of an Indicator Function is the Measure of the Set give $\nu_{\bar\rho}(B)=\sum_{j=1}^{n}t_{j}\mu_{j}(B)=\bar\mu(B)$. Hence $\bar\mu=\nu_{\bar\rho}$ is a measure, $\bar\mu(\mathbb{R}^{q})=\sum_{j=1}^{n}t_{j}=1$, so $\bar\mu\in\mathcal{P}(\mathbb{R}^{q})$; and (6) holds with $\bar\mu$ in place of $\nu_{\bar\rho}$.

Step 9 (Claim 3). The nonnegative Borel function $x\mapsto\lVert x\rVert^{2}$ is integrable with respect to each $\mu_{j}$, as $M_{2}(\mu_{j})<\infty$; so (6) gives $M_{2}(\bar\mu)=\sum_{j=1}^{n}t_{j}M_{2}(\mu_{j})<\infty$ and $\bar\mu\in\mathcal{P}_{2}(\mathbb{R}^{q})$. By (E) in dimension $q$, each $\mu_{j}$ has finite relative entropy with respect to $\gamma_{q}$ and $\mathrm{Ent}(\mu_{j})=H(\mu_{j}\,|\,\gamma_{q})+\log c_{q}-\tfrac12M_{2}(\mu_{j})$. Let $h:\mathbb{R}^{q}\to\mathbb{R}$ be bounded Lipschitz, hence bounded Borel, and let $L=\log\int\exp\circ h\,d\gamma_{q}$. By (6) with $f=h$, claims 2 and 3 of Properties of Finite Sums with $\sum_{j=1}^{n}t_{j}=1$, Relative Entropy on Euclidean Space: the Gibbs Inequality, a Variational Criterion for Finite Relative Entropy, and Closed Sublevel Sets under Weak Convergence §gibbs, claim 5 of Elementary Arithmetic in an Ordered Field ($0\le t_{j}$) and claim 1 of Comparison and Absolute Value Bounds for Finite Sums of Real Numbers,

By Relative Entropy on Euclidean Space: the Gibbs Inequality, a Variational Criterion for Finite Relative Entropy, and Closed Sublevel Sets under Weak Convergence §criterion, $\bar\mu$ has finite relative entropy with respect to $\gamma_{q}$ and $H(\bar\mu\,|\,\gamma_{q})\le\sum_{j=1}^{n}t_{j}H(\mu_{j}\,|\,\gamma_{q})$. By (E), $\bar\mu$ has finite entropy, so $\bar\mu\in\mathcal{P}_{2}^{\mathrm{Ent}}(\mathbb{R}^{q})$, and by the same finite-sum claims