TheoremBase

Relative Entropy of the Image under a Bimeasurable Bijection that Moves the Reference Measure by a Positive Density

If a bimeasurable bijection T carries the reference measure to a measure with a positive density g, then T carries a density f of a measure to the density (f∘T^{-1})g, and the relative entropy changes by the integral of log g∘T.

Statement

In the settings of The Real Numbers: Standing Notation and Background and Measure Spaces and the Lebesgue Integral: Standing Notation, the measure space of the latter being instantiated at each use by the measure space named, let (S,S)(S,\mathcal{S}) be a measurable space and let γ\gamma and ν\nu be probability measures on it. Densities with respect to γ\gamma are those of The Radon-Nikodym Theorem for a Finite Measure and a Sigma-Finite Measure, and Uniqueness of Densities; finite relative entropy with respect to γ\gamma and H(⋅ ∣ γ)H(\cdot\,|\,\gamma) are those of that definition; log⁡\log is the natural logarithm. Let T:S→ST:S\to S be a bijection such that TT and its inverse map T−1T^{-1} are measurable with respect to S\mathcal{S}; for B⊆SB\subseteq S the preimage of BB under TT is then the image of BB under T−1T^{-1}, and both are written T−1(B)T^{-1}(B). For a probability measure λ\lambda on (S,S)(S,\mathcal{S}) let T#λT_{\#}\lambda be the image measure B↦λ(T−1(B))B\mapsto\lambda(T^{-1}(B)) of claim 1 of that lemma. Let g:S→Rg:S\to\mathbb{R} be measurable with 0<g(s)0<g(s) for every s∈Ss\in S, and suppose that gg is a density of T#γT_{\#}\gamma with respect to γ\gamma.

1. (Density of the image) If ff is a density of ν\nu with respect to γ\gamma, then (f∘T−1) g(f\circ T^{-1})\,g is a density of T#νT_{\#}\nu with respect to γ\gamma.

2. (Relative entropy of the image) Suppose that ν\nu has finite relative entropy with respect to γ\gamma and that the function s↦log⁡g(T(s))s\mapsto\log g(T(s)) is integrable with respect to ν\nu. Then T#νT_{\#}\nu has finite relative entropy with respect to γ\gamma, and

H(T#ν ∣ γ)=H(ν ∣ γ)+∫Slog⁡g(T(s)) ν(ds).H(T_{\#}\nu\,|\,\gamma)=H(\nu\,|\,\gamma)+\int_{S}\log g(T(s))\,\nu(ds).

Proofs

Log in to submit a proof.

Loading...

Citations

Loading…

Dependencies

Loading…

Related

0 relations

Curated associations between results. These are editable and subjective — they do not replace the dependency graph, which is derived from the references in the text.

No relations recorded yet.

Comments

Log in to comment.

Loading…