Caratheodory's Extension Theorem
From MIT 6.436 lecture notes. Pre-requisite notes. Pre-requisite topics:
If we take any old collection \(C\) of subsets of \(\Omega\), and we define \(\mathbb{P}_C\) for \(C\), then it is not the case that \(\mathbb{P}_C\) can be uniquely extended to some \(\mathbb{P}\) on \(\sigma(C)\). However, this is possible if \(C\) is an algebra (see below).
It turns out that this is also true if \(C\) is a \(p\) -system (see independence note for def. of \(p\) -system).
1. theorem
Let \(\mathcal{F}_0\) be an algebra on \(\Omega\). And let \(\mathcal{F} = \sigma(\mathcal{F}_0)\) be the \(\sigma\) -algebra that \(\mathcal{F}_0\) generates. Then, let \(\mathbb{P}_0\) be a function \(\mathbb{P}_0 : \mathcal{F}_0 \rightarrow [0,1]\) that satisfies (i) \(\mathbb{P}_0(\Omega) = 1\) and (ii) \(\mathbb{P}_0\) is countably additive on \(\mathcal{F}_0\). THEN \(\mathbb{P}_0\) can be extended uniquely to a probability measure \(\mathbb{P}\) on \((\Omega, \mathcal{F})\). That is, there is a unique \(\mathbb{P}\) such that \(\mathbb{P}(A) = \mathbb{P}_0(A)\) for all \(A\in \mathcal{F}_0\).
2. using the theorem
Usually the theorem is used in the following context: we want to assign a probability measure to some \((\Omega, \mathcal{F})\). To do this, we first define a \(\mathcal{F}_0 \subset \mathcal{F}\). We show that \(\mathcal{F}_0\) is an algebra and that \(\sigma(\mathcal{F}_0) = \mathcal{F}\). Then, we define the probabilities that we want to define, but only for \(\mathcal{F}_0\). If we can show that our probability measure as defined is countably additive, then we can extend it uniquely to all of \(\mathcal{F}\).
The notes here give examples of how to do this for \(\Omega = \{0,1\}^\infty\), which is the space of infinite-length sequences of coin tosses and \(\Omega = [0,1]\).