conditional probability
From the 6.436 lecture notes here.
Prerequisite concepts: probability space, Measures and Probability Measures, Sigma Field
1. definition: conditional probability
Consider a probability space \((\Omega, \mathcal{F}, \mathbb{P})\) and an event \(B\in \mathcal{F}\) with \(\mathbb{P}(B) > 0\). Then for every \(\mathcal{A}\), the probability that \(A\) occurs, given that \(B\) has occured is: \[ \mathbb{P}(A \mid B) = \frac{\mathbb{P}(A \cup B)}{\mathbb{P}(B)} \]
2. Theorem 1: conditional probability
Let \((\Omega, \mathcal{F}\), \mathbb{P}) be a probability space.
2.1. 1
If \(B\in \mathcal{F}\) is an event with \(\mathbb{P} > 0\), then \(\mathbb{P}(\Omega \mid B) = 1\) and for any sequence \{Ai\} of disjoint events, we have: \[ \mathbb{P}\left(\cup_{i=1}^{\infty} A_i \mid B \right) = \sum_{i=1}^{\infty} \mathbb{P}(A_i \mid B) \] So (see probability space), we have that \(\mathbb{P}_B : \mathcal{F} \rightarrow [0,1]\), defined as \(\mathbb{P}_B = \mathbb{P}(\cdot \mid B)\), is a probability measure on \((\Omega, \mathcal{F})\).
2.2. 2
Let \(A\) be an event. If the events \(B_i\) for \(i \in \mathcal{N}\) form a partition of \(\Omega\) with \(\mathbb{P}(B_i) > 0\) for every \(i\), then: \[ \mathbb{P}(A) = \sum_{i=1}^{\infty} \mathbb{P}(A \mid B_i) \mathbb{P}(B_i) \]
2.3. 3
(Baye's rule) Let \(A\) be an event with \(\mathbb{P}(A) > 0\). If the events \(B_i\), \(i\in \mathbb{N}\), form a partition of \(\Omega\), then for every \(B_i\) \[ \mathbb{P}(B_i \mid A) = \frac{\mathbb{P}(A \mid B_i)\mathbb{P}(B_i)}{\sum_{j=1}^{\infty}\mathbb{P}(A\mid B_j)\mathbb{P}(B_j)} \]
2.4. 4
For any sequence \(\{A_i\}\) of events, \[ \mathbb{P}(\cap_{i=1}^{\infty} A_i) = \mathbb{P}(A_i)\prod_{i=2}^{\infty} \mathbb{P}(A_i \mid A_1 \cap \cdots \cap A_{i-1}) \] provided that each conditional probability is well defined.