statistical model

Let \(\Omega\) be an outcome space (see probability space). Then, a statistical model is the pair: \[ (\Omega, (P_{\theta})_{\theta\in\Theta}) \] where \(P_{\theta}\) is a probability distribution on \(\Omega\) for each \(\theta\in \Theta\)

If there is some \(P_{\eta}\) generating our data, then \(\eta\) is called the true parameter and the goal of statistics is to estimate \(\eta\).

• a model is called identifiable if \(P_{\theta_1} = P_{\theta_2} \Rightarrow \theta_1 = \theta_2\)
  • That is, \(\theta \mapsto P_\theta\) is injective
  • So if we take enough samples to distinguish \(P_{\theta_1}\) and \(P_{\theta_2}\), we can know that their true parameters are different
  • otherwise the situation would be hopeless, we could take as many samples as we like, and there would still be fundamental ambiguity as to what the true parameter is

• binomial: \((\{0,1\}, (\text{Ber}(p))_{p\in(0,\infty)})\)

• parametric vs non-parametric models