# estimator

From 18.650 lectures:

- given \(X_1,...,X_n\) independent samples from \(P_{\theta}\) for an unknown \(\theta\) (see statistical model), an
*estimator*of \(\theta\) is any function of \(X_1,\dots,X_n\) that does not depend on \(\theta\)

## 1. consistent

- a sequence of estimators \(\hat{\theta}_n\) is strongly
*consistent*if \(\theta_n \rightarrow \theta\) almost surely as \(n\rightarrow \infty\)- it is weakly consistent if the sequence converges in probability to \(\theta\)

## 2. risk, variance, and bias

- for a given \(n\), the
*bias*of an estimator is \(\mathbb{E}[\hat{\theta}_n] - \theta\)- is the average estimator at \(\theta\)? NOTE: it could be that none of the estimators are at \(\theta\), but the expectation is still at \(\theta\).

- the variance of \(\hat{\theta}\) is: \(\mathbb{E}[\hat{\theta}^2] - \mathbb{E}[\hat{\theta}]^2\)
- what is the spread of the estimators?

- the
*risk*of an estimator is \(\mathbb{E}[|\hat{\theta}-\theta|^2]\)- how far is the estimator from \(\theta\) on average?

- it turns out: \(\text{risk} = \text{bias}^2 + \text{variance}\). To see this, expand RHS terms and use linearity of expectation

## 3. example

See example in confidence interval note.