receiver operating characteristic

\[\begin{align*} P_D &= \mathbb{P}(\hat{H}(\mathsf{y}) = H_1 \mid \mathsf{H} = H_1) = \int_{\mathcal{Y}_1} p_{\mathsf{y} \mid \mathsf{H}} (\mathbf{y} \mid H_1) d\mathbf{y}\\ P_F &= \mathbb{P}(\hat{H}(\mathsf{y}) = H_1 \mid \mathsf{H} = H_0) = \int_{\mathcal{Y}_0} p_{\mathsf{y} \mid \mathsf{H}} (\mathbf{y} \mid H_0) d\mathbf{y} \end{align*}\] where \(\mathcal{Y}_i\) is the set of all \(\mathbf{y}\) such that \(\hat{H}(\mathbf{y}) = H_i\)

Then

• \(P_D\) is called the "probability of detection". I think of it as:
  • "Imagine that the sender is sending a bit. The bit can be corrupted. For all the true 1's that we receive, what is the fraction that we will end up detecting?"
  • or "In the case that hypothesis 1 is true, what is the probability that it will produce an observation that we will classify as being from hypothesis 1?"
  • or "if a person has a disease, what is the probability that they will present symptoms so that our decision rule will give them a positive diagnosis"
• \(P_F\) is called the "false alarm probability"
  • "For all the true 0's that we receive, what is the fraction that we will end up incorrectly identifying as 1's?"
  • "In the case that hypothesis 0 is true, what is the probability that it will produce an observation that we will classify as being from hypothesis 1?"

For a binary decision rule, our decision rule, e.g. likelihood ratio test (LRT), may assign a quantity to each \(\mathbf{y}\). Then, we can make our decision based on whether that value is below or above a certain threshold \(\eta\). So \(LRT(\mathbf{y}) \underset{H_2}{\overset{H_1}{\gtreqless}} \eta\). Then, for each choice of \(\eta\), we will have a different \((P_D, P_F)\). When \(\eta\rightarrow \infty\), we classify nothing as \(H_1\), so we detect nothing, but we also have no false positives. When \(\eta \rightarrow 0\), we will classify everything as \(H_1\), so we will have all false positives, but we will also detect everything.

All the points in between trace out the ROC curve.

• \(P_F\) is often called the size of the test (think: you get large \(P_F\) by making your detection area "large" and covering more observations)
• \(P_D\) is often called the power of the test (think: a powerful test is one that will say capture as many positive diagnoses as possible)
• \(P_F\) is also called "probability of Type 1 error" – false positive
• "probability of Type 2 error" is \((1-P_D)\) – false negative
• \(P_D\) is also called recall