Precision-recall Curve

What is it

The precision-recall curve is a plot that precisely calibrates the balance between precision and recall, as applied to a data set.

Why it matters

The precision-recall curve helps engineers make decisions in real-world scenarios, for example excluding the most uncertain data points, thus allowing engineers to focus on the data points with the highest probability.

How it works and when it’s used

The average of precision and recall can be plotted as a single point on the precision-recall curve. This is known as the F1 score, and it represents an equal trade-off between true positive rate (TPR) and false positive rate (FPR).

The curve can be recalibrated in one direction or another, to include more positive results, or fewer, depending on priorities. The threshold can be changed, adjusting the trade-off between the true positive rate and false positive rate. The recalibrated curve will now exclude those positives the model deems less probable.

When minimizing false negatives is crucial (such as serious damage to railway tracks), weighting of recall over precision can be doubled using the F2 score. (F3 triples the weighting, and so on.) With the adjusted threshold, the curve now includes more data points.

In contrast, in less critical cases, when you wish to prioritise precision and minimise false positives – for example, discarded rails, ballast bags or graffiti on the trackside – use the F0.5 score. The curve now includes fewer data points.