precision recall curve - no skill model

44 views Asked by At

I am using this tutorial for the precision-recall curves and it is great: https://machinelearningmastery.com/roc-curves-and-precision-recall-curves-for-classification-in-python/

Though, I cannot understand the rational to have a horizontal line for the no skill model where the value of x-axis corresponds to the class imbalance. Could someone explain why this is the case and how to get the (x, y) coordinates to plot?

1

There are 1 answers

1
cheddar On

Probabilistic definition will be helpful to understand: https://en.wikipedia.org/wiki/Precision_and_recall#No-Skill_Classifiers

For a no skill classifier, The precision only depends on actual positive ratio in the dataset while the recall only depends on the positive ratio in the predictions, which is dependent on threshold picked. Since no skill classifier predicts every instance as positive at 50% chance, if threshold selected is less than 0.5, then all predictions are positive (prob=0.5>threshold)and recall is 1. Otherwise, all predictions are negative and recall is 0. Therefore, precision recall curve is a horizontal line for no skill classifier.

By (x,y) I assume you mean precision and recall values for a model. You can get them from precision_recall_curve() in sklearn.metrics