[FAQ] #484

Course

machine-learning-zoomcamp

Question

Why use ROC AUC instead of accuracy?

Answer

Short answer: accuracy depends on one threshold and on class balance. ROC AUC depends on neither. It measures how well the model ranks positives above negatives, across every possible threshold.

1. Accuracy depends on one fixed threshold

A logistic regression outputs a probability, such as 0.63. Accuracy only makes sense after you choose a cut-off, usually 0.5, and turn those probabilities into yes/no predictions. Change the threshold and accuracy changes too.

ROC AUC sweeps through all thresholds and summarizes the whole trade-off in one number.

2. Accuracy can be misleading with imbalanced classes

The lead-scoring data is 57.7% converted (2,883 of 5,000 leads). A model that predicts "converted" for everyone gets about 58% accuracy without learning anything.

The problem gets worse with more imbalance. If only 1% of cases are positive, always predicting "negative" gives 99% accuracy.

ROC AUC is not fooled this way. A constant prediction always scores 0.5, which is random-guess level, no matter how the classes are split.

3. AUC has a clear meaning

AUC = the probability that a randomly chosen positive example gets a higher score than a randomly chosen negative one.

That's why Question 1 can use a raw feature such as lead_score as a "prediction". AUC only cares about ordering, so the feature does not need to be a probability.

Checklist

• I have searched existing FAQs and this question is not already answered
• The answer provides accurate, helpful information
• I have included any relevant code examples or links