[github-actions]

✨ FAQ NEW

Course: machine-learning-zoomcamp
Section: module-4 (This directly belongs to the classification evaluation metrics topic in Module 4, which covers accuracy, ROC curves, and ROC AUC.)
Related Issue: #484

Question

Why use ROC AUC instead of accuracy?

Decision Rationale

The proposal explains why ROC AUC is preferred over accuracy (threshold dependence and class-imbalance effects) and provides an interpretation of AUC. None of the existing FAQs cover this specific “ROC AUC vs accuracy” justification with the same points.

Placement Details

• Section ID: module-4
• Sort Order: End of section
• Filename Slug: why-use-roc-auc-instead-of-accuracy

⚠️ Warnings

• Order set to -1; appended to end of module-4 section.

---

🤖 Generated by FAQ Bot

Closes #484

[saeidsoheily]

Both Accuracy and ROC AUC evaluate classification models, but they measure different aspects of performance.

Accuracy: Percentage of correct predictions, Depends on a classification threshold (usually 0.5), Can be misleading in imbalanced data

ROC AUC: Ability to distinguish positive and negative classes, Evaluates performance across all thresholds, Often more informative, though not always for imbalanced data

Example: Imbalanced dataset
Imagine a dataset with 1,000 customers:

• 950 customers will not churn (95%).
• 50 customers will churn (5%).
  A model predicts that nobody will churn.

Its accuracy is 950/1000=0.95
Even though the accuracy is 95%, the model fails to identify any customers who will churn.

While its ROC AUC would be 0.5, assuming the model assigns everyone the same prediction score. This indicates no ability to distinguish churners from non-churners.

Understanding ROC AUC
ROC AUC can be interpreted as the probability that the model assigns a higher prediction score to a randomly selected positive example than to a randomly selected negative example (with ties counted as half).

• AUC = 1.0: Perfect ranking.
• AUC = 0.5: Random-level ranking.
• AUC below 0.5: Worse-than-random ranking.

Key takeaway for ML Zoomcamp: ROC AUC is particularly useful when you want to compare models based on their ability to rank positive cases above negative cases, without choosing a classification threshold first. Accuracy is still useful when your goal is to measure correct classifications at a specific threshold.

For severely imbalanced datasets, also consider Precision, Recall, F1-score, and PR AUC, especially when detecting the positive class is your main objective.