Erroneous precision recall curve and PR-AUC for null classifier

[longjp]

If a classifier predicts all same value, then precision-recall curve should be horizontal line at y= # positivies / # observations. PR-AUC should be = # positivies / # observations (integrating constant from 0 to 1). However this is not what the pr_curve or pr_auc function returns.

library(yardstick)
library(ggplot2)
library(tidyr)

head(two_class_example)


## real pr curve
pr <- pr_curve(two_class_example, truth, Class1)
pr %>%
  ggplot(aes(x = recall, y = precision)) +
  geom_path() +
  coord_equal() +
  theme_bw()
pr_auc(two_class_example, truth, Class1)




## pr curve with random predictions, looks fine
two_class_example$noisy_pred <- rnorm(nrow(two_class_example))
pr <- pr_curve(two_class_example, truth, noisy_pred)
pr %>%
  ggplot(aes(x = recall, y = precision)) +
  geom_path() +
  coord_equal() +
  theme_bw() +
  ylim(c(0,1))
pr_auc(two_class_example, truth, noisy_pred)
## result is near 0.5 because class1 and class2 roughly balanced


## pr curve with always 0 predictions, WRONG result
two_class_example$fake_pred <- 0
pr <- pr_curve(two_class_example, truth, fake_pred)
pr %>%
  ggplot(aes(x = recall, y = precision)) +
  geom_path() +
  coord_equal() +
  theme_bw() +
  ylim(c(0,1))
pr_auc(two_class_example, truth, fake_pred)
## result is 0.758, way too high

erroneous-pr-curve.pdf