Fix the metrics for a rare disease classifier
Your task
An AI evaluated a rare disease classifier, with about 1.5% positive cases, using accuracy alone. It printed 98.6% and called the model "highly performant", but a model that says "negative" to everyone scores 98.5%. The fix reports precision, recall and the majority baseline next to accuracy.
An assistant wrote evaluate_rare_class(y_true, y_pred). It is in the editor. It has a bug. Find it and fix it.
Input and output:
y_true: the real labels.y_pred: the model's predictions, at the same index.- Return a list of four numbers: accuracy, precision, recall and baseline.
Rules the tests check:
- If the lists differ in length, use only the first n pairs, where n is the shorter length. If n is 0, return
[0, 0, 0, 0]. - A label is positive only when it is exactly 1. Every other value is negative.
- Accuracy: the share of pairs where the prediction and the real label are both positive or both negative.
- Precision: of the pairs predicted positive, the share that are really positive. 0 if nothing is predicted positive.
- Recall: of the pairs that are really positive, the share predicted positive. 0 if there are no real positives.
- Baseline: the accuracy of always predicting the larger class, which is the larger of the positive and negative counts, divided by n.
- Answers within 0.000001 of the expected values pass.
Examples
evaluate_rare_class(y_true: list[int], y_pred: list[int]) -> list[float]Example 1
Input
evaluate_rare_class([1, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0])Output
[0.9, 0, 0, 0.9]Example 2
Input
evaluate_rare_class([1, 1, 0, 0, 0, 0, 0, 0, 0, 0], [1, 0, 0, 0, 0, 0, 0, 0, 0, 1])Output
[0.8, 0.5, 0.5, 0.8]Example 3
Input
evaluate_rare_class([1, 0, 1, 0], [1, 0, 1, 0])Output
[1, 1, 1, 0.5]