Accuracy misleads on imbalanced data
Review this Python fraud-detection evaluation.
The model prints 99.51% accuracy. Should it ship?
What a strong answer looks like
Separate real bugs from style. Rank issues by severity, point at the root cause rather than the symptom, and suggest a concrete fix, specific and kind.
0:00 of about 16 min
Mark a line and say what kind of problem it is.0 findings
1import numpy as np
2from sklearn.ensemble import RandomForestClassifier
3from sklearn.metrics import accuracy_score
4
5# y: 0 = legit, 1 = fraud; about 0.5% of rows are fraud
6def evaluate(X_tr, y_tr, X_te, y_te):
7 clf = RandomForestClassifier(n_estimators=200, random_state=0)
8 clf.fit(X_tr, y_tr)
9 preds = clf.predict(X_te)
10 acc = accuracy_score(y_te, preds)
11 print(f"accuracy: {acc:.4f}") # prints 0.9951
12 if acc > 0.99:
13 print("shipping it")
14 return acc
Which questions mattered is sealed until you submit. Telling you now would just be handing over the edge cases.
Run or narrate your approach, then ask the coach.