Fix the holdout accuracy for a random forest
Your task
An AI fit a random forest on a training split and then scored it with model.predict(X_train) against y_train. It reported 99.2% accuracy and recommended shipping, but that number measures how well the model remembers its training data, not how it does on new data. The fix scores only rows the model never trained on.
An assistant wrote holdout_accuracy(y_true, y_pred, in_train). It is in the editor. It has a bug. Find it and fix it.
Input and output:
y_true,y_predandin_train: for each row, the real label, the model's prediction, and whether the model was trained on that row.- Return the accuracy as a float, or -1.
Rules the tests check:
- If the lists differ in length, use only the first n rows, where n is the shortest length.
- Score only the rows where
in_trainis false. - Accuracy is the number of scored rows where the prediction equals the real label, divided by the number of scored rows. Labels are compared exactly as given.
- If no row is scored, return -1.
- Answers within 0.000001 of the expected value pass.
Examples
holdout_accuracy(y_true: list[int], y_pred: list[int], in_train: list[bool]) -> floatExample 1
Input
holdout_accuracy([1, 0, 1, 0], [1, 0, 1, 1], [True, True, False, False])Output
0.5Example 2
Input
holdout_accuracy([1, 1], [1, 1], [True, True])Output
-1