Fix the holdout accuracy for a random forest

Review the AI's codeEasy8 min

Your task

An AI fit a random forest on a training split and then scored it with model.predict(X_train) against y_train. It reported 99.2% accuracy and recommended shipping, but that number measures how well the model remembers its training data, not how it does on new data. The fix scores only rows the model never trained on.

An assistant wrote holdout_accuracy(y_true, y_pred, in_train). It is in the editor. It has a bug. Find it and fix it.

Input and output:

  • y_true, y_pred and in_train: for each row, the real label, the model's prediction, and whether the model was trained on that row.
  • Return the accuracy as a float, or -1.

Rules the tests check:

  • If the lists differ in length, use only the first n rows, where n is the shortest length.
  • Score only the rows where in_train is false.
  • Accuracy is the number of scored rows where the prediction equals the real label, divided by the number of scored rows. Labels are compared exactly as given.
  • If no row is scored, return -1.
  • Answers within 0.000001 of the expected value pass.

Examples

holdout_accuracy(y_true: list[int], y_pred: list[int], in_train: list[bool]) -> float
Example 1
Inputholdout_accuracy([1, 0, 1, 0], [1, 0, 1, 1], [True, True, False, False])
Output0.5
Example 2
Inputholdout_accuracy([1, 1], [1, 1], [True, True])
Output-1