Fix the per-fold scaler in a churn pipeline
Your task
An AI wrote a scikit-learn churn pipeline that calls StandardScaler().fit_transform(X) on the whole dataset and then cross-validates. Cross-validation reported 0.94 ROC-AUC, and production was far worse. Your team checks the numbers by standardizing each validation fold the leak-free way, one feature at a time.
An assistant wrote standardize_within_fold(values, fold_ids, fold). It is in the editor. It has a bug. Find it and fix it.
Input and output:
values: one feature's value for each row.fold_ids: each row's fold number, at the same index.fold: the fold being validated. Its rows are the held-out rows, and every other row is a training row.- Return the standardized held-out values, in input order.
Rules the tests check:
- If the lists differ in length, use only the first n rows, where n is the shorter length.
- Compute the mean and the standard deviation from the training rows only. Use the population standard deviation: divide by the number of training rows, not that number minus 1.
- If there are no training rows, return an empty list.
- If the standard deviation is 0, use 1 instead.
- Each held-out value becomes its value minus the mean, divided by the standard deviation.
- Answers within 0.000001 of the expected values pass.
Examples
standardize_within_fold(values: list[float], fold_ids: list[int], fold: int) -> list[float]Example 1
Input
standardize_within_fold([2, 4, 6, 8], [0, 0, 1, 1], 1)Output
[3, 5]Example 2
Input
standardize_within_fold([2, 4, 6, 8], [0, 0, 1, 1], 0)Output
[-5, -3]Example 3
Input
standardize_within_fold([2, 4, 6, 8], [0, 0, 0, 0], 0)Output
[]