Model predicts retention calls perfectly
Review this Python feature-engineering step for a churn model.
The model hits 0.99 AUC on a proper held-out split. Why is that a red flag?
What a strong answer looks like
Separate real bugs from style. Rank issues by severity, point at the root cause rather than the symptom, and suggest a concrete fix, specific and kind.
0:00 of about 18 min
Mark a line and say what kind of problem it is.0 findings
1import pandas as pd
2
3def build_features(df: pd.DataFrame) -> pd.DataFrame:
4 # df has: customer_id, monthly_spend, tenure, churned (0/1)
5 df = df.copy()
6 df["spend_per_tenure"] = df["monthly_spend"] / (df["tenure"] + 1)
7 # operations team flags churners for a retention call
8 df["got_retention_call"] = df["churned"] # 1 if they were called
9 df["high_value"] = (df["monthly_spend"] > 200).astype(int)
10 feature_cols = ["spend_per_tenure", "got_retention_call", "high_value"]
11 return df[feature_cols], df["churned"]
Which questions mattered is sealed until you submit. Telling you now would just be handing over the edge cases.
Run or narrate your approach, then ask the coach.