ML system design questions.

67 ml system design questions from the bank, ready to practice out loud.

Learn the ideas first · free with an account

All 67 questions

Video recommendation system designDesign an ML system to recommend videos to logged-in users on a streaming service with 200 million users.Training data pipelineSenior–staff+

Monitoring credit-decisioning modelsDesign the monitoring story for a credit-decisioning model that's been in production for six months. What do you alert on?Drift detectionSenior–staff+

Real-time fraud detection systemDesign a fraud-detection ML system that has to decide every transaction in under 100ms at 10K transactions per second.Serving architectureSenior–staff+

Evaluating news summarization systemsYou're building a system that summarizes news articles. Walk me through your evaluation strategy beyond a few automatic metrics.Eval designSenior–staff+

Offline-online performance gapYour offline model evaluation looks great but online metrics show degraded performance. Walk me through the most common reasons.Online/offline gapSenior–staff+

Triggering model retraining signalsDesign a system that decides when to retrain a customer-churn model. What signals would trigger a retrain?Retraining cadenceSenior–staff+

Feature store design at scaleDesign a feature store for a company with a hundred ML use cases. What's non-negotiable in the design?Training data pipelineStaff+–leadership

E-commerce ranking with model fallbackDesign a system where your ML model serves predictions for an e-commerce ranking page. What happens when the model is unavailable?Fallback behaviorSenior–staff+

Serving LLM inference at scaleDesign a system for serving large-language-model inference to a million daily users, balancing cost and latency.Serving architectureSenior–staff+

Evaluating customer support chatbotsDesign an evaluation framework for a chatbot that handles customer support. What does success look like and how do you measure it?Eval designSenior–staff+

Labeling pipeline for image classificationDesign the labeling pipeline for a system that classifies user-uploaded images into 200 categories. Labels are noisy.Training data pipelineSenior–staff+

Drift detection across model fleetWalk me through how you'd build a drift-detection system for a fleet of fifty production ML models.Drift detectionStaff+–leadership

Personalized search ranking systemDesign a search-ranking system that personalizes results per user using behavioral features. Latency budget is 50ms.Serving architectureSenior–staff+

Detecting training-serving skewHow would you instrument a deployed model so that you can detect training-serving skew quickly?Online/offline gapSenior–staff+

Delivery time prediction pipelineDesign the data pipeline for training a delivery-time prediction model that needs to update daily.Training data pipelineSenior–staff+

Safety system for LLM outputsAn LLM-based product feature occasionally produces unsafe outputs. Design the safety system around it.Fallback behaviorSenior–staff+

Heuristic to ML guardrailsDesign an A/B test for replacing a heuristic ranker with an ML model. What guardrails do you set?Eval designSenior–staff+

AI-generated product descriptionsDesign a feature for showing AI-generated product descriptions on an e-commerce site. Quality and cost both matter.Serving architectureSenior–staff+

Automated model promotion systemDesign a system where you continuously evaluate a model against a champion baseline and promote new versions automatically.Retraining cadenceStaff+–leadership

Reproducible experiment infrastructureHow would you architect a system that lets data scientists run dozens of experiments per day against the same offline dataset, reproducibly?Training data pipelineStaff+–leadership

Detecting out-of-distribution inputsWalk me through how you'd detect that a deployed image-classification model is being shown a new class of input it wasn't trained on.Drift detectionSenior–staff+

Evaluating code-completion model usefulnessDesign an evaluation harness for a code-completion model. How do you measure usefulness beyond exact-match accuracy?Eval designSenior–staff+

Serving many small models economicallyDesign the serving infrastructure for a system that runs many small models — say, one per customer — economically.Serving architectureStaff+–leadership

Handling train-serve feature skewYou discover that the feature store generates one set of values offline and a slightly different one online. How do you design around that?Online/offline gapStaff+–leadership

ML-assisted loan application reviewDesign a system where your ML model assists a human reviewer in approving loan applications. What does the integration look like?Fallback behaviorSenior–staff+

Model training with deletion requestsDesign a system to train a model on user data while respecting deletion requests. What does compliance with right-to-be-forgotten look like?Training data pipelineStaff+–leadership

Nightly retraining latencyYour model retrains nightly but stakeholders want it to react to events within minutes. How do you architect a system that supports both?Retraining cadenceSenior–staff+

Evaluating retrieval-augmented generationDesign an evaluation strategy for a retrieval-augmented generation system. What's actually being measured?Eval designSenior–staff+

Privacy-constrained reply suggestionsDesign a system that uses ML to suggest replies in a messaging product, with privacy as a hard constraint.Serving architectureSenior–staff+

Preventing pricing model errorsYour team's pricing model occasionally produces obviously wrong prices. What system around the model prevents customer harm?Fallback behaviorSenior–staff+

Training data for spam classificationYou're building a spam classifier for email. Walk me through how you'd collect and prepare training data, including how you'd handle the class imbalance between spam and legitimate emails.Training data pipelineEntry–mid

Drift detection for price predictionDesign a simple drift detection check for a model that predicts house prices. What metrics would you track on incoming prediction requests to know if the model is seeing different data than it was trained on?Drift detectionEntry–mid

Serving sentiment analysis modelsYou're deploying a sentiment analysis model that scores product reviews. Should you serve it via a REST API, batch processing, or both? Explain your reasoning based on the use case.Serving architectureEntry–mid

Metrics for ticket classificationYou've built a model that tags support tickets by topic. Beyond overall accuracy, what other metrics would you track to ensure it's working well across all ticket categories?Eval designEntry–mid

Offline-online gap in recommendationsYour movie recommendation model has 85% accuracy offline but users aren't clicking the recommendations in production. What are three possible reasons for this gap?Online/offline gapEntry–mid

Retraining frequency for delivery timesYou've deployed a model that predicts restaurant delivery times. How would you decide how often to retrain it—daily, weekly, monthly? What factors influence this decision?Retraining cadenceEntry–mid

Fallback behavior during model downtimeYour product search ranking model goes down during a deployment. What should the search page show to users while the model is unavailable?Fallback behaviorEntry–mid

Train-test split for click-through ratesYou're training a click-through rate model on web traffic data. How would you split your data into train/validation/test sets, and what time period should each cover?Training data pipelineEntry–mid

Detecting job title distribution shiftYour job title classification model was trained on 2023 data. It's now 2025. What simple checks would you implement to detect if the distribution of job titles has changed significantly?Drift detectionEntry–mid

Precomputing vs on-demand profile scoringYou need to serve predictions from a lightweight model that scores user profiles. Would you precompute all scores and cache them, or compute on-demand? What trade-offs matter here?Serving architectureEntry–mid

Evaluating toxic comment detectionYou built a model to detect toxic comments. How would you evaluate whether it works equally well across different types of toxicity (hate speech, harassment, threats)?Eval designEntry–mid

Retraining during market disruptionYour model predicts customer lifetime value and is retrained monthly. Suddenly, a competitor launches and customer behavior changes overnight. How would you adjust your retraining approach to respond faster?Retraining cadenceEntry–mid

Training pipeline with late-arriving eventsDesign a training data pipeline for a recommendation model that needs to incorporate user interactions from the last 24 hours. How do you handle late-arriving events and ensure reproducibility?Training data pipelineMid–senior

Latency optimization for ranking modelsYou're serving a personalized ranking model that takes 300ms to score items, but the page needs to load in under 200ms. Walk me through your architecture options and trade-offs.Serving architectureSenior–staff+

Drift detection with delayed labelsDesign a drift detection system for a loan-approval model where ground truth labels arrive 6-12 months after predictions. What proxy metrics do you monitor in the meantime?Drift detectionSenior–staff+

Evaluating medical imaging across hospitalsYour medical imaging model was trained on data from five hospitals but will deploy to hundreds. Design your evaluation strategy to catch distribution shift before it impacts patient care.Eval designSenior–leadership

Fallback behavior for dynamic pricingDesign the fallback behavior for a dynamic pricing model in a ride-sharing app. The model goes down during Friday evening rush hour—what happens?Fallback behaviorMid–senior

Content moderation serving at scaleYou're building a content moderation system that must review 50K user posts per minute. Design the ML serving architecture, including how you handle spikes to 200K per minute.Serving architectureSenior–staff+

Retraining strategy for search rankingDesign a retraining strategy for a product search ranking model in an e-commerce platform. How do you decide between daily, weekly, or event-triggered retraining?Retraining cadenceMid–senior

Debugging offline-online metric gapYour A/B test shows the new model has better offline metrics but users are engaging less. Walk me through your debugging process to identify what's wrong.Online/offline gapMid–senior

Training imbalanced fraud detection dataDesign a training data pipeline for a fraud detection model where only 0.1% of transactions are fraudulent. How do you handle class imbalance and ensure the model sees recent fraud patterns?Training data pipelineMid–senior

Multi-market recommendation evaluationYou're deploying a recommendation model across 20 countries with different languages and cultures. Design your evaluation framework to ensure quality across all markets.Eval designSenior–staff+

Detecting adversarial gaming of predictionsDesign a system to detect when your ad-click prediction model is being gamed by adversarial actors. What metrics and patterns do you monitor?Drift detectionSenior–staff+

Serving search ranking with rerankingYour search ranking model serves results in under 50ms, but you want to add a large language model reranker. Design the serving architecture to maintain latency SLAs.Serving architectureSenior–staff+

Fallback for inventory allocation failureDesign the fallback strategy for a real-time inventory allocation model in a warehouse management system. The model fails—how do you ensure orders still get fulfilled?Fallback behaviorMid–senior

Multi-source pipeline reliabilityYou're building a training pipeline for a model that uses features from ten different upstream data sources with varying SLAs. How do you design for reliability and debuggability?Training data pipelineSenior–staff+

Retraining signals for conversational AIDesign a retraining system for a conversational AI model. What signals indicate the model needs updating versus just prompt engineering or retrieval improvements?Retraining cadenceSenior–staff+

Bridging attribution gaps in evaluationYour offline metrics use last-click attribution but your online system uses multi-touch attribution. Design an evaluation strategy that bridges this gap.Online/offline gapSenior–staff+

Drift detection across regional modelsDesign a drift detection system for a fleet of region-specific models (50+ regions). How do you distinguish between legitimate regional differences and actual model degradation?Drift detectionStaff+–leadership

Embedding serving architecture at scaleYou're serving embeddings for a semantic search system to 100M users. Design the caching and serving architecture to minimize cost while maintaining relevance.Serving architectureSenior–staff+

Evaluating code generation modelsDesign an evaluation framework for a code-generation model used by developers. How do you measure quality beyond pass-at-k and what does production success look like?Eval designSenior–staff+

Forecasting accuracy during promotionsYour supply chain demand forecasting model performs well on average but badly during promotional periods. Walk me through how you'd diagnose and address this online-offline gap.Online/offline gapMid–senior

Training data from scanned documentsDesign a training data pipeline for a document understanding model where documents arrive as scanned PDFs with varying quality. How do you ensure consistent training data quality?Training data pipelineMid–senior

Fallback for drug interaction checkerDesign the fallback behavior for an ML-powered drug interaction checker in a pharmacy system. What happens when the model is unavailable or returns low-confidence predictions?Fallback behaviorSenior–staff+

Lookback window for time-series retrainingYou're designing a retraining pipeline for a time-series forecasting model. How do you decide the lookback window for training data and when historical data becomes obsolete?Retraining cadenceMid–senior

Multi-modal search serving optimizationDesign a serving architecture for a multi-modal model (text + image) that powers product search. How do you optimize for cost when image encoding is 10x more expensive than text?Serving architectureSenior–staff+

Investigating rising default ratesYour credit risk model shows concept drift: default rates are rising but feature distributions look stable. Design your investigation process and mitigation strategy.Drift detectionSenior–staff+