ML system design questions.
67 ml system designquestions from the bank — open to read. Pick one and practice it out loud; a coach note comes back in seconds.
Learn the ideas first
All 67 questions
Design an ML system to recommend videos to logged-in users on a streaming service with 200 million users.Design the monitoring story for a credit-decisioning model that's been in production for six months. What do you alert on?Design a fraud-detection ML system that has to decide every transaction in under 100ms at 10K transactions per second.You're building a system that summarizes news articles. Walk me through your evaluation strategy beyond a few automatic metrics.Your offline model evaluation looks great but online metrics show degraded performance. Walk me through the most common reasons.Design a system that decides when to retrain a customer-churn model. What signals would trigger a retrain?Design a feature store for a company with a hundred ML use cases. What's non-negotiable in the design?Design a system where your ML model serves predictions for an e-commerce ranking page. What happens when the model is unavailable?Design a system for serving large-language-model inference to a million daily users, balancing cost and latency.Design an evaluation framework for a chatbot that handles customer support. What does success look like and how do you measure it?Design the labeling pipeline for a system that classifies user-uploaded images into 200 categories. Labels are noisy.Walk me through how you'd build a drift-detection system for a fleet of fifty production ML models.Design a search-ranking system that personalizes results per user using behavioral features. Latency budget is 50ms.How would you instrument a deployed model so that you can detect training-serving skew quickly?Design the data pipeline for training a delivery-time prediction model that needs to update daily.An LLM-based product feature occasionally produces unsafe outputs. Design the safety system around it.Design an A/B test for replacing a heuristic ranker with an ML model. What guardrails do you set?Design a feature for showing AI-generated product descriptions on an e-commerce site. Quality and cost both matter.Design a system where you continuously evaluate a model against a champion baseline and promote new versions automatically.How would you architect a system that lets data scientists run dozens of experiments per day against the same offline dataset, reproducibly?Walk me through how you'd detect that a deployed image-classification model is being shown a new class of input it wasn't trained on.Design an evaluation harness for a code-completion model. How do you measure usefulness beyond exact-match accuracy?Design the serving infrastructure for a system that runs many small models — say, one per customer — economically.You discover that the feature store generates one set of values offline and a slightly different one online. How do you design around that?Design a system where your ML model assists a human reviewer in approving loan applications. What does the integration look like?Design a system to train a model on user data while respecting deletion requests. What does compliance with right-to-be-forgotten look like?Your model retrains nightly but stakeholders want it to react to events within minutes. How do you architect a system that supports both?Design an evaluation strategy for a retrieval-augmented generation system. What's actually being measured?Design a system that uses ML to suggest replies in a messaging product, with privacy as a hard constraint.Your team's pricing model occasionally produces obviously wrong prices. What system around the model prevents customer harm?You're building a spam classifier for email. Walk me through how you'd collect and prepare training data, including how you'd handle the class imbalance between spam and legitimate emails.Design a simple drift detection check for a model that predicts house prices. What metrics would you track on incoming prediction requests to know if the model is seeing different data than it was trained on?You're deploying a sentiment analysis model that scores product reviews. Should you serve it via a REST API, batch processing, or both? Explain your reasoning based on the use case.You've built a model that tags support tickets by topic. Beyond overall accuracy, what other metrics would you track to ensure it's working well across all ticket categories?Your movie recommendation model has 85% accuracy offline but users aren't clicking the recommendations in production. What are three possible reasons for this gap?You've deployed a model that predicts restaurant delivery times. How would you decide how often to retrain it—daily, weekly, monthly? What factors influence this decision?Your product search ranking model goes down during a deployment. What should the search page show to users while the model is unavailable?You're training a click-through rate model on web traffic data. How would you split your data into train/validation/test sets, and what time period should each cover?Your job title classification model was trained on 2023 data. It's now 2025. What simple checks would you implement to detect if the distribution of job titles has changed significantly?You need to serve predictions from a lightweight model that scores user profiles. Would you precompute all scores and cache them, or compute on-demand? What trade-offs matter here?You built a model to detect toxic comments. How would you evaluate whether it works equally well across different types of toxicity (hate speech, harassment, threats)?Your model predicts customer lifetime value and is retrained monthly. Suddenly, a competitor launches and customer behavior changes overnight. How would you adjust your retraining approach to respond faster?Design a training data pipeline for a recommendation model that needs to incorporate user interactions from the last 24 hours. How do you handle late-arriving events and ensure reproducibility?You're serving a personalized ranking model that takes 300ms to score items, but the page needs to load in under 200ms. Walk me through your architecture options and trade-offs.Design a drift detection system for a loan-approval model where ground truth labels arrive 6-12 months after predictions. What proxy metrics do you monitor in the meantime?Your medical imaging model was trained on data from five hospitals but will deploy to hundreds. Design your evaluation strategy to catch distribution shift before it impacts patient care.Design the fallback behavior for a dynamic pricing model in a ride-sharing app. The model goes down during Friday evening rush hour—what happens?You're building a content moderation system that must review 50K user posts per minute. Design the ML serving architecture, including how you handle spikes to 200K per minute.Design a retraining strategy for a product search ranking model in an e-commerce platform. How do you decide between daily, weekly, or event-triggered retraining?Your A/B test shows the new model has better offline metrics but users are engaging less. Walk me through your debugging process to identify what's wrong.Design a training data pipeline for a fraud detection model where only 0.1% of transactions are fraudulent. How do you handle class imbalance and ensure the model sees recent fraud patterns?You're deploying a recommendation model across 20 countries with different languages and cultures. Design your evaluation framework to ensure quality across all markets.Design a system to detect when your ad-click prediction model is being gamed by adversarial actors. What metrics and patterns do you monitor?Your search ranking model serves results in under 50ms, but you want to add a large language model reranker. Design the serving architecture to maintain latency SLAs.Design the fallback strategy for a real-time inventory allocation model in a warehouse management system. The model fails—how do you ensure orders still get fulfilled?You're building a training pipeline for a model that uses features from ten different upstream data sources with varying SLAs. How do you design for reliability and debuggability?Design a retraining system for a conversational AI model. What signals indicate the model needs updating versus just prompt engineering or retrieval improvements?Your offline metrics use last-click attribution but your online system uses multi-touch attribution. Design an evaluation strategy that bridges this gap.Design a drift detection system for a fleet of region-specific models (50+ regions). How do you distinguish between legitimate regional differences and actual model degradation?You're serving embeddings for a semantic search system to 100M users. Design the caching and serving architecture to minimize cost while maintaining relevance.Design an evaluation framework for a code-generation model used by developers. How do you measure quality beyond pass-at-k and what does production success look like?Your supply chain demand forecasting model performs well on average but badly during promotional periods. Walk me through how you'd diagnose and address this online-offline gap.Design a training data pipeline for a document understanding model where documents arrive as scanned PDFs with varying quality. How do you ensure consistent training data quality?Design the fallback behavior for an ML-powered drug interaction checker in a pharmacy system. What happens when the model is unavailable or returns low-confidence predictions?You're designing a retraining pipeline for a time-series forecasting model. How do you decide the lookback window for training data and when historical data becomes obsolete?Design a serving architecture for a multi-modal model (text + image) that powers product search. How do you optimize for cost when image encoding is 10x more expensive than text?Your credit risk model shows concept drift: default rates are rising but feature distributions look stable. Design your investigation process and mitigation strategy.