System designHardsd-g083

Subject RelevanceLevel Senior–Staff~50 minCommon in Distributed systems interviewsIndustries Technology, Software development

Question

Design the relevance-evaluation and experimentation platform for a large search team — the system that decides whether a ranking change actually improves results before it ships to all users. It must support offline metrics (from judged data + logs), online A/B tests, guard against metric gaming, and detect when an experiment is harming a user segment.

What a strong answer looks like

Clarify scale and constraints first. Propose a clean component breakdown, then go deep on the hard parts — data model, bottlenecks, consistency, failure modes — and name the trade-offs you are making.

Learn the concepts

Narrate your design

Loading whiteboard…

Run or narrate your approach, then ask the coach.