System designMediumsd-g595

Subject Ml inference edgeLevel Mid–Senior~45 minCommon in ML systems interviewsIndustries Technology

Question

Design an on-device + cloud ML inference system for a mobile keyboard's next-word/auto-correct prediction used by 500M devices, where predictions must run locally in under 20ms with no network dependency, models must update over the air, and personalization must respect on-device privacy (typed text shouldn't leave the device). Cover the on-device model, the update mechanism, how personalization works without sending raw data, and the cloud's role.

What a strong answer looks like

Clarify scale and constraints first. Propose a clean component breakdown, then go deep on the hard parts — data model, bottlenecks, consistency, failure modes — and name the trade-offs you are making.

Learn the concepts

Narrate your design

Loading whiteboard…

Run or narrate your approach, then ask the coach.