Multimodal joint embedding alignment
Design a multimodal embedding system for a marketplace where a user can search the catalog by uploading a photo OR typing text, and both must hit the same product index ('find me products that look like this image' and 'red leather ankle boots' against one shared store). You have 200M products, each with images and text descriptions. Design how you produce embeddings so images and text live in a comparable joint space, how you index for both query modalities, and how you keep the two modalities aligned as either model evolves.
What a strong answer looks like
Clarify scale and constraints first. Propose a clean component breakdown, then go deep on the hard parts (data model, bottlenecks, consistency, failure modes) and name the trade-offs you are making.
Clarify5:00 left
Estimate5:00 planned
Design15:00 planned
Deep dive12:00 planned
Failure8:00 planned
Which questions mattered is sealed until you submit. Telling you now would just be handing over the edge cases.
Run or narrate your approach, then ask the coach.