Code RoomMultimodal joint embedding alignment
HardPrep Room Coding #3233

Multimodal joint embedding alignment

System designML systemsSenior–Staff~45 min

Design a multimodal embedding system for a marketplace where a user can search the catalog by uploading a photo OR typing text, and both must hit the same product index ('find me products that look like this image' and 'red leather ankle boots' against one shared store). You have 200M products, each with images and text descriptions. Design how you produce embeddings so images and text live in a comparable joint space, how you index for both query modalities, and how you keep the two modalities aligned as either model evolves.

What a strong answer looks like

Clarify scale and constraints first. Propose a clean component breakdown, then go deep on the hard parts (data model, bottlenecks, consistency, failure modes) and name the trade-offs you are making.

Clarify5:00 left
Estimate5:00 planned
Design15:00 planned
Deep dive12:00 planned
Failure8:00 planned
0:00
Which questions mattered is sealed until you submit. Telling you now would just be handing over the edge cases.