Question

Design a streaming ETL that enriches a high-volume click/event stream (80k/sec) by joining it to slowly-changing reference data (user profiles, product catalog, ~500M rows updated via their own CDC stream) and a second event stream (impressions, joined to clicks within a 30-minute window). You need low latency, must handle reference updates and out-of-order/late events on both streams, and can't fit all reference data in memory on every worker. Design the join strategy, state, and the correctness pitfalls.