Recommendation & ranking

Picking the best 10 items out of a billion in less than 200 milliseconds.

The idea

You can't run a heavy Machine Learning model on 1 billion videos to see which one a user wants next. It's too slow. Instead, modern systems use a Two-Stage Funnel.

Stage 1 is Candidate Generation (Retrieval). It uses fast, cheap heuristics (like "videos liked by similar users") to narrow 1 billion items down to ~500 candidates. Stage 2 is Ranking. Now that we only have 500 items, we can afford to run a heavy, highly accurate Deep Neural Network to score and sort them, returning the top 10.

All Items (1,000,000) 1. Candidate Generation (Fast / Cheap) 2. Heavy Ranker (Slow / Accurate)
Waiting for user request.

How it works (Two-Stage Pipeline)

def get_feed(user_id):
    # STAGE 1: Candidate Generation (O(1) lookups)
    # Fast! Narrows 1B items to 500.
    candidates = set()
    candidates.update( get_trending_videos() )
    candidates.update( get_videos_from_subscriptions(user_id) )
    candidates.update( get_collaborative_filtering_matches(user_id) )
    
    # STAGE 2: Ranking (Heavy ML)
    # Slow! But we only score 500 items.
    scored_items = []
    for video in candidates:
        score = deep_neural_net.predict_click_probability(user_id, video)
        scored_items.append( (score, video) )
        
    scored_items.sort(reverse=True)
    return scored_items[:10] # Top 10 to UI