Content moderation pipeline

DesignHard55 minMeta and Google style

Your task

Design a content-moderation ML pipeline for a platform ingesting 50M pieces of user content/day (text, images, video) that must detect policy violations (hate, violence, CSAM, spam) with both proactive (pre-publish) and reactive (post-report) paths. Pre-publish scoring for text/images must complete within 300ms; video can be asynchronous. Cover the model architecture, the human-review loop, and how you trade off catching harm against over-removal.

It covers

  • Multimodal
  • Human in the loop
  • Precision recall