Automatic caption generation and alignment

DesignMedium40 minGoogle style

Your task

Design the automatic caption/subtitle generation and alignment pipeline for a video platform that must produce time-synced captions (and translations into 30+ languages) for every uploaded video, including live streams where captions must appear within ~3s of the spoken word. Captions must be accurately word-timed for highlighting, support creator edits without re-running ASR, and handle a backlog of millions of pre-existing videos.

It covers

  • Captions
  • Asr alignment
  • Subtitle pipeline