Brief
A high-throughput video understanding pipeline that semantically chunks long-form video into searchable scenes, transcripts and entity timelines, processing around eight hours of video per hour on commodity hardware.
<12ms
Inference
30 FPS
Throughput
8h
Video / Hour
Problem
Long-form video is effectively unsearchable. Finding a specific moment means someone scrubbing a timeline, which does not scale past a small library and makes the archive dead weight.
Solution
- Scene boundary detection combining visual and audio embeddings, so chunks follow meaning rather than fixed intervals.
- Per-frame multi-class object and action tagging sustained at 30 FPS.
- Vector search across scenes, so "find every shot where X happens" is a query rather than an afternoon.
- Usage-based billing tied to processed minutes, so cost tracks value rather than seat count.
Result
The pipeline sustains 30 FPS with model latency under 12ms, processing roughly eight hours of video per hour and turning an unsearchable archive into a queryable one.
Under the hood
| Vision | YOLOv8, ONNX, CUDA |
|---|---|
| Audio | Whisper transcription |
| Search | Pinecone vector search over scene embeddings |
| Backend | FastAPI |
