Local‑First Video Indexing Pipeline Using Claude Code and LM Studio
Implement local‑first video indexing pipeline.
Implement local‑first video indexing pipeline.
Summary
The author details a local‑first video indexing pipeline built with Claude Code, LM Studio, and other open‑source tools to process an archive of thousands of raw video clips. The pipeline extracts metadata using ffprobe and exiftool, performs reverse geocoding, and generates five evenly spaced frames per clip at 1920px resolution. WhisperX transcribes the audio with word‑level alignment and speaker diarization, while insightface detects faces and stores 512‑dimensional embeddings in a SQLite database for cross‑archive queries. A vision model then reads the frames, transcript snippet, and folder context to produce YAML frontmatter and a prose description, which is written to a .description.md sidecar file. The entire process is orchestrated by a Python script written by Claude Code, with defensive constructor handling for WhisperX API changes. The pipeline supports 97 languages and uses LM Studio’s Gemma 4 31B Q4 model for vision inference, requiring significant memory and swap usage on the author’s MacBook Pro M1 Max.
Key changes
- Local‑first index with .description.md sidecar files
- Uses ffprobe, exiftool, WhisperX, insightface, and vision model
- Supports 97 languages and 512‑dimensional face embeddings
- Runs on LM Studio Gemma 4 31B Q4, requiring significant memory
- Handles swap memory and defensive constructor for WhisperX API changes
- CLI permission errors handled gracefully
- Pipeline processes five frames per clip at 1920px