Built an open-source one-prompt-to-cinematic-reel pipeline on a single GPU
Run the eight‑stage cinematic pipeline on a single MI300X to generate 45‑minute videos from a single prompt.
Deploy the pipeline on your own GPU to evaluate performance and tweak the failure‑label retry logic.
Summary
Built an open‑source one‑prompt‑to‑cinematic‑reel pipeline that runs entirely on a single AMD Instinct MI300X, producing a 45‑minute end‑to‑end video from a single English sentence.
The eight‑stage pipeline uses Qwen3.5‑35B‑A3B for the Director Agent, FLUX.2 [klein] for character masters and per‑shot keyframes, Wan2.2‑I2V‑A14B for 81‑frame animation at 16 fps, a Qwen3.5‑35B vision critic with ten structured failure labels, ACE‑Step v1 for 30‑second music, Kokoro‑82M for nine‑language narration, and ffmpeg for mixing. Performance tuning includes ParaAttention FBCache (2× speed), torch.compile on transformer_2, and AITER MoE acceleration on the director, cutting end‑to‑end time from 25.9 min to 10.4 min per 720p clip. The MI300X’s 192 GB HBM3 allows all models to share the same card sequentially, whereas a 24 GB consumer GPU would need 4–5 cards. The pipeline is released under Apache 2.0 and the code is available at https://github.com/bladedevoff/studiomi300. A Hugging Face space demonstrates the workflow, though it is temporarily offline. The project showcases how to build a full cinematic pipeline on a single GPU with modern LLMs and diffusion models.
Key changes
- Director Agent uses Qwen3.5‑35B‑A3B to plan 6 shots and output JSON
- Character masters and per‑shot keyframes use FLUX.2 [klein] without LoRA training
- Animation uses Wan2.2‑I2V‑A14B at 81 frames @ 16 fps native
- Vision critic uses Qwen3.5‑35B with 10 structured failure labels for retry
- Music generated by ACE‑Step v1 (30 s instrumental)
- Narration by Kokoro‑82M in 9 languages
- ffmpeg mixes per‑shot voice‑over with adelay
- Performance gains: ParaAttention FBCache 2×, torch.compile, AITER MoE, reducing runtime to 10.4 min per 720p clip