DomainShuttle – Domain‑Aware Video Generation with Domain‑MoT and AdaLN
Add DomainShuttle to your video generation pipeline to leverage Domain‑MoT, AdaLN, DualRoPE, and Cross‑Pair Consistent Loss for domain‑aware subject modeling.
Integrate DomainShuttle into your video generation workflows, experiment with AdaLN and DualRoPE, and evaluate consistency improvements on domain‑varied datasets.
Summary
DomainShuttle is a new video‑generation model built on Wan‑2.2 that introduces several domain‑specific innovations. The model decouples video and reference features and adds Domain‑MoT, a module that separates subject‑level spatial modeling from reference image tokens. Domain‑MoT is paired with a domain‑aware AdaLN, which adapts layer normalization to the domain of the reference image, improving consistency across different visual styles. The Video‑Reference DualRoPE scheme places reference image tokens and video tokens in separate RoPE spaces, enabling precise subject‑level spatial modeling. Cross‑Pair Consistent Loss is introduced to extract intrinsic subject features that are unaffected by irrelevant background features. The project is hosted on Hugging Face and has an accompanying website detailing the architecture.
These innovations aim to improve the fidelity and consistency of video generation, especially when using reference images from varied domains. The model’s architecture is designed to be modular, allowing developers to plug in different reference encoders or loss functions.
Overall, DomainShuttle offers a new set of tools for domain‑aware video generation that can be integrated into existing pipelines.
Key changes
- Domain‑MoT decouples video and reference features for subject‑level modeling
- Domain‑aware AdaLN adapts layer normalization to reference image domain
- Video‑Reference DualRoPE places reference and video tokens in separate RoPE spaces
- Cross‑Pair Consistent Loss extracts intrinsic subject features, ignoring irrelevant background
- Model built on Wan‑2.2 with modular architecture for easy integration