FedorAiToolkit – LoKR Training with SAM 3D Body
Clone FedorAiToolkit and run the Krea2 LoKR config with SAM 3D Body to refine body likeness.
Clone FedorAiToolkit and run the provided config to train LoKR with SAM 3D Body for better body likeness.
Summary
FedorAiToolkit is a fork of the original ai‑toolkit that adds a second reward‑based training pass using Meta’s SAM 3D Body to enforce body shape consistency during LoKR training. The workflow consists of a standard LoRA/LoKr supervised fine‑tuning stage followed by a 15–60 step reward stage that generates images, scans them with SAM 3D Body, and optimizes the LoRA towards the scanned 3D shape while also using ArcFace embeddings for facial identity. The reward stage is implemented with DRaFT‑K, differentiating only through the final denoising steps to keep GPU usage manageable. The fork supports Krea 2 and LoKr models, with training times of ~30 min for face‑only and ~60 min for face + body on an RTX 5090. The repository (github.com/CliffNodes/FedorAiToolkit) includes instructions for cloning, setting up the SAM 3D Body model, and adjusting reward steps. This approach yields more consistent full‑body likeness without hand‑tuning body captions, making it attractive for artists who need accurate body proportions in generated images.
Key changes
- Adds second reward‑based training pass using SAM 3D Body for body shape consistency
- Uses DRaFT‑K to differentiate only through final denoising steps, reducing GPU load
- Compares 3D body shape and ArcFace embeddings for face identity during reward stage
- Supports Krea 2 and LoKr models, training times ~30 min (face‑only) or ~60 min (face + body) on RTX 5090
- Forked from ai‑toolkit, includes instructions for SAM 3D Body setup and reward step adjustment
- Improves full‑body likeness without hand‑tuning body captions