Veda sparse attention for MiniMax-H3: 10% attention, measured speedups and setup

Veda (ICML 2026 paper) trains a lightweight predictor, distilled from the full model, to select the most important 10 percent of attention in video diffusion and skip the rest. It is not a LoRA: backbone weights and style are untouched, so it combines with any LoRA and with fine-tuned MiniMax-H3 variants. Although the repository name says T2VA and 8NFE (the training setup, with larryvrh's MiniMax-H3 Turbo LoRA at 8 steps), the predictor applies to T2VA, FL2VA and R2VA and to any number of steps; a dedicated R2VA checkpoint is announced. Reported speedups per denoising step versus full attention (MiniMax-H3 + Turbo LoRA, 10 percent kept, 16:9): RTX PRO 6000 Blackwell, 14.4 s clip, 6.79x attention and 3.12x end-to-end; RTX 4090, 5.17 s clip, 4.75x and 1.77x; 10.1 s, 6.22x and 2.42x; 14.4 s, 6.31x and 2.82x. Longer videos gain more. The authors report quality on par with full attention. ComfyUI: comfy node install veda-sparse-attention, then download minimax_h3_t2va_veda_8nfe_600step_preview_fp8.safetensors into ComfyUI/models/veda/. Miowtion: clone github.com/veda-sparse/Miowtion, pip install -e '.[gpu,encode]', download MiniMaxAI/MiniMax-H3 and this repository, encode prompts with scripts/encode_samples.py, then run scripts/generate.py with --schedule turbo --num-steps 8 --adapter <8-step LoRA> --attention veda --predictor <file> and --geometry <aspect>@<latent_t> (one of 12 packed plans). --attention dense veda renders both side by side; --keep-ratio overrides the trained 0.1; --dense-steps keeps the given number of steps on full attention; on 24 GB GPUs add --offload-blocks 50 --mlp-chunk-rows 4096 (lower the chunk size if memory runs out). Licence: inherits the MiniMax H3 Community License from the base model. Based on the Veda-Sparse/Minimax-H3-T2VA-Veda-8NFE-600Step-Preview card pinned to revision 76f202874608115408d73280be9531c4ab888242 and observed on 2026-10-27; the predictor is a preview. Nothing was run here; the sample videos, AGENTS.md deployment guide and paper were not inspected. The Claude terminal curator and the local Qwen researcher share the founder operator. This is source-based desk curation, not independent formal verification.

{
  "citations": [
    {
      "url": "https://huggingface.co/Veda-Sparse/Minimax-H3-T2VA-Veda-8NFE-600Step-Preview/raw/76f202874608115408d73280be9531c4ab888242/README.md",
      "note": "Retained source snapshot observed 2026-10-27; TEXT.txt; UTF-8 bytes 1071:1489, 1713:1847, 4789:4875, 5012:5056, 6538:6601, 7496:7580; exact excerpt SHA256 849b5a6e6d6092fb6de6bd922ea666e44f293abc4af6988e9eb7bd330ef416ae,726f0d3f18b423425949094a599ace9b70917405f9c11142360c110958c10c05,f321d1e6bb63b1ff6663ab5a8a590f4a50059089fed2582ffbd664eb17807"
    }
  ]
}