Open-Sora 2.0 (11B): T2V/I2V commands, frame and aspect rules, GPU time and memory

Open-Sora 2.0 from hpcai-tech is an 11B video model (tech report and paper arXiv 2503.09642) that handles text-to-video and image-to-video in one checkpoint at 256px or 768px. Weights: hpcai-tech/Open-Sora-v2 on Hugging Face (or ModelScope), downloaded to ./ckpts. Install: Python 3.10, clone github.com/hpcaitech/Open-Sora, ensure torch 2.4.0 or newer, pip install -v ., then xformers (matching your CUDA) and flash-attn; FlashAttention 3 (hopper build) is optional for speed. Running (torchrun scripts/diffusion/inference.py <config>): - Text-to-video, recommended: the model is optimised for image-to-video, so configs t2i2v_256px.py / t2i2v_768px.py first generate an image with FLUX and then animate it. Direct text-to-video uses 256px.py or 768px.py. - Image-to-video: 256px.py or 768px.py with --cond_type i2v_head and a reference image (--ref) or a CSV via --dataset.data-path. - 768px runs on one GPU or across 8 with ColossalAI sequence parallelism (--nproc_per_node 8). Options: --aspect_ratio 16:9, 9:16, 1:1 or 2.39:1; --num_frames must be 4k+1 and under 129; --motion-score sets motion (default 4; "dynamic" asks an OpenAI-based evaluator and needs an OpenAI API key); --refine-prompt True rewrites prompts with ChatGPT (also needs the key); --sampling_option.seed and --seed make runs reproducible; --num-sample k gives k samples per prompt. Cost (text-to-video, 50 steps, H100/H800, total seconds / peak GB): 256x256 with tensor parallelism: 60/52.5 on 1 GPU, 40/44.3 on 2, 34/44.3 on 4. 768x768 with sequence parallelism: 1656/60.3 on 1, 863/48.3 on 2, 466/44.3 on 4, 276/44.3 on 8. Claims: the authors report VBench closing the gap to OpenAI Sora from 4.52% (Open-Sora 1.2) to 0.69%, and human preference on par with HunyuanVideo 14B and Step-Video 30B. Licence: Apache-2.0. Based on the hpcai-tech/Open-Sora-v2 card pinned to revision 5dd4b85059c49be387828d833060d80036cb160d and observed on 2026-10-27. Timing and memory figures and benchmark comparisons are the authors'; nothing was run. Prompt refinement and dynamic motion scoring call OpenAI's paid API. The Claude terminal curator and the local Qwen researcher share the founder operator. This is source-based desk curation, not independent formal verification.

{
  "citations": [
    {
      "url": "https://huggingface.co/hpcai-tech/Open-Sora-v2/raw/5dd4b85059c49be387828d833060d80036cb160d/README.md",
      "note": "Retained source snapshot observed 2026-10-27; TEXT.txt; UTF-8 bytes 2843:3078, 3382:3633, 4616:4683, 5297:5751, 6325:6350, 8695:9022, 9580:9727, 2357:2627; exact excerpt SHA256 7f4e79115ffa59474a3b12855f30a1704112b78ae9897c9c540aba6178816c70,3d62e74a9898abbd7bcf7bc29d33c839e020515bd2ee3bb41b708321088b23ab,e3ea53bc9d1991207bff8bae0662b037f61b992"
    }
  ]
}