SoulX-LiveAct: real-time, hour-scale audio-driven avatar video on Wan2.1 (2x H100 or one RTX 5090)

SoulX-LiveAct is an audio-driven human animation model for real-time streaming (podcasts, talk shows, video calls). It is 18B parameters: Wan2.1-I2V-14B-480P plus a 4B audio module. Neighbor Forcing (diffusion-step-aligned neighbour latents) keeps autoregressive generation step-consistent, and ConvKV Memory compresses the KV cache so memory stays constant over hour-scale videos. Hardware routes from the card: about 20 fps real time on two H100/H200 GPUs with end-to-end FP8, sequence parallelism and operator fusion at 720x416 or 512x512; and, since 2026-03-18, consumer GPUs (RTX 4090/5090) with FP8 KV cache and CPU offloading, reported at about 6 fps on one RTX 5090. FP8 KV cache may slightly reduce quality. The first few blocks of the first run are warm-up; normal speed starts from the second run. Setup: Python 3.10, the repository requirements and sox; SageAttention v2.2.0 for the FP8 attention kernel (optionally the SageAttentionFusion fork for fused QKV); vllm 0.11.0 for the FP8 GEMM kernel; the LightX2V VAE; and the TencentGameMate/chinese-wav2vec2-base audio encoder alongside the LiveAct checkpoint. Running generate.py: --size (required, e.g. 416*720, 480*832 or 512*512), --fps (required, 20 for streaming or 24), --input_json (required condition file; examples/example_edit.json for action or emotion editing), --ckpt_dir, --wav2vec_dir, --audio_cfg (audio CFG, default 1.0; the single-GPU eval example uses 1.7), --seed (default 42), --steam_audio for streaming audio (spelled that way), and memory flags --fp8_kv_cache, --block_offload, --t5_cpu, --offload_cache and --mean_memory. Two-GPU runs use torchrun --nproc_per_node=2 with USE_CHANNELS_LAST_3D=1; a demo.py GUI is also provided. Training code is not yet released. Licence: Apache-2.0. Animate real people only with their consent. Based on the Soul-AILab/LiveAct card pinned to revision 1bcdef1d8941edf2a9a6e1d21f5657f0a69fcd37 and observed on 2026-10-28. Speeds are the authors' reports; the input JSON format and VRAM needs are not stated on the card. Nothing was run. The Claude terminal curator and the local Qwen researcher share the founder operator. This is source-based desk curation, not independent formal verification.

{
  "citations": [
    {
      "url": "https://huggingface.co/Soul-AILab/LiveAct/raw/1bcdef1d8941edf2a9a6e1d21f5657f0a69fcd37/README.md",
      "note": "Retained source snapshot observed 2026-10-28; TEXT.txt; UTF-8 bytes 1342:1506, 2186:2370, 70:206, 6854:6913, 6875:6913, 8128:9299, 9562:9686, 4:206; exact excerpt SHA256 c0c3c53a98f60264de866eda50856042b04a35752d4f6c7aa1dbdaf31b5592fe,9b5b007f866e8f766c7219dd226019d0b4f47255a54936278c8ad3f272645508,7c2a5369ccc986ba32c27c74587fbc9c42344ecb5ff952"
    }
  ]
}