Why does OmniHuman-1 train with text, audio and pose conditions at decreasing ratios, and what inference settings (CFG on audio/text only, 5 motion frames) does it report for long talking-body videos?

Problem. Audio-driven human animation models that need clean lip-sync data throw away most video; stronger conditions such as pose dominate weaker ones such as audio. Conditions. You are choosing or evaluating a one-stage Diffusion Transformer avatar model (image + audio, optional pose) and want to understand why it behaves as it does. Method (OmniHuman-1, v3 of the paper). A three-stage mixed-condition training: stage 1 text+image only, stage 2 adds audio (pose dropped), stage 3 adds pose. Principle 1: stronger-conditioned tasks may reuse weaker-conditioned data, so clips failing lip-sync filters still train the text/image tasks. Principle 2: the stronger the condition the lower its training ratio, because with pose present the model leans on pose and under-learns audio; ratios for text, audio and pose are progressively halved when present. Data: 18.7K hours of in-house human video filtered for aesthetics, quality and motion amplitude, of which 13 percent met lip-sync and pose-visibility criteria. Inference: classifier-free guidance applied to audio and text but not pose, scale 6.5; captions come from an image-captioning model; for long videos the last five frames of the previous segment are motion frames for the next. Failure modes. Not reported in the text read: the section on limitations was not located; the training set is in-house and not available. Limitations. Evaluation on 100 videos sampled from CelebV-HQ and RAVDESS for portraits plus CyberHost's half-body test set; metrics are IQA, ASE, Sync-C, FID, FVD; vendor-style paper, no independent replication seen. Version/date. arXiv 2502.01061 v3 (revised 30 Jun 2025). Curator qualifications: All cited documentation excerpts located on the live pages by curator GET; no video sources. Correction folded 2026-10-24 from C3-B733-01 (curator-reviewed discovery audit): Problem. C3-B716-02 says the OmniHuman-1 limitations section was not located. Conditions. Same paper, arXiv 2502.01061 v3. Correction. The paper has a Limitation paragraph: because audio correlates weakly with motion, uncoordinated or overly expressive movements still occur and object interactions can appear unrealistic; the authors attribute this to insufficient training data and out-of-distribution input images; a relatively high CFG scale is needed for stability, which can overfit. Failure modes. Use these as OmniHuman-1 failure modes alongside the 6.5 CFG setting. Limitations. Vendor paper, no replication checked. Version/date. arXiv 2502.01061 v3. Limitations: Medium-high. Settings quoted are from the paper's implementation details for the research model; the commercial OmniHuman v1.5/v2 services may differ. I did not read the limitations section or the ablation table beyond the Table 1 caption. Corrected by C3-B733-01; where the correction disagrees, the correction governs. Evidence status: statements located in the cited sources by a correlated desk review (Claude); not executed; not independently reviewed.

{
  "citations": [
    {
      "url": "https://arxiv.org/html/2502.01061v3",
      "note": "Sec 3.3 Principle 2; Sec 4.1 Implementation Details; Sec 4.1 Implementation Details; Sec 4.1 Dataset"
    }
  ]
}