What are Audio2Face-3D's documented failure cases (noise, non-speech sounds, upper face, idle and listening motion)?

Problem. Speech-driven faces look right on clean dialogue and wrong elsewhere. Conditions. Applies to Audio2Face-3D as described by its authors. Method. Prepare audio accordingly: use clean, dry speech stems, hand-animate or layer non-verbal sounds (breaths, laughs), and plan separate eye, brow and idle layers. The authors state that severe noise, strong non-verbal sounds, or lip shapes outside the training data can break lip accuracy; upper face and eyeball motion is simple because the model sees only a short audio window and lacks semantic context; and it cannot generate natural idle or listening motion for long silences or a conversation partner's speech. Failure modes. A conversational avatar that only listens will look frozen unless idle or listening motion is added elsewhere (inference). Limitations. Limits are the authors' own statement, with no numeric thresholds. Version/date. arXiv 2508.16401 (2025). Curator qualifications: Some cited evidence could not be re-located by curator GET (see mechanical_checks); no video sources. Second-tier recheck (C3R-0614a) re-located the excerpts by timed GET. Limitations: No quantitative thresholds given for what counts as severe noise; whether the same limits hold for the open v3.0 weights versus the paper's networks is not separately stated. Evidence status: statements located in the cited sources by a correlated desk review (Claude); not executed; not independently reviewed.

{
  "citations": [
    {
      "url": "https://arxiv.org/pdf/2508.16401",
      "note": "Sec. 7 Limitations; Sec. 7 Limitations; Sec. 7 Limitations; researcher reasoning"
    }
  ]
}