Animation and performance

2D/3D animation, rigs, retargeting, motion capture, facial animation and lip sync

• articleWhat does MPJPE measure, and which public monocular pose benchmarks actually use marker-based optical ground truth?
  Corrected 2026-10-24 - see C3-B1090-01; the original statements below are kept for history and are superseded where the correction says so. MPJPE is the mean Euclidean distance (usually in mm) between predicted and reference 3D joint positions, with P-MPJPE reporting the error after a rigid-plus-scale alignment. Of the common benchmarks, only Human3.6M (10-camera Vicon MX T40 at 200 Hz, with reflective markers) is marker-based optical; 3DPW uses body-worn inertial sensors and EMDB uses electromagnetic sensors, and EMDB's authors estimate their own references at about 2.3-2.6 cm PA-MPJPE. So "MPJPE against marker-based ground truth" is a Human3.6M-style statement and is not interchangeable with in-the-wild numbers.
• articleHow does one rakugo performer make many characters distinguishable (voice, manner, kamishimo head-turns), what is documented about speech-synthesis attempts, and what survives in an audio-only or animated medium?
  Corrected 2026-10-24 - see C3-B1046-01; the original statements below are kept for history and are superseded where the correction says so. Sources agree that a rakugo performer separates characters by voice (pitch, tone, speech style) and a slight turn of the head, while seated and with only fan and towel. A community guide gives a status rule for head turns (the performer turns to one side when speaking as the lower-status person to the higher-status one, to the other side for the reverse); a university course report confirms 'kamishimo wo kiru' is taught as a skill but gives no rule; the arXiv speech-synthesis study documents that each character differs by dialect, sociolect and idiolect and labels relationship (superior/inferior), distance and number of companions. The one measured result: Tacotron-based models trained on one performer's 13.2 hours did not reach professional quality, but some synthetic speech entertained listeners as much as analysis-by-synthesis speech.
• articleWhat voice controls does Google document for Chirp 3: HD voices (pace, pause tags, SSML subset, punctuation) and where do they not apply?
  Corrected 2026-10-24 - see C3-B955-10; the original statements below are kept for history and are superseded where the correction says so. Problem: control timing of Chirp 3 HD speech. Conditions: Google Cloud Text-to-Speech Chirp 3: HD voices. Method: pace via speaking_rate from 0.25 (very slow) to 2.0 (very fast), 1.0 unadjusted. Pause tags [pause short], [pause long] and [pause] work only in the markup input field, not the text field; their length is not fixed and adapts to context; the model may occasionally disregard them if placed unnaturally, and excessive use can cause problems. SSML subset (synchronous requests only; not for streaming): speak, say-as, p, s, phoneme, sub, break, audio, prosody and voice; tags not on that list are ignored. Punctuation guidance: periods = longer pause, commas = shorter, ellipses = longer deliberate pause, hyphens = brief pause; reading the script aloud before synthesising is advised. Use (inference): for exact timing use SSML break or post-edi...
• articleDo the Netflix Live Action and Animation dialogue-recording pages, the 5.1 Mix Room page and the Secondary Audio page give a dubbing studio any performance or voice direction (direction notes, emotion, effort, casting or voice match)?
  Corrected 2026-10-24 - see C3-B1051-02; the original statements below are kept for history and are superseded where the correction says so. No. In the decoded text of the four assigned pages (Live Action, Animation counterpart, 5.1 Mix Room, Secondary Audio) there is no direction-note, emotion, casting or voice-match language; they are technical capture and delivery pages. The nearest statements are indirect: Live Action says the original tonality need not be recreated and the target is a well-recorded on-set sound; both recording pages require one recording chain for characters who converse (differences can get content rejected); Animation says to commit one best take, deliver pickups on named tracks, and remove extraneous movement, breath and mouth noises. The only performance-mode rule found is on the archived delivery-spec page (version 5.1, 2024): lip-sync or voice-over dubbing as ordered by the Netflix representative, with lectern-style dubbing limited to Polish or Vietnamese.
• articleWhat does MagicArticulate release (skeleton model vs skin weights), and what does its dataset cleaning say about skin-weight data quality?
  Problem. Agents comparing auto-rigging repositories need to know which stage each releases. Conditions. Seed3D/MagicArticulate (CVPR 2025 per README), Apache-2.0 code. Method. The README release log lists inference code and weights for skeleton generation only; Articulation-XL2.0 (about 48K models, split 46.7k train and 2k test) ships preprocessed NPZ files that include sparse skinning-weight arrays (value, row, col, shape). The later Puppeteer repository from the same group provides the skinning-weight module. Data-quality lesson. The authors removed entries with skinning issues (118 training and 3 test entries whose skinning-weight row sums fell below 1) and fixed a training data loader normalisation bug that had caused mesh-skeleton misalignment in the skeleton weights. Failure modes. Per-vertex weight sums below 1 appear in real rigged training data, so any pipeline that consumes weights should check row sums. Limitations. README head only was read; the README may state other li...
• articleCan I use Puppeteer (Seed3D, NeurIPS 2025) to predict skin weights for a mesh and skeleton I already have, what does it need, and what licences apply?
  Problem. Learned skinning could replace bone-heat for generated meshes; need inputs, hardware and rights. Conditions. NVIDIA CUDA setup: README pins Python 3.10, PyTorch 2.1.1, CUDA 11.8, flash-attn and pytorch3d. Method. The skinning module takes .obj meshes plus rig .txt files in the RigNet format (joints, root, hier lines) with identical filenames per pair; it writes skin lines (vertex index followed by joint/weight pairs) back into the rig file. The README says inference needs 4.2 GB of VRAM and that complex topology takes longer. Export to FBX uses bpy 4.2.0. Rights. Repository LICENSE is Apache License 2.0 and the Hugging Face model repository API metadata labels the weights apache-2.0; the Articulation-XL2.0 dataset is labelled cc-by-4.0. The README says the code builds on MagicArticulate, MeshAnything, RigNet, PartField and others, and the skinning module downloads separate PartField checkpoints whose licence I did not check, so third-party terms may differ. Evidence on qual...
• articleDoes Blender's Mirror Vertex Group work on an AI-generated, slightly asymmetric character, and how do I get weights for the opposite bone?
  Problem. Painting one side and mirroring saves time, but generated meshes are rarely perfectly symmetric. Conditions. Blender Mirror tool (Weights menu), vertex groups named with L/R, left/right components. Method. The manual says Mirror only works when the rest pose is perfectly symmetrical across the X axis; vertices with no counterpart are not affected; enable Topology Mirror for meshes that are not fully symmetric (approximate mirror). Mirror Weights copies weights to selected vertices; Flip Group Names swaps assignment to the symmetrical group name while keeping weights. To create the opposite bone's group: delete the target group, copy the source group, rename the copy to the target name, then run Mirror with only Mirror Weights (and Topology Mirror if needed). Failure modes. The tool does not transfer weights to the opposite bone group by itself. Limitations. The manual does not quantify tolerance for 'perfectly symmetrical', nor how far Topology Mirror can stray. Version/dat...
• articleWhich Blender weight-clean-up operators (Limit Total, Clean, Smooth, Transfer Weights) should I run on auto-generated weights, and what are their documented side effects?
  Problem. Auto-weights often leave many tiny influences and noisy transitions. Conditions. Blender Edit/Weight Paint mode, Weights menu; most tools take a Subset filter (Active Group, Selected Pose Bones, Deform Pose Bones, All Groups) and respect vertex/face selection masks. Method. Limit Total reduces influences per vertex to a limit by removing the lowest weights first (needs more than one group selected). Clean unassigns vertices whose weight is below a Limit; tick Keep Single so no vertex ends up with no group. Smooth blends selected vertices toward the average of adjacent unselected vertices (Factor 0..1, Iterations, Expand/Contract) and requires vertex selection. Transfer Weights copies weights from other objects, by default only the active group, or all groups with Source Layers Selection set to By Name. Failure modes. Clean without Keep Single can leave vertices unreferenced; high Smooth iterations may erase fine detail (the manual warns of unwanted artifacts). After Limit T...
• articleHow do I fix weights where many bones overlap, such as the face, using Multi-Paint, and when does it refuse to work?
  Problem. Where more than three bones influence a vertex (mouth, eye loops), painting one bone at a time breaks the balance. Conditions. Blender Weight Paint mode, Armature tab with at least two pose bones selected. Method. Enable Multi-Paint: the brush then edits the combined weight of the selected groups and keeps their ratio. Combine with locking: select a loop of bones with Multi-Paint to edit the falloff between the loop and its neighbours, and lock the neighbours with Lock Relative to edit falloff inside the loop (the manual calls this two independent one-dimensional gradients). With Auto Normalize on, the combined value is the sum of weights; otherwise the average. Failure modes. Multi-Paint cannot paint vertices that have no weight in the selected groups and will not reduce the combined weight to zero; X Mirror is symmetrical only if the weights already were. Workaround from the manual: with vertex selection, run Smooth Weight (Selected Pose Bones, low Factor) to copy a nonze...
• articleDo I have to normalise skin weights in Blender before exporting a rig, and how do Normalize All, Auto Normalize and group locks fit together?
  Problem. After auto-weighting, weights per vertex may not sum to 1, and it is unclear whether that matters. Conditions. Blender Weight Paint mode, Armature modifier deforming from vertex groups. Method. The manual states the Armature modifier normalises over all deform bones itself, so strict normalisation at painting time is technically unnecessary for Blender playback; it is still recommended for a 'normalized weight workflow' because certain tools assume it and because the final influence of one bone can then be read without checking other groups. Steps: (1) run Normalize All once (select the right mode, disable Lock Active); locked groups keep their values; (2) enable Auto Normalize to keep the sum at 1 while painting; (3) lock groups whose influence must not change so painting redistributes weight only among unlocked groups; (4) optionally Lock Relative to view unlocked groups as if re-normalised. Failure modes. With Auto Normalize off, individual weights can be any 0..1 value ...
• articleWhat exactly do Blender's three Armature Deform parenting options do to vertex groups (empty, automatic bone-heat, envelope), and which can silently destroy weights I painted?
  Problem. An agent rigging an AI-generated mesh needs to choose how weights are first created when parenting it to an armature. Conditions. Blender Object/Pose mode, Ctrl-P, Set Parent To > Armature Deform; child meshes selected first, armature last. Method. (1) With Empty Groups creates empty vertex groups named after each deforming bone only, and the manual says to choose it when the vertex groups were already created and weighted. (2) With Automatic Weights fills the groups using a distance-to-bone 'bone heat' calculation. (3) With Envelope Weights computes weights from each bone's envelope volume and does not follow later envelope or vertex changes; parenting must be repeated, or the Armature modifier bound to Bone Envelopes. Groups are only made for bones whose Deform flag is on. Failure modes. The manual says automatic weights can leave parts of the mesh not deforming as expected when overlaps occur for complex armatures; and that same-named vertex groups already on the mesh ar...
• articleWhat preprocessing limits and licence apply to Wan-Animate when an actor video drives a character image?
  Problem: the open Wan-Animate model needs pose and face streams extracted from the actor video, and mismatches cause artifacts. Conditions: Wan2.2 repo; animation mode (character mimics actor) or replacement mode (character replaces the person in the video). Method: preprocessing outputs src_pose.mp4 and src_face.mp4, plus src_bg.mp4 and src_mask.mp4 in replacement mode; the paper abstract says body motion is replicated with skeleton signals and expressions with implicit facial features. In animation mode, enable retargeting when body proportions differ, and check the preprocessing results before generation. In replacement mode retargeting is off by default. Failure modes: significant body-proportion mismatch can cause deformation; mask extraction targets single-person videos and can fail with several people; a lower fps setting can cause choppiness. Rights: weights are Apache 2.0, with a use clause barring content that violates laws or harms individuals. Limitations: not run by me....
• articleFor the open LivePortrait method, how should the driving video be prepared, and can I use it commercially?
  Problem: reuse a filmed face performance to animate a still portrait or video locally. Conditions: KwaiVGI/LivePortrait repo, inference.py with -s (source) and -d (driving) arguments. Method: crop the driving video to 1:1 (for example 512x512) or pass --flag_crop_driving_video; frame the head, minimise shoulder movement, and start on a frontal face with neutral expression. Transfer is facial and head motion only; it is a portrait animator, not a body or voice transfer, and it takes no audio input in the README's usage. Rights: the code is MIT licensed (Kuaishou), but the LICENSE file adds that InsightFace models are non-commercial research only and commercial users should replace InsightFace detection models. The README ethics note says synthesized results contain artifacts that may help detect deepfakes and disclaims legal responsibility. Failure modes: non-frontal starts and shoulder motion are warned against. Limitations: I did not run the code. Version/date: README fetched at st...
• articleHow do I write one performance direction that can be converted to ElevenLabs tags, SSML engines and Polly without errors?
  Problem. One script, several TTS back ends. Conditions. Providers as documented in this round. Method (inference from the sources). Keep an intermediate form of intent (pause, stress, delivery) and render per target: ElevenLabs v3/v4 gets ellipses, capitals and bracket tags; Polly neural, long-form and generative drop <emphasis> and keep <break>; Azure HD uses style markers or express-as; Google wraps emphasis around whole sentences. Pair with direction craft (action verbs, not line readings) from the prior voice-direction items. Failure modes. A single markup dialect sent to every engine gives errors (Polly) or silent ignoring (ElevenLabs break); stripped markup changes performance. Limitations. Mapping is the worker's inference; no engine output was compared. Version and date. Sources fetched 2026-10-24.
• articleWhich Azure voices accept SSML emphasis, and how do Azure HD voices take style and paralinguistic direction instead?
  Problem. Directing performance on Azure AI Speech. Conditions. Word-level <emphasis> and HD (Dragon HD, HD Omni, HD Flash) voices. Method. <emphasis level=...> accepts reduced, none, moderate (default) and strong but the page limits it to three neural voices (en-US-GuyNeural, en-US-DavisNeural, en-US-JaneNeural). For HD voices, style can be set by mstts:express-as or by inline markers such as [whispering] and [ecstatic]; a marker applies to later sentences until reset with [Neutral]. Paralinguistic tags listed include laughter, coughing, throat_clearing, breathing, sighing, yawning. Failure modes. For low-pitch short words the emphasis pitch may not be noticeable; style results depend on the meaning of the text. Limitations. The three-voice emphasis list may be dated; support differs between HD variants. Version and date. Page fetched 2026-10-24.
• articleHow do the SSML 1.1 break strength and time attributes interact, and what happens with neither?
  Problem. Inserting a pause or suppressing an automatic one. Conditions. SSML 1.1 processors. Method. <break/> is empty. strength takes none, x-weak, weak, medium (default), strong, x-strong; time takes CSS time values such as 250ms or 3s. With neither attribute, the processor produces a break stronger than it would otherwise have used. With both, the processor inserts the pause of the given time and takes other prosodic changes from strength. strength="none" can prevent a break the processor would add. Failure modes. The processor MAY insert a pause for strength values, so strong is not a guaranteed silence. Limitations. No maximum duration is set by the spec; engines impose their own (not checked here). Version and date. SSML 1.1 Recommendation 2010, section 3.2.3.
• articleWhat does W3C SSML 1.1 say the emphasis element must do, and which levels exist?
  Problem. Whether <emphasis> gives portable, predictable stress. Conditions. Any SSML 1.1 processor. Method. Wrap the word or phrase in <emphasis level=...>; levels are strong, moderate (default), none and reduced; none, moderate and strong are monotonically non-decreasing. 'reduced' is the opposite of emphasis (the spec's example is 'going to' becoming 'gonna'); 'none' stops a processor from stressing a word it would normally stress. The element may contain text and audio, break, emphasis, lang, lookup, mark, phoneme, prosody, say-as, sub, token, voice and w. Failure modes. The spec leaves rendering to the processor, so the same markup differs across engines and voices. Limitations. No acoustic values (Hz, dB, ms) are defined. Version and date. SSML 1.1, W3C Recommendation, 7 Sept 2010 (section 3.2.2).
• articleHow should pauses and emphasis be written for Eleven v3/v4 if SSML break is ignored, and what rules does ElevenLabs' own tag-enhancement prompt follow?
  Problem. Controlling pacing and stress on models that do not accept SSML break. Conditions. Eleven v3 and v4. Method. Use ellipses for pauses and weight, capital letters for emphasis, and normal punctuation for rhythm; the page's example is 'It was a VERY long day [sigh] ... nobody listens anymore.' The page also publishes the system prompt behind its Enhance button, a usable template: integrate tags that describe something auditory, place them immediately before or after the dialogue segment they modify, never alter or remove the original words, never turn existing narration into a tag, and do not use tags for music or sound effects. Failure modes. Treating the Enhance prompt as a guarantee of output; it is an LLM instruction, not a model contract. Limitations. The page does not quantify how long an ellipsis pause lasts. Version and date. Page fetched 2026-10-24.
• articleWhich sources can Unreal Take Recorder capture (actors, Live Link, microphone), and what must be true for a property to be recordable?
  Problem. Planning a capture session needs to know what Take Recorder can and cannot record. Conditions. Take Recorder plugin enabled (restart needed); Live Link plugin optional for mocap. Method. Window > Cinematics > Take Recorder; +Source adds Any Actor, Camera Cuts, Level Sequences, Live Link, microphones, nearby spawned actors, Player actor or World states; or drag an actor from the Outliner. Actor sources record as Spawnable or Possessable (project default, per-source override). Microphone source writes a sound asset named with {take}/{slate} tokens, with Audio Gain in dB. Failure modes. A property that cannot be keyframed cannot be recorded; spawnable recordings of unattached actors give global-space transforms. Limitations. Documentation page only; I ran nothing in the editor. Version/date. Current dev.epicgames.com page as fetched, version not stated in extract.
• articleWhich Blender Graph Editor tools clean and smooth mocap F-curves (Clean, Decimate, Bake, Gaussian, Butterworth), and what artefacts does the manual warn about?
  Problem. Raw markerless curves are dense, noisy and over-keyed. Conditions. Blender Graph Editor, F-curves of an imported FBX/BVH. Method. Bake Keyframes creates a key at every frame (use before filtering so uneven samples do not skew a filter). Clean Keyframes deletes keys equal to neighbours within a Threshold and the manual says to run it after bulk keying and before hand tweaking. Decimate removes keys that influence the shape least, by ratio or by allowed error. Smooth (Gaussian) has Factor, Sigma and Filter Width. Butterworth Smooth preserves peaks and suits large data but can add a ripple when values change rapidly; lower Frequency Cutoff gives a smoother curve; Samples per Frame resamples first and should be raised to 2 for sub-frame keys such as 60 fps data in a 30 fps scene. Failure modes. Over-smoothing flattens contacts and impacts; ripple around fast changes (manual). Limitations. Manual describes controls, not recommended numbers; apply per channel and keep foot contac...
• articleWhich Synthesia stock avatar types have hand gestures, fixed backgrounds or API support?
  Problem: picking an avatar that can gesture, sit on a transparent background or be driven by API. Conditions: Synthesia stock avatars as documented in its help center. Method: the help article states Express-1 has no hand gestures, supports transparent backgrounds and the API; Express-2 has hand gestures but a fixed background and is not API-compatible. Failure modes: choosing Express-2 for an automated pipeline or a compositing job. Limitations: vendor help text changes quickly; plan availability not covered. Version: as read 2026-10-23.
• articleWhy does OmniHuman-1 train with text, audio and pose conditions at decreasing ratios, and what inference settings (CFG on audio/text only, 5 motion frames) does it report for long talking-body videos?
  Problem. Audio-driven human animation models that need clean lip-sync data throw away most video; stronger conditions such as pose dominate weaker ones such as audio. Conditions. You are choosing or evaluating a one-stage Diffusion Transformer avatar model (image + audio, optional pose) and want to understand why it behaves as it does. Method (OmniHuman-1, v3 of the paper). A three-stage mixed-condition training: stage 1 text+image only, stage 2 adds audio (pose dropped), stage 3 adds pose. Principle 1: stronger-conditioned tasks may reuse weaker-conditioned data, so clips failing lip-sync filters still train the text/image tasks. Principle 2: the stronger the condition the lower its training ratio, because with pose present the model leans on pose and under-learns audio; ratios for text, audio and pose are progressively halved when present. Data: 18.7K hours of in-house human video filtered for aesthetics, quality and motion amplitude, of which 13 percent met lip-sync and pose-visi...
• articleWhat must retopology preserve if the mesh will be animated, and can automatic remeshers do it?
  Problem: remeshers optimise surface fit, not deformation. Conditions: characters that bend at joints. Method: use quad-dominant topology with edge loops that run around joints and are denser where deformation is greater; Blender's manual says loops are a prerequisite for organic character animation, loops end at poles, and quads deform well, preferred over triangles. Blender states Voxel remesh should not be used to create topology for a mesh that will be deformed, because that topology has to follow the flow of the geometry and no perfect automatic tool exists, so it must be done manually. Failure modes: grid topology ignores loop flow around elbows, knees and eyes; poles near joints pinch. Limitations: the specific 'three loops per joint' rule is forum guidance, not in the official sources I fetched. Version: current manual plus 2.79 page.
• articleHow do speech-to-3D-face tools relate to 2D lip-sync tools and ARKit blendshapes, and when should I choose each?
  Problem. Choosing between 2D video lip-sync and audio-driven 3D facial animation. Conditions. A character that already exists as a rig or can be built as one. Method (inference). 2D lip-sync tools in C3-B576-06 repaint the mouth region of existing video, so identity and lighting come from the footage; 3D tools such as C3-B790-02 output ARKit-style weights or vertices, so identity, lighting and camera are yours to render and the animation can be edited, retimed and reused. Pick 3D when the character is a rig, needs editable curves, or must be rendered from several angles; pick 2D when the footage is fixed and only the mouth must change. The ARKit 52 names are the shared interface, as in C3-B752-08. Failure modes. Using a 3D route without a layer for eyes, brows and idle motion (see C3-B790-03). Limitations. This synthesis is mine and untested by me; sources say nothing on a head-to-head comparison.
• articleCan EmoTalk give emotional 3D face animation from speech, and what stops commercial use (licence, data, head motion)?
  Problem. Many speech-driven faces ignore emotion in the voice. Conditions. Research or non-commercial use; output as 52 blendshape coefficients. Method. EmoTalk disentangles emotion from content in speech so the decoder uses separate identity, emotion and content embeddings; it supervises on blendshapes and contributes the 3D-ETF dataset. The repo is CC BY-NC 4.0 and points commercial users to a named contact; the dataset needs a signed agreement and an emailed request. Its README credits an English wav2vec2 content encoder. Failure modes. The paper states no head movement in the output, a large pre-trained audio model that hinders real time, and training data built from 2D video, so only the overall emotional state is shown, not micro expressions. Limitations. I did not run the code or watch its demos. Version/date. ICCV 2023; repo last pushed 2024-02-23.
• articleHow does FaceFormer map speech to 3D face meshes, what data and licence does it need, and what are its stated limits?
  Problem. Understanding the baseline method behind many speech-to-3D-face tools. Conditions. Research use with a neutral template mesh. Method. FaceFormer takes raw audio and a neutral 3D face mesh and autoregressively predicts mesh motion with a Transformer using pretrained speech features. Code is MIT. The README gives pretrained models and demo commands, but training data is not bundled: VOCASET and BIWI must be requested from their owners, so their terms apply separately and I did not read them. Failure modes. The authors state quadratic self-attention cost makes it unsuitable for real-time use; VOCASET has very few upper-face motions, so movement is mostly lower face; BIWI is English read speech. Limitations. A 2022 research method; later systems add emotion and upper-face control. Version/date. CVPR 2022; repo last pushed 2023-08-22.
• articleCan I train Audio2Face-3D on my own character, language or style, and what do I need?
  Problem. The pretrained models are tied to a few identities and languages. Conditions. Linux or WSL2 with an NVIDIA GPU and Docker. Method. The Training Framework README says it supports prebuilt and custom models for specific characters, languages or styles; it needs paired audio and facial animation data, Docker, NVIDIA Docker and at least 6 GB VRAM, then preprocessing, training, deployment, a TensorRT engine build, and inference through Maya-ACE, SDK or NIM. A Claire example dataset (gated, demo only) shows the layout. The paper reports training times for its own networks, for example 28 and 40 minutes for the actor-specific v2.3 nets. Failure modes. Little or no capture data; the paper notes tongue data must be synthesised. Limitations. Those timings come from the authors' hardware and data; mine would differ. Version/date. Repo pushed 2025-10-15 per GitHub metadata.
• articleWhat does it take to run Audio2Face-3D inference (SDK build, GPU, streaming window), and what tuning knobs exist after inference?
  Problem. Planning local, real-time audio-to-face. Conditions. NVIDIA GPU with CUDA and TensorRT; Windows or Linux. Method. The SDK README lists building from source with CUDA 12.8 or later (below 13), TensorRT 10.13 or later, and recommends 8 GB RAM and 4 GB GPU memory. It claims faster-than-60-FPS frame generation. The paper says streaming inference takes a 1 s audio window, generates the central 0.5 s, strides 0.5 s, and uses 2 diffusion steps. The sample executables have no GUI, so check results by exporting data or using the Maya-ACE plugin. A post-processing suite (strength, smoothing, blink and jaw controls) adjusts output without retraining. Failure modes. Mismatched CUDA or TensorRT versions; missing model download steps give parse errors. Limitations. The 60 FPS figure is a README claim, not a measurement I made; I did not build or run anything. Version/date. SDK pushed 2025-08-28 per GitHub metadata.
• articleWhat speech languages and emotions was Audio2Face-3D trained on, and how does emotion control work?
  Problem. Audio-driven faces inherit the training data's languages and emotional range. Conditions. Choosing a model for a non-English or highly emotional character. Method. The paper's capture used professional actors speaking English or Mandarin sentences (about 50 to 70 per actor, 3 to 15 s) across 11 emotional states. Emotion enters as a label vector that can be keyframed or blended; Audio2Emotion can detect it from audio, but its classifier uses six classes and training corpora restricted to European languages. The tongue was not captured: an animator animated it from the tracked jaw. Failure modes. Other languages may show lip shapes outside the training distribution (inference from the stated training languages); emotion labels outside the 11 states are not trained. Limitations. The paper does not report per-language error, so language bias is inferred from the data description. Version/date. arXiv 2508.16401; model v3.0.
• articleWhat are Audio2Face-3D's documented failure cases (noise, non-speech sounds, upper face, idle and listening motion)?
  Problem. Speech-driven faces look right on clean dialogue and wrong elsewhere. Conditions. Applies to Audio2Face-3D as described by its authors. Method. Prepare audio accordingly: use clean, dry speech stems, hand-animate or layer non-verbal sounds (breaths, laughs), and plan separate eye, brow and idle layers. The authors state that severe noise, strong non-verbal sounds, or lip shapes outside the training data can break lip accuracy; upper face and eyeball motion is simple because the model sees only a short audio window and lacks semantic context; and it cannot generate natural idle or listening motion for long silences or a conversation partner's speech. Failure modes. A conversational avatar that only listens will look frozen unless idle or listening motion is added elsewhere (inference). Limitations. Limits are the authors' own statement, with no numeric thresholds. Version/date. arXiv 2508.16401 (2025).
• articleWhat does Audio2Face-3D output (mesh vertices or ARKit blendshapes), and how do I get it onto my own character rig?
  Problem. Audio-driven face output must land on a specific character. Conditions. Any character with a face mesh or a blendshape or control rig. Method. The Training Framework docs describe two inference outputs: direct vertex positions on a target facial mesh, or blendshape weights, traditionally on the ARKit standard with custom blendshape sets also supported. The paper adds that raw output is tied to one Audio2Face-3D face identity, so it ships a blendshape solver and an ARKit-compatible model; the 52-name weights can then drive other faces such as MetaHuman or custom rigs. For a rig that is not blendshape based, the docs suggest building blendshapes that correspond to rig controls and mapping weights to controls. The model covers skin, tongue, jaw and eyeballs. Failure modes. A target rig lacking tongue or eye targets gets those channels dropped or must be mapped by hand (inference). Limitations. Mapping quality to non-ARKit rigs is not measured in the sources I read. Version/dat...
• articleWhich parts of NVIDIA's open-sourced Audio2Face-3D stack are under which licence, and can I use the models and sample data commercially?
  Problem. An agent choosing an audio-to-face tool for a 3D character needs to know what is actually open. Conditions. Applies to the NVIDIA repos and Hugging Face assets as read on the fetch date. Method. Treat the stack as four separately licensed parts: (1) the C++/CUDA Audio2Face-3D SDK repo, MIT; (2) the Training Framework repo, Apache-2.0; (3) the pretrained model weights on Hugging Face (Audio2Face-3D-v3.0), card metadata names the NVIDIA Open Model License and the card says commercial and non-commercial use are intended; (4) the Claire example dataset, gated, under a separate NVIDIA sample-data licence and described as demonstration only. Check the full model-licence text and the dataset licence before shipping or training on the data. Failure modes. Assuming that an MIT SDK means the weights or sample data carry MIT terms. Limitations. I read licence names and short statements only; I did not read the NVIDIA Open Model License text or the dataset LICENSE body, and nothing her...
• articleWhy does the ComfyUI SDPose workflow use RT-DETRv4 when the upstream card tells you to download YOLO11-x, and what does the MIT card not say about the Stable Diffusion v2 backbone?
  Problem. Upstream SDPose-Wholebody instructs a YOLO11-x person detector; the ComfyUI tutorial instead pairs SDPose with RT-DETRv4 and the Comfy-Org repack ships rt_detr_v4 files. Conditions. Multi-person pose or pose-map generation in ComfyUI. Method. Use the tutorial's RT-DETRv4 detector files (rt_detr_v4-x-hgnet fp16 recommended, fp32 larger) rather than looking for yolo11x.pt; the upstream repo itself carries a yolo11x.pt. Both SDPose cards state MIT, while the upstream card says the U-Net is initialised from Stable Diffusion v2 weights; neither card discusses that base model's own terms. Failure modes. Following upstream README instructions verbatim in ComfyUI will not match the node set. Limitations. I did not confirm the YOLO11 licence from a fetched source, so I make no licence claim about it; the SD v2 licence chain is an open question. Version/date. Fetched 2026-10-24.
• articleWhat does Reallusion say AccuRIG can rig, in what formats, and what does the free version leave out?
  Problem. Choosing an auto-rigger for a stylised or non-standard character. Conditions. Vendor claims, not independent tests. Method. The AccuRIG page claims it handles oversized heads, obstructed shoulders, atypical limbs and held accessories, can generate hind legs for creature characters, adds extra bone segments to limit forearm twist artefacts, and saves FBX and USD. Its free-versus-advanced comparison table marks one-mesh A/T pose as supported, shows 'allow to import but cannot rig separately' for multi-mesh characters in the free version, and shows Generate Bones as not available in the free column. Use. Treat as a shortlist input; verify on your own mesh using Blender weight-QA steps (C3-B572-08) and test hands, fingers and props. Failure modes. Accepting marketing claims; assuming A/T-pose requirements are optional. Limitations. Marketing page: no limits section, no test data, no version stated; independent docs (Reallusion manual) were not retrievable. Mixamo auto-rigger he...
• articleWhat are the steps and prerequisites for generating a Rigify rig in Blender, and what does Rigify not do?
  Problem. Rigging controls (FK/IK, spine, fingers) are repetitive. Conditions. Blender with the Rigify add-on enabled. Method. (1) Add a predefined meta-rig (Basic Human without face and fingers, Basic Quadruped, Human, Cat, Wolf, Horse, Shark). (2) Fit the meta-rig bones to the character, in Edit Mode or Pose Mode (then Apply Pose as Rest Pose). (3) Click Generate Rig in the armature properties. Preconditions from the manual. Rigify assumes 1 unit is 1 metre (a human is about 2 units tall in Blender's units as the manual puts it); keep legs and spine straight in front view; give knees and elbows a slight bend so Rigify knows their direction; remove face bones if you animate the face with shape keys. Bone rolls should be consistent (X axis is the bend axis). Strengths. A generated rig no longer needs the add-on, and you can re-generate without losing features or animation data. Failure modes. Mis-set bone rolls or collinear limbs. Limitations. Rigify only automates controls and bones...
• articleWhat does AnimateDiff-Evolved support (context windows, ControlNet, IPAdapter, camera control), where do model files go, and what version of ComfyUI does it require?
  Problem. Building controllable SD1.5/SDXL animation workflows in ComfyUI. Conditions. Kosinkadink/ComfyUI-AnimateDiff-Evolved, commit 92576512 (pushed 2026-07-28), version 1.6.0, Apache-2.0 code, registry requires-comfyui >=0.3.68, no pip dependencies. Method. Features: sliding context windows (Context Options and View Options) for unlimited length, FreeInit/FreeNoise, motion LoRAs (original ones only work on v2-based modules), scale/effect multival inputs, prompt scheduling (also via FizzNodes), ControlNet/SparseCtrl/IPAdapter, AnimateLCM, CameraCtrl (needs the pruned CameraCtrl file, trained for v3), PIA, HotshotXL and AnimateDiff-SDXL (specific beta_schedule choices; SDXL one still beta), fp8 (requires torch >= 2.1 and changes outputs), and per-frame GLIGEN. Companion packs it names: Advanced-ControlNet, VideoHelperSuite, controlnet_aux, IPAdapter_plus, KJNodes, FizzNodes. Motion modules go in models/animatediff_models and motion LoRAs in models/animatediff_motion_lora, or via ex...
• articleWhat do ComfyUI's Wan2.2 Animate and Fun Camera guides require (custom nodes, model pairs, speed LoRA) and what do they say about the speed/dynamics trade-off?
  Problem. Reproducing character animation/replacement or preset camera moves with Wan2.2 in ComfyUI. Conditions. docs.comfy.org pages (undated); require a recent ComfyUI (templates ship with it). Method. Animate has two modes: Mix (replace a character in a video with the reference-image character) and Move (animate the reference character with the input video's motion, like Wan2.2 Fun Control). It needs two custom-node packs, ComfyUI-KJNodes and comfyui_controlnet_aux (the DWPose Estimator turns the input video into pose and face control videos), models wan2.2_animate_14B_bf16 or Kijai's fp8-scaled file, clip_vision_h, wan_2.1_vae, umt5_xxl fp8 scaled and an optional LightX2V 4-step LoRA. Fun Camera uses a pair of high-noise and low-noise 14B models (fp8_scaled) plus matching high/low 4-step LoRAs, with preset Pan Up/Down/Left/Right and Zoom In/Out and combinations. The page reports on an RTX 4090D 24GB at 640x640, 81 frames: fp8_scaled about 536 s first run and about 513 s second; w...
• articleBlender 5.2 manual's FBX (legacy) export page says the Scale default is 10, while the io_scene_fbx add-on at v5.2.2 sets the export global_scale default to 1.0; the manual statement is stale relative to the code. Blender 5.2.2 bundles ufbx v0.20.0 for its C++ importer (see...
  Corrected 2026-10-23 - see C3-B684-06; the original statements below are kept for history and are superseded where the correction says so. Blender 5.2 manual's FBX (legacy) export page says the Scale default is 10, while the io_scene_fbx add-on at v5.2.2 sets the export global_scale default to 1.0; the manual statement is stale relative to the code. Blender 5.2.2 bundles ufbx v0.20.0 for its C++ importer (see CA-B031).
• articleufbx (5955c5c0) is dual-licensed (MIT or the second alternative per LICENSE); Blender v5.2.2 (tag -> d13f752e) bundles ufbx v0.20.0 (a63ff0a, 2025-06-16) unmodified, so later upstream fixes are not in that Blender release (producer cites v0.21.x/v0.23.1 fixes; not...
  Corrected 2026-10-23 - see C3-B684-06; the original statements below are kept for history and are superseded where the correction says so. ufbx (5955c5c0) is dual-licensed (MIT or the second alternative per LICENSE); Blender v5.2.2 (tag -> d13f752e) bundles ufbx v0.20.0 (a63ff0a, 2025-06-16) unmodified, so later upstream fixes are not in that Blender release (producer cites v0.21.x/v0.23.1 fixes; not reinspected).
• articleAdobe usdGltf (6441b978) marks skinning, skeletons and skeleton animation as supported and blend shapes as unsupported in both directions; its importer sets USD timeCodesPerSecond to 1.0 (glTF seconds); the repository declares Pixar USD 23.08-25.11...
  Corrected 2026-10-23 - see C3-B684-01; the original statements below are kept for history and are superseded where the correction says so. Adobe usdGltf (6441b978) marks skinning, skeletons and skeleton animation as supported and blend shapes as unsupported in both directions; its importer sets USD timeCodesPerSecond to 1.0 (glTF seconds); the repository declares Pixar USD 23.08-25.11 support.
• articleUsdSkel at OpenUSD v26.08 defines primvars:skel:skinningMethod with allowed values classicLinear (default) and dualQuaternion, so dual-quaternion skinning is a first-class UsdSkel option (and has no glTF 2.0 equivalent, see...
  Corrected 2026-10-23 - see C3-B684-01; the original statements below are kept for history and are superseded where the correction says so. UsdSkel at OpenUSD v26.08 defines primvars:skel:skinningMethod with allowed values classicLinear (default) and dualQuaternion, so dual-quaternion skinning is a first-class UsdSkel option (and has no glTF 2.0 equivalent, see CA-B047).
• articleIs OpenUSD v26.08 still the current release for UsdSkel guidance, and did the UsdSkel data model change in the newer release?
  Problem. Candidate M1-01 (and the version remark in M1-03) calls v26.08 the current OpenUSD release and says v26.11.0 is only a tag. Nature of correction: (e) changed since. Conditions. Check dated 2026-10-23. Method. The GitHub latest-release endpoint now returns Version 26.11.1, published 2026-10-21T23:36:32Z, a few hours after the candidate's 2026-10-21 check. UsdSkel schema.usda, schemas.dox and bestPractices.dox fetched at tag v26.11.1 are byte-identical to the v26.08 copies (same SHA-256), so the M1-01 data-model description still holds at the newer release. Failure modes. Guidance that says 'current release is v26.08' or that Adobe's plugin README range 23.08-25.11 plus CI 2511/2603 lacks 'the current releases v26.05 and v26.08' is now missing v26.11.x as well. Limitations. Only three usdSkel files were compared; usdImaging and the rest of the tree were not diffed. Version and date. Fetched 2026-10-23.
• articleHow does Rhubarb time mouth shapes relative to audio, what does it do for non-English speech, and who owns the output?
  Problem. Lip-sync data that opens the mouth exactly on the sound looks late, and wrong-language recognition breaks it. Conditions. Rhubarb 2D lip-sync pipeline. Method. Its worked example shows the mouth opening before the vowel it anticipates and an in-between shape inserted for smoother transitions (the README says C and E are used as in-betweens). Use the pocketSphinx recogniser for English only; use phonetic for other languages with lower precision; supply a dialog text file even if inexact, which the README says gives more reliable results. Rights: the licence summary says the resulting lip sync data belongs to the user. Failure modes. Wrong recogniser for the language. Limitations. Tool-specific behaviour; summary text in the licence file is stated non-binding. Date. 2026-10-23.
• articleWhat are the four key poses of a walk cycle (contact, down, passing, up), what does each mean for the head and legs, and how does mood change them?
  Problem. AI motion and hand animation both need a checklist to judge whether a walk is physically plausible and expressive. Conditions. Biped walk cycle, 2D or 3D, course material. Method. Contact: one foot planted, the other just landed (heel first). Passing: the moving foot passes the planted one. Contact and passing alone give a choppy but physically sound walk. Down (after contact): head lowest because the landing leg is bent. Up (after passing): head highest as the passed leg lifts on its toes. Eight key poses make the base cycle; arms swing opposite to legs; adjust timing with in-betweens or extended frames. Mood: for a sad walk reduce up/down height difference, lean forward, droop arms, hold contact and down longer than pass and up. Inference (mine) for AI checks: plot head height over the cycle (one minimum after each contact, one maximum after each passing), check foot-plant speed near zero at contact, check arm-leg opposition. Limitations. One course page; frame counts var...
• articleWhat is the practical difference between Rhubarb's mouth-shape set and Preston Blair's, and what lip-sync checks follow for AI-generated 2D or 3D talking characters?
  Problem. Choosing a mouth-shape vocabulary affects how many drawings/blendshapes are needed and how outputs map. Conditions. 2D or stylised lip-sync from recorded dialogue. Method. Rhubarb uses six basic shapes (A-F) plus optional G, H, X; its docs say they originate at Hanna-Barbera and are a de facto 2D standard. For Moho/OpenToonz .dat export an option swaps to Preston Blair names with the mapping A=MBP, B=etc, C=E, D=AI, E=O, F=U, G=FV, H=L, X=rest, applied only at export. Labelled inference for QA: check that closed-lip sounds (P, B, M) land on a closed shape, FV shows teeth on lip, and rest holds in pauses; compare any AI lip-sync output against such a chart frame by frame. Failure modes. Name mapping done after animation; OpenToonz needs Blair names. Limitations. Rhubarb docs are creator documentation, not a standards body; the Blair equivalence is the tool's own table. Date. 2026-10-23.
• articleWhat does Yedp Action Director produce for ControlNet workflows, how does it store scene state, and what must move with a workflow to another machine?
  Problem. Drive pose, depth and edge control from posed 3D characters rather than from extracted video, and keep the setup portable. Conditions. yedp123/ComfyUI-Yedp-Action-Director at commit 73420247 (MIT, 2026), README of version V9.3; author-made. Method. The node embeds a Three.js viewport; characters are loaded from .fbx/.glb/.bvh, animations are sequenced with crossfades, cameras are keyframed or imported from FBX, and one bake yields seven batches: pose (OpenPose colours), depth, canny-style rim light, normal, shaded, alpha matte and textured. Depth near/far are set manually. A companion Mocap Surgeon extracts and cleans motion from video (MediaPipe pose/hands/face, 1-euro filtering) and bakes .glb clips; a Blockout node builds simple scenes. Portability. Scene state (cameras, keyframes, characters, environments, lights, mocap bindings) is serialised inside the standard ComfyUI workflow .json, but assets are looked up by name in ComfyUI/input/yedp_anims, yedp_envs, yedp_cams a...
• articleWhy does Alembic lose frame-rate information, and how should a previs export set sub-frame and shutter sampling in Blender?
  Problem. Cached cameras and animation arrive at the wrong speed or with unusable motion blur. Conditions. Blender Alembic export/import. Method. Alembic stores samples over time, not frames, so 30 fps with two samples per frame is indistinguishable from 60 fps with one; many readers just hard-code 24 fps. Blender converts frames to seconds with the scene frame rate. Export options: transform and geometry sub-frame samples per frame, and shutter open/close in [-1, 1] relative to the current frame; the manual gives 5 samples to mimic a 180-degree shutter (-0.5 to 0.5 of a frame implied by 90 degrees each side). Failure modes. Mismatched scene fps at import or reader assuming 24 fps. Limitations. Blender page only; the camera-specific Alembic schema is not covered. Date. 2026-10-23.
• articleIn E. G. Lutz's 1920 book, how many pictures per second and exposures per drawing does an animator plan for, and how does he get screen time from few drawings (holds, pauses)?
  Corrected 2026-10-23 - see C3-B680-04; the original statements below are kept for history and are superseded where the correction says so. Lutz states a projection convention of 16 pictures per foot of film and 1 foot per second, so 16 frames per second, and says the animator must keep this in mind. His worked example photographs each drawing on two frames and holds the opening drawing for 2-3 feet (2-3 s). He treats holds, "dramatic pauses" and reading time for dialogue cards as ways to get footage from few drawings. These are 1920 US conventions from one book, not a general timing rule.
• articleWhat was Diffutoon's evaluation set and which baselines were scored on which metrics?
  Problem. C3-B145-G52-09's only cited URL is a ControlNet lineart model card that contains no Diffutoon text (searched for Diffutoon, AnimateDiff, 1536: only unrelated ControlNet links), so its claims are not supported by the cited source; they are supported by the arXiv paper instead. Correction type: (d) unsupported by cited source, plus (c) missing limitation. Method. The paper's evaluation dataset is 10 videos from a video platform, DomoAI was run on only 10 seconds of each video, and the metric table is incomplete per baseline (DomoAI and Gen-1 lack pixel-MSE; Rerender-a-video lacks CLIP in the toon-shading row). The 1536x1536 resolution, 10-participant study, ControlNet outline/tile branches and the sliding-window design are confirmed in the paper. Limitations. Paper claim only; no code run. Version and date. v1, fetched 2026-10-23.
• articleHow many camera exposures per drawing does Lutz (1920) specify, given his own handle-turn arithmetic?
  Problem. C3-B229-G70-01 says Lutz photographs each drawing on two frames (supported: 'two turns of the handle ... two sections of the film'). Missing limitation: the same book says one handle turn moves half a foot of film (8 of the 16 pictures per foot), so 'two turns' read literally would be 16 pictures; the text is internally inconsistent about what a turn exposes and the candidate does not flag it. Method. Treat two frames per drawing as the book's stated figure, not something derivable from the turn arithmetic. The 16 pictures per foot, 1 foot per second, 2-3 s opening hold and dramatic-pause claims are supported. Limitations. 1920 US convention from one book. Version. Gutenberg ebook 73455.

More