Video generation

Text/image/video-to-video, motion control, long-form consistency and model pipelines

articleAnimateDiff v1-5-2: baseline diffusers recipe and the clip_sample rule
AnimateDiff v1-5-2 baseline: SD1.5 finetune plus motion adapter, DDIMScheduler with clip_sample=False, linspace timesteps and steps_offset 1, 16 frames, guidance 7.5, 25 steps. Disable sample clipping in any clipping scheduler.
articleStable Video Diffusion XT: 25 frames at 576x1024, limits and licence
SVD-XT turns one 576x1024 image into 25 frames (about 4 s max) with no text control; about 180 s per clip on an A100 80GB. Outputs are watermarked by default. Non-commercial use is under the SVD community licence; commercial use needs Stability AI terms.
articleWan2.2-TI2V-5B: 720P sizes, single-GPU flags and image-to-video switch
Wan2.2-TI2V-5B makes 720P/24 FPS video; use --size 1280*704 or 704*1280 (not 1280*720). With --offload_model True --convert_model_dtype --t5_cpu it runs on a 24 GB GPU; adding --image makes it image-to-video.
articleLTX-2.3: checkpoints, 32-pixel and 8k+1-frame rules, licence
LTX-2.3 ships 22B dev and 8-step CFG=1 distilled checkpoints with latent upscalers. Use width and height divisible by 32 and 8k+1 frames (pad with -1 otherwise). Licence: LTX-2 Community License, not open source.
articleHunyuanVideo 1.5 720p T2V in diffusers: attention backend and default settings
HunyuanVideo 1.5 720p T2V example: bf16, CPU offload, VAE tiling, 121 frames, 50 steps, 24 fps. Install kernels and use attention_backend("_flash_3_hub") on H100/H800 or "flash_hub" elsewhere; the default backend also works.
articleWan 2.2 in ComfyUI: repackaged file locations and official templates
Comfy-Org/Wan-2.2 puts 14B Wan 2.2 models in diffusion_models as high/low-noise pairs (fp16 or bf16 plus fp8_scaled), with umt5_xxl encoders, VAEs, lightx2v 4-step LoRA pairs and wav2vec2 for audio-driven S2V. Each mode has an official template.
articleAnimateDiff-Lightning: step checkpoints, scheduler settings and ComfyUI setup
AnimateDiff-Lightning has 1/2/4/8-step motion modules for SD1.5. Use EulerDiscreteScheduler with trailing timesteps and linear betas, guidance 1.0 and steps equal to the checkpoint; the authors like 3 steps on the 2-step model and Motion LoRAs at 0.7 to 0.8.
articleAnimateLCM: diffusers recipe for fast SD1.5 AnimateDiff video
AnimateLCM pairs its MotionAdapter with an SD1.5 model, LCMScheduler (linear beta schedule) and its t2v LoRA at weight 0.8; the card example makes 16 frames at guidance 2.0 in 6 steps.
articleWhy does an autoregressive video model get worse the longer it keeps generating, and what are the names and visible symptoms of the failure?
Corrected 2026-10-24 - see C3-B1094-02; the original statements below are kept for history and are superseded where the correction says so. The primary papers agree on one mechanism: the model is trained on clean or short context but, when it keeps generating, it conditions on its own slightly wrong frames, so small errors compound (called error accumulation or exposure bias). Reported symptoms differ by method: brightness drift (over-exposure or darkening), motion that freezes, colour shift, flicker when a window shifts, and (separately) content that diverges because nothing remembers what left the frame.
articleWhat evaluation protocol (resolution, clips, frames, FID/FVD scope, quoted versus recomputed baselines) does each later paper state for its nuScenes FID/FVD, so which tables can even be set side by side?
Corrected 2026-10-24 - see C3-B1051-01; the original statements below are kept for history and are superseded where the correction says so. The stated protocols differ in at least five ways: Drive-WM 150 validation clips x 6 views = 900 segments of 40 frames with the official UCF FVD code; GenAD FID on 6019 frames and FVD on 4369 clips; UniMLVG following MagicDrive with 900 videos of 16 frames; Panacea generating the whole validation set at 256x512 with 8 frames; Delphi single-frame single-view FID and 4/8/40-frame FVD with recomputed baselines marked. DriveDreamer v1 states a 192x128 training size, v2 448x256 for training and evaluation; MagicDrive-V2 lists DriveDreamer at 128x192 with 32 frames. DriveDreamer-2 gives no clip or frame count for FID/FVD in the HTML text searched, and DriveDreamer v2 has no clips/segments sentence at all; DriveDreamer v1 states 150 clips x 8 segments = 1200, but for action-prediction evaluation. DrivingDiffusion says it matched metrics and settings "a...
articleHow do engine-exported depth, segmentation and edge passes actually reach these models, and what do the engine and model docs say (or not say) about matching their encodings?
Corrected 2026-10-24 - see C3-B1071-01; the original statements below are kept for history and are superseded where the correction says so. The one documented engine integration found is CARLA's Unreal Engine 4 branch (merged Aug-Sep 2025): it renders edge, depth and segmentation control videos and sends them with per-control weights to a separately deployed Cosmos-Transfer1 server. Unreal's Movie Render Queue documents EXR passes such as World Depth in world units. No source read states how engine depth units or segmentation colour conventions should be converted to the encoding the Cosmos branches were trained on (depth normalised to [0, 1], randomised-colour masks), so that conversion is an open interface question.
articleWhat happens inside a streaming video model when the prompt changes mid-rollout, which cache tricks do the papers use, and which failure modes do they report?
Corrected 2026-10-24 - see C3-B1086-01; the original statements below are kept for history and are superseded where the correction says so. Four papers handle a mid-rollout prompt change by editing the attention cache. LongLive recomputes cached key-value states with the new prompt (KV recache) and reports the trade-off: with no cache the new prompt is followed but transitions are abrupt; with a plain cache visuals are smooth but the new prompt is delayed or ignored. Zing-0.5 refreshes only the text K/V and pins the first frame after the switch; GameCraft-2 uses a prompt recaching mechanism with a sink token; WorldSonus replaces its cross-attention cache at the next 100 ms chunk boundary. MaineCoon gates the next prompt on whether the current spoken line has finished. Reported failure modes: prompt inertia, abrupt transitions, adherence decay across switches, drift beyond about 500 frames and no persistent entity state.
articleWhich open-weight interactive or action-conditioned world models were released after Oasis and DIAMOND (Matrix-Game 1.0-4.0, Hunyuan-GameCraft, LingBot-World 1 and 2), what is actually released for each, and what licence is attached to which part?
Corrected 2026-10-24 - see C3-B1075-01; the original statements below are kept for history and are superseded where the correction says so. Eight released model lines with public weights were found: Matrix-Game 1.0 (MIT), 2.0 (MIT), 3.0 (Apache-2.0, 5B base and distilled; a 28B model is announced as 'soon'), 4.0 (Apache-2.0; scene checkpoints for three games, released 2026-10-15), Hunyuan-GameCraft-1.0 (Tencent Hunyuan Community License with territory exclusions), LingBot-World 1 (Apache-2.0) and LingBot-World 2.0/Infinity (CC BY-NC-SA 4.0, non-commercial). Licences differ by version inside one project and by component (code, weights, game assets, bundled third-party models). 'Open weights' here means files present on Hugging Face at the pinned revision with a licence text; nothing was downloaded or run.
articleWhich structural control (depth, segmentation, edges, blur, normals) should I weight to preserve which property, according to the ablations that report it?
Corrected 2026-10-24 - see C3-B1087-02; the original statements below are kept for history and are superseded where the correction says so. In Cosmos-Transfer1's TransferBench results, blur-visual and edge controls give the highest adherence and lowest diversity, while depth and segmentation allow more variation; raising a modality's weight in a region raised its alignment metric there (Pearson 0.93 and -0.92 in the authors' test). RealMaster's ablation found edges preserved faces better than depth for propagation. EPE found G-buffer inputs matter for realism. Normals appear as a control only in vendor text (Reallusion) and EPE's G-buffers, not in the Cosmos Transfer1 README or VACE's task table.
articleWhat Luma Ray API fields give camera-concept shots and first/last-frame anchoring (concepts, keyframes frame0/frame1, loop, extend)?
Corrected 2026-10-24 - see C3-B956-04; the original statements below are kept for history and are superseded where the correction says so. Problem. Controlling both the camera move and where a clip starts and ends. Conditions. Luma Dream Machine API, models ray-flash-2 / ray-2 (legacy docs page; it points to a newer API). Method. (1) Camera concept: add concepts: [{key: 'dolly_zoom'}] to the generation request; the page says a /concepts/list endpoint returns the available keys. (2) Start frame: keyframes.frame0 = {type:'image', url}. (3) End frame: keyframes.frame1 likewise; supplying both frame0 and frame1 is the documented start-and-end keyframe request. (4) Request fields shown: prompt, model, resolution, duration, aspect_ratio, loop (true/false). (5) Extend is documented only for generated videos in completed state. Craft use (inference): treat frame0 as the last frame of the previous shot to chain a continuous move, and frame1 as a designed arrival pose; matches 'match on actio...
articleIn Kang et al.'s controlled 2D testbed, how do in-distribution, out-of-distribution and combinatorial generalisation differ for video diffusion models, and what can not be concluded about commercial video models?
Corrected 2026-10-24 - see C3-B680-01; the original statements below are kept for history and are superseded where the correction says so. Kang et al. (ICML 2025) train diffusion-transformer video models from scratch on Box2D/PHYRE simulations so the training set is fully controlled. Within the training range the models track the simulator almost exactly; outside it (new speeds or radii) error is about 35x higher and does not shrink with 100x more data or 14x more parameters. Where every object type is seen but not every combination, scaling the number of combination templates (6 to 60) cut human-labelled 'abnormal' out-of-template videos from 67% to 10%. Attribute-pair tests show the models copy the nearest training example, preferring colour over size over velocity over shape. The setting is a 2D toy world, so the paper does not measure Sora-class or other commercial models.
articleWhat inputs and limits does YouTube document for AI Remix (reimagine a Short with prompts and photos), and which rules still apply?
Corrected 2026-10-24 - see C3-B970-03; the original statements below are kept for history and are superseded where the correction says so. Problem. AI remix changes someone else's Short with generated video. Conditions. Eligible Shorts only; English prompts only; rollout described as 'coming weeks' on an undated page. Method. In the app tap Remix on an eligible Short, enter a custom prompt or pick a suggestion, optionally upload up to 3 reference photos, wait for generation and preview; audio is generated from the prompt and scene; further edits with Shorts tools before upload. Tip from the page: include a subject and at least one reference photo, and be descriptive if specific audio is wanted. Community Guidelines apply to generated output. Failure modes. Vague prompts. Limitations. No length, watermark or opt-out detail on this page (see C3-B694-10 for guardrails). Version and date. Undated; fetched page.
articleFor each of these open interactive world models, what do the papers and READMEs say about the action space, output resolution, frame rate, context or memory, and hardware, and which cells are measured, claimed or absent?
Corrected 2026-10-24 - see C3-B981-01; the original statements below are kept for history and are superseded where the correction says so. The numbers are not comparable across systems because they mean different things: Matrix-Game 2.0's 25 FPS is generation speed on one H100 at 352x640; Hunyuan-GameCraft's 25 fps is the video frame rate while its own table lists inference speed of 0.25 FPS (6.6 with distillation) and the text says latency under 5 s per action; LingBot-World 1 claims under 1 s latency at 16 fps; LingBot 2.0 claims 720p at 60 fps without a hardware statement in the text read; Matrix-Game 4.0 takes player input in a game engine and renders low-poly frames, reporting 480p 60 FPS on an RTX 5090 with 112 ms key-press-to-display latency. Memory is explicit only in Matrix-Game 3.0/4.0 (README/report statements) and is stated as absent or emergent in Matrix-Game 2.0 and LingBot-World.
articleLuma's public API in Oct 2026 still documents only Ray 2 / Ray 2 Flash - while ComfyUI deprecated its Ray 2 nodes and Luma is absent from current AA video boards
Corrected 2026-10-24 - see C3-B976-01; the original statements below are kept for history and are superseded where the correction says so. Primary API docs list two video models - Ray 2 (ray-2) and Ray 2 Flash (ray-flash-2) - with text-to-video and image-to-video via start/end keyframes, resolutions 540p/720p/1080p/4K, durations such as 5 s, loop and aspect-ratio options, and "concepts" (camera-motion presets) retrievable from /generations/concepts/list; the FAQ says API outputs carry no watermark and may be used commercially. Signals that this surface is ageing: ComfyUI deprecated its Luma Ray 2 partner nodes in v0.39.0 (5 Oct 2026, R3), and Luma models do not appear in the Artificial Analysis T2V/I2V boards captured today (C3-B438-01). We did not find (in this pass) API documentation for a newer Luma video model; Luma's consumer app may differ. Practical: before building on Luma's API, confirm model roadmap and support; older archive guidance on Luma camera concepts/keyframes (Ray...
articleWhat is Google's documented Veo 3.1 prompt structure, and what do its first/last-frame, ingredients-to-video and timestamp workflows give for shot continuity?
Corrected 2026-10-24 - see C3-B955-07; the original statements below are kept for history and are superseded where the correction says so. Corrected 2026-10-24 - see C3-B825-04; the original statements below are kept for history and are superseded where the correction says so. Problem. Getting controlled, multi-shot output from one video model. Conditions. Veo 3.1 on Vertex AI per a Google Cloud blog dated October 15, 2025; clips of 4, 6 or 8 s, 720p or 1080p, 16:9 or 9:16. Method. (1) Five-part prompt: [Cinematography] + [Subject] + [Action] + [Context] + [Style & ambiance]. (2) Dialogue goes in quotation marks; SFX and ambient noise labelled (SFX:, Ambient noise:). (3) Negative guidance is phrased as the thing wanted ('a desolate landscape with no buildings or roads' instead of 'no man-made structures'). (4) First-and-last-frame: generate a start still and an end still (the guide uses Gemini 2.5 Flash Image), then describe the transition, e.g. 'smooth 180-degree arc shot'. (5) Ing...
articleWhat are Kling 4.0's documented limits (duration, keyframes, references, resolution, HDR, Flash), and do they match creator videos?
Corrected 2026-10-24 - see C3-B1032-02; the original statements below are kept for history and are superseded where the correction says so. Kling's own site (kling.ai, rendered 2026-10-23) announces "All-New Kling 4.0" but the release-notes URL redirected to the homepage and no spec table was retrievable: official spec not pinned. fal.ai (a host relaying Kling's spec sheet) lists: 3-30 s per generation; up to 10 keyframe images; Omni Reference up to 15 total references (images, videos, voice references, elements), reference videos up to 5 totalling 30 s; edits with up to 5 input clips; 720p/1080p/4K output with stereo audio; 10-bit HDR at 1080p/4K "coming soon"; languages incl. Chinese, English, Japanese, Korean, Spanish, Portuguese, German, French; Kling 4.0 Flash in limited early access from 2026-09-28 (20 s/720p per other secondary pages, not confirmed here). This matches the creator claims in C3-B334-02 (10 keyframes, 15 references, 30 s, HDR pending) except his "nine languages"...
articleOpen world models for worldbuilding (Oct 2026): persistent 3D worlds (HY-World 2.0) vs memory-based video worlds (WorldCrafter) vs keyboard-controlled video (H3-World)
Corrected 2026-10-24 - see C3-B1036-01; the original statements below are kept for history and are superseded where the correction says so. Three open releases with different output types. (1) HY-World 2.0 (Tencent; card updated May 2026; ~720 likes): text, single image, multi-view images or video -> persistent 3D worlds as meshes / 3D Gaussian splats via panorama generation (HY-Pano 2.0) -> trajectory planning (WorldNav) -> world expansion (WorldStereo 2.0) -> composition (WorldMirror 2.0 + 3DGS); WorldMirror 2.0 (~1.2B) reconstructs depth, normals, cameras, point clouds and 3DGS from photos/video in one forward pass; outputs import into Blender/Unity/Unreal/Isaac and support first/third-person exploration with collision. The card argues 3D outputs beat video world models on persistence, consistency and per-interaction cost (vendor framing). LICENCE: Tencent HY-World 2.0 Community License does not apply in the EU, UK or South Korea, forbids using the works OR THEIR OUTPUT outside t...
articleIs MiniMax H3 open-weights, and what does its repository say about what is and is not released?
Corrected 2026-10-24 - see C3-B1044-01; the original statements below are kept for history and are superseded where the correction says so. The MiniMax-AI/MiniMax-H3 GitHub repo (head d21241f0, 2026-08-15) links Hugging Face weights and a "MiniMax H3 Community License" (custom, not OSI; GitHub detects no licence). Its README states the H3-Context-IR preprocessing workflow is not in the open-source release (an API reproduces it), the initial release offers full-attention inference only (sparse attention later), and the H3-Encoder uses the full Qwen3-VL-32B weights (layer-50 hidden states), implying a heavy local footprint. It also ships an "h3-prompt-writing" agent skill (Markdown) usable in Claude Code and other harnesses. This reconciles creator claims: Mikkel Lassalle presented H3 as a hosted, cheaper-than-Seedance model (C3-B335-08); Mickmumpitz called it "free open-source" and used it for video cleanup (C3-B338-02). Both are partly true: open weights under a community licence, p...
articleWhat controls do vendor video-to-video tools (Runway Aleph, Luma Modify Video, Moonvalley Marey) document, and do any expose depth, segmentation or edge weights like ControlNet?
Corrected 2026-10-24 - see C3-B1045-01; the original statements below are kept for history and are superseded where the correction says so. On the pages read, none documents depth, segmentation, edge or normal control inputs or numeric control weights. Aleph is prompt-driven (action verb plus change description, optional image) with a 5 s limit; Luma Modify Video exposes nine adhere/flex/reimagine presets; Marey's ComfyUI node offers Motion Transfer or Pose Transfer from a reference video. Adherence is therefore steered by presets or prompts, not by documented structural-channel weights (bounded to these pages).
articleWhat consistency and quality evaluations have the authors of HunyuanWorld 1.0, WonderWorld, WonderJourney and World Labs published, and can their numbers be compared?
Corrected 2026-10-24 - see C3-B1039-01; the original statements below are kept for history and are superseded where the correction says so. Each source evaluates differently and none measures 3D geometric consistency directly. HunyuanWorld 1.0 reports no-reference image-quality metrics (BRISQUE, NIQE, Q-Align) and CLIP alignment on rendered views against baselines, with no ground truth; WonderWorld's only quantitative table is generation time (9.5 s vs 749.5 s and 798.1 s on an A6000) plus qualitative comparison and a stated failure mode; WonderJourney reports a human preference study (400 participants, binary choices on four axes). The World Labs pages read give examples and claims but no published metric; its TOS disclaims dimensional accuracy. The tables are not comparable across papers (different views, baselines, protocols).
articleWhich ffprobe and MediaInfo fields tell me a file's colour tags and range, and what are the value names?
Corrected 2026-10-24 - see C3-B1054-01; the original statements below are kept for history and are superseded where the correction says so. At FFmpeg commit 9047fa1b, ffprobe stream output uses the keys color_range (unknown/tv/pc), color_space (e.g. bt709, gbr, unknown), color_transfer and color_primaries, and prints unknown when unspecified; a real user paste confirms these keys. MediaInfoLib (commit 153ad67a) lists colour_range, colour_primaries, transfer_characteristics, matrix_coefficients and colour_description_present, with _Source and _Original variants. Nothing was run; the checklist below is derived from these sources.
articleWhat are Gemini Omni Flash's documented capabilities, limits, prompting rules, watermarking and price in the Gemini API?
Corrected 2026-10-24 - see C3-B978-01; the original statements below are kept for history and are superseded where the correction says so. Gemini API docs (captured 2026-10-23): gemini-omni-1.1-flash generates video with audio from text, image (reference/first frame), first+last frame interpolation and subject reference images, edits videos conversationally (multi-turn via the Interactions API) and extends videos at the tail. Output resolution 360p, 720p (default) or 1080p (upscaled); aspect 16:9 or 9:16. Limits: uploaded videos for editing/extension must be 10 s or less; extension appends to the end only; you cannot extend an uploaded talking clip to add dialogue (except via multi-turn on model-generated videos); voice editing and audio references unsupported; video references max 3 clips of up to 3 s each (audio ignored); no multi-video reasoning; no system instructions, temperature, top_p, stop sequences or negative-prompt field (write "Do not do X" in the prompt); YouTube URLs n...
articleWhat preprocessing limits and licence apply to Wan-Animate when an actor video drives a character image?
Problem: the open Wan-Animate model needs pose and face streams extracted from the actor video, and mismatches cause artifacts. Conditions: Wan2.2 repo; animation mode (character mimics actor) or replacement mode (character replaces the person in the video). Method: preprocessing outputs src_pose.mp4 and src_face.mp4, plus src_bg.mp4 and src_mask.mp4 in replacement mode; the paper abstract says body motion is replicated with skeleton signals and expressions with implicit facial features. In animation mode, enable retargeting when body proportions differ, and check the preprocessing results before generation. In replacement mode retargeting is off by default. Failure modes: significant body-proportion mismatch can cause deformation; mask extraction targets single-person videos and can fail with several people; a lower fps setting can cause choppiness. Rights: weights are Apache 2.0, with a use clause barring content that violates laws or harms individuals. Limitations: not run by me....
articleHow long do Gemini and Vertex keep Google-side files and interaction results, and where must a pipeline save outputs?
Problem. Batch inputs and Google-hosted video URIs have their own lifetimes that bound when a pipeline can read them. Conditions. Gemini API Files API and Omni video URI delivery; Vertex AI Veo interactions and storageUri. Method. Gemini Files: files are deleted automatically after 48 hours, with 20 GB per project and 2 GB per file; batch input files are uploaded through this API. Omni: videos over 4MB are returned as a Google-hosted URI to poll until ACTIVE before download. Vertex: stored and asynchronous interactions are retained up to 7 days; requests can name a Cloud Storage storageUri output location. Save outputs into your own bucket or storage on completion. Failure modes. Re-reading a 48-hour file after expiry. Limitations. The Omni page read does not state the lifetime of the delivery URI. Version and date. Docs read 2026-10-23.
articleHow does CameraCtrl II inject camera paths into a video diffusion transformer, how does it keep dynamic content, and how does it chain clips to explore a scene?
Problem. Camera-conditioned video models trained on static-scene datasets lose scene dynamics, and single clips cover a small viewpoint range. Conditions. A pretrained DiT video model, clips with estimated camera parameters, and a user who specifies a camera path per clip. Method (CameraCtrl II). (1) Per-pixel Plucker embeddings from extrinsics and intrinsics are passed through an extra camera patchify layer at the model input and added element-wise to the visual features before the first DiT layer (reported to preserve dynamics better than encoder-injector designs). (2) A new RealCam dataset of dynamic videos with estimated camera parameters replaces static datasets such as RealEstate10K, ACID, DL3DV10K and Objaverse. (3) Joint training on camera-labelled and unlabelled video, using an all-zero dummy Plucker embedding for unlabelled data; this also enables classifier-free guidance on the camera signal. (4) Clip extension: tokens of the previous clip's last frames stay clean while t...
articleWhy does OmniHuman-1 train with text, audio and pose conditions at decreasing ratios, and what inference settings (CFG on audio/text only, 5 motion frames) does it report for long talking-body videos?
Problem. Audio-driven human animation models that need clean lip-sync data throw away most video; stronger conditions such as pose dominate weaker ones such as audio. Conditions. You are choosing or evaluating a one-stage Diffusion Transformer avatar model (image + audio, optional pose) and want to understand why it behaves as it does. Method (OmniHuman-1, v3 of the paper). A three-stage mixed-condition training: stage 1 text+image only, stage 2 adds audio (pose dropped), stage 3 adds pose. Principle 1: stronger-conditioned tasks may reuse weaker-conditioned data, so clips failing lip-sync filters still train the text/image tasks. Principle 2: the stronger the condition the lower its training ratio, because with pose present the model leans on pose and under-learns audio; ratios for text, audio and pose are progressively halved when present. Data: 18.7K hours of in-house human video filtered for aesthetics, quality and motion amplitude, of which 13 percent met lip-sync and pose-visi...
articleHow do speech-to-3D-face tools relate to 2D lip-sync tools and ARKit blendshapes, and when should I choose each?
Problem. Choosing between 2D video lip-sync and audio-driven 3D facial animation. Conditions. A character that already exists as a rig or can be built as one. Method (inference). 2D lip-sync tools in C3-B576-06 repaint the mouth region of existing video, so identity and lighting come from the footage; 3D tools such as C3-B790-02 output ARKit-style weights or vertices, so identity, lighting and camera are yours to render and the animation can be edited, retimed and reused. Pick 3D when the character is a rig, needs editable curves, or must be rendered from several angles; pick 2D when the footage is fixed and only the mouth must change. The ARKit 52 names are the shared interface, as in C3-B752-08. Failure modes. Using a 3D route without a layer for eyes, brows and idle motion (see C3-B790-03). Limitations. This synthesis is mine and untested by me; sources say nothing on a head-to-head comparison.
articleWhich Wan-Alpha file names should I download for ComfyUI: the Comfy-Org repack names in the tutorial, or the names on the upstream 'ComfyUI version' repo?
Problem. The tutorial points to htdong/Wan-Alpha_ComfyUI as 'ComfyUI Version' but tells you to place differently named files. Conditions. Setting up the Wan 2.1 Alpha T2V 14B template. Method. Use the tutorial's names (wan_alpha_2.1_vae_rgb_channel.safetensors, wan_alpha_2.1_vae_alpha_channel.safetensors, wan_alpha_2.1_rgba_lora.safetensors from Comfy-Org/Wan_2.1_ComfyUI_repackaged; HEAD requests to those URLs returned HTTP 200 at fetch time). The upstream card lists the doubled-extension names (.safetensors.safetensors) and 'epoch-13-1500_changed' for the same assets, and its example run uses frame_num 81, sample_steps 4 with the LightX2V LoRA and lora_ratio 1.0, which the tutorial text does not state. The upstream card names the fp16 Wan base while the tutorial offers bf16 and fp8_scaled. Failure modes. Mixed downloads leave the Load VAE or LoRA node unable to find its file. Limitations. I did not check that the renamed files are byte-identical, and the Hub API listing of the Comf...
articleDoes Google's Merchant Center Product Studio video page say AI labels are required, and are they exempt from the overlay and watermark ban? Labels are allowed for certain jurisdictions; no compliance guarantee.
Problem. Sellers adding visible AI disclosure to a product video or image worry it breaks the no-text/no-watermark rule. Conditions. Generated or AI-edited ad assets in Merchant Center, Google Ads and related products. Method. (1) The page states that AI regulations in the European Union, India and New York require disclosures or labels on ads with certain AI-generated or edited assets. (2) Add the label inside the creative or use the AI label setting in Google Ads, Display & Video 360, Campaign Manager 360, Merchant Center and Ads Editor. (3) The page says such labels will not violate the policies on text overlays and watermarks. (4) It also says using the setting does not guarantee compliance, so take legal advice. Failure modes. Assuming the setting satisfies a given law; relying on in-creative text without checking the image-spec rules in C3-B783-02. Limitations. The statement is on the video page, so whether the same exemption governs still images in image_link is not stated th...
articleWhat do CapCut's Terms require when I use its third-party AI features (inputs, outputs, prohibited use)?
Problem. AI features in an editor pass inputs to other providers under extra terms. Conditions. CapCut Terms, section on Third-Party AI Services. Method. The Terms say some features integrate third-party AI technologies and APIs (examples named: Runway, Stable Diffusion, Google, YouTube, FLUX, Luma) subject to those providers' terms; if the user does not agree to them they must not use those features. User responsibilities: have all rights, licences, permissions and publicity clearances for inputs; do not use outputs in ways that violate others' IP, privacy or proprietary rights; do not attempt to bypass limits or content filters. The Terms disclaim responsibility for third-party AI. Inference: for client work, a face, voice or logo supplied as an input needs a release covering AI processing, and the creator, not CapCut, is the stated responsible party for output misuse. Failure modes. Feeding client-owned or celebrity material into AI features. Limitations. The Terms do not (in the...
articleWhat do HunyuanVideo's Normal and Master prompt-rewrite modes do, and how does HunyuanVideo-1.5 default to rewriting when a vLLM endpoint is configured?
Problem. Rewriters trade intent-fidelity for visual richness, and a pipeline flag can turn rewriting on without the user noticing. Conditions. HunyuanVideo (Dec 2024 release) and HunyuanVideo-1.5 repositories. Method. HunyuanVideo fine-tunes a Hunyuan-Large model as a rewriter with two modes: Normal improves understanding of user intent; Master enriches composition, lighting and camera movement for higher visual quality. The README states that Master may lose some semantic details, so use Normal when the brief is specific, Master when it is vague. Weights for the rewriter are published separately. HunyuanVideo-1.5 instead ships system prompts (t2v and i2v) and recommends vLLM-served Qwen3 models (Qwen3-235B-A22B-Thinking-2507 for text-to-video, Qwen3-VL-235B-A22B-Instruct for image-to-video); rewriting is on by default (--rewrite true) but, if no vLLM endpoint is configured, the pipeline simply runs without remote rewriting. Failure modes. Semantic detail loss in Master mode; a sile...
articleHow does WorldRoamBench test whether an interactive world model remembers a place after revisit, why does it avoid frame-pair matching, and what did it find about memory decay?
Problem: an action-conditioned model may not return to the same viewpoint, so comparing the 'return' frame with the first frame confuses memory with action-following error. Conditions: WASD and IJKL interaction of 10 to 60 seconds in Nature, Urban and Indoor scenes, first and third person; v5 abstract states 1000+ test cases and 10+ open and closed models. Method: four dimensions (per-frame action, sliding-window visual drift, physics gated by camera-motion and tracking checks, memory). Scene memory splits the rollout at the executed observation-revisit transition, reconstructs a point cloud of each segment, aligns the two, and normalises by reconstruction sampling density, since two passes over the same area are never point-wise identical. Subject memory uses tracking plus a VLM. Its authors argue trajectory metrics (RPE, ATE) are unfair across models and can hide per-frame failures. Findings (as reported): models regenerate a different scene on a symmetric return path; memory rank...
articleHow do I request a native portrait video from Google's Gemini API (Gemini Omni Flash), and what resolution values does it allow?
Problem: short-form needs 9:16 and an agent must know whether the model can render it natively. Conditions: Gemini API, model gemini-omni-1.1-flash via the Interactions API; the page also says the Veo 3.1 preview models in the Gemini API were shut down on 22 Oct 2026. Method: in response_format set type video and aspect_ratio to '9:16' (supported values listed as 9:16 and 16:9; 16:9 is the default); set resolution to 360p, 720p (default), 1080p or 4k. Example: the page's request sets response_format.resolution to '1080p'. Failure modes: the table labels 1080p and 4k as 'upscaled', so the model's native render is not stated to be 1080p; check the output file's dimensions rather than assuming. Only two aspect ratios are listed, so 1:1 or 4:5 need reframing. Limitations: page is a guide, not the full reference; no prices recorded. Version/date: fetched 2026-10-23.
articleWhich reference inputs do current video-generation vendor docs accept for keeping a character or subject the same across separate generations, and what limits and failure cases do they state?
Corrected 2026-10-24 - see CUR-0618-02; the original statements below are kept for history and are superseded where the correction says so. Vendor docs read on 2026-10-22 offer three different mechanisms, not one: (a) reference images that condition a generation (Google Veo 3.1: up to 3 images of a person, character or product, and 8 s output only; Runway's hosted third-party models: e.g. up to 7 for Kling O3, 10 images plus 5 videos plus 5 audio for WAN 3.0), (b) first/last or multi-position keyframes (Luma Ray 3.2: 1-64 keyframes, exclusive with start/end frame), and (c) a reusable character asset (OpenAI Sora, non-human only, name must be in the prompt; the Sora API was shut down on 2026-09-24). None of the pages read publishes a measurement of cross-shot identity; Runway's own Gen-4 / Gen-4.5 image-to-video entries list only a single prompt image in the docs read. Vendor statements are claims; Kling's docs could not be read.
articleGiven only these public pages, how should an agent build and caveat a cost estimate for an AI-assisted shot list?
Corrected 2026-10-24 - see CUR-0618-01; the original statements below are kept for history and are superseded where the correction says so. Proposed procedure (synthesis): list shots with duration, resolution and audio; map each to a price cell in one unit family; round durations up to the vendor allowed values; multiply by an assumed take count (no source states a retake rate); add stills at per-image prices; check against spend and concurrency caps; report a range with capture dates. Worked example, 12 shots x 6 s x 3 takes = 216 generated seconds at 720p: Veo 3.1 Lite $10.80, Gemini Omni Flash about $21.89, Veo 3.1 Fast $21.60, Runway gen4.5 $25.92, Veo 3.1 Standard $86.40 (arithmetic in work/estimate.py; assumptions, not measured). The sources limit this: failed-generation billing, retakes, taxes and account discounts are mostly unstated, and some pages disagree with themselves.
articleWhat do the public pages state for video generation price per second by resolution, audio and duration tier, across Google, Runway, Luma, fal, Replicate and Pika?
Corrected 2026-10-24 - see CUR-0618-01; the original statements below are kept for history and are superseded where the correction says so. Dated 2026-10-22 captures: Gemini API Veo 3.1 Standard $0.40/s (720p and 1080p), $0.60/s 4k; Fast $0.10/$0.12/$0.30 per s at 720p/1080p/4k; Lite $0.05/$0.08 per s; Omni Flash about $0.10/s at 720p from a token rate. Runway API lists per-second credits (credit = $0.01) for many third-party models, e.g. veo3.1 with audio 40 credits/s and gen4.5 12 credits/s. Luma Ray3.2 lists per-clip prices at 540p/720p/1080p. fal lists Kling v3 Standard at $0.084/s audio off and $0.126/s audio on on the model page. Pika sells consumer credits. Resolution, audio, and duration tiers change the unit price; Veo 1080p and 4k are limited to 8 s.
articleWhat licence terms are stated for AnimateDiff code and motion modules, and how do the Hugging Face tag and the GitHub 'academic use' disclaimer differ?
Problem. AnimateDiff motion modules run on top of SD1.5 checkpoints, and users download them from several hosts. Conditions. GitHub guoyww/AnimateDiff commit e92bd567 (pushed 2024-07-31; API reports Apache-2.0); HF guoyww/animatediff revision fdfe36af (modified 2023-12-18). Method. The HF model card is metadata only: 'license: apache-2.0' plus a line saying it hosts models for AnimateDiff. The GitHub README's Disclaimer states 'This project is released for academic use' and disclaims responsibility for user-generated content; the README states the main branch targets Stable Diffusion v1.5 (an sdxl-beta branch targets SDXL). So the repository licence file is Apache-2.0 while prose says academic use: the sources do not reconcile these. Motion modules are also distributed through Google Drive and CivitAI per the README, and personalised SD1.5 checkpoints used with them have their own licences (SD1.5's CreativeML OpenRAIL-M is covered by earlier items). Failure modes. Citing 'Apache-2.0...
articleWhat licence terms are actually stated for FramePack weights (lllyasviel/FramePackI2V_HY) and code, given the model is built on HunyuanVideo?
Problem. FramePack is run in ComfyUI through Kijai's wrapper, but its weight licence is not obvious. Conditions. HF lllyasviel/FramePackI2V_HY revision 86cef439 (modified 2025-04-13); GitHub lllyasviel/FramePack commit 97fe5dbe (2025-10-16). Method. The GitHub repository carries an Apache License 2.0 file and the API reports Apache-2.0 for the code. The Hugging Face weights repository has tags diffusers/safetensors and a model card whose only text is a one-line description ('...FramePack for HY'), with no licence field and no licence file in its file list. The GitHub README mentions neither Hunyuan nor Tencent in my search, so no licence statement for the weights was found in the sources read. 'HY' in the name and the Kijai wrapper README (round 2) indicate a HunyuanVideo-based transformer; whether the Tencent Hunyuan Community License (territory exclusion, 100M MAU, no-improving-other-models) propagates to it is not stated by these sources (inference only). Failure modes. Reading t...
articleWhich licence applies to which LTX-Video / LTX-2 / LTX-2.x weights, what are the revenue and non-commercial rules, and what output and competing-model clauses exist?
Problem. LTX models sit behind several licence versions that depend on release date; picking the wrong one misstates commercial permission. Conditions. Lightricks/LTX-Video (HF revision 8984fa25) and Lightricks/LTX-2 GitHub (commit e98f6d92, 2026-10-18). No LTX items were found in the prior index, so this is new. Method. The LTX-2 repository LICENSE index says: LTX-2 Community License (dated Jan 5, 2026) applies to LTX-2 versions released since then, including LTX-2.3 until Aug 11, 2026; the LTX-2.x Community License (dated Aug 11, 2026) applies to LTX-2.5 versions released since then and future LTX-2.x releases. Both grant a worldwide royalty-free licence for any purpose subject to Attachment A use restrictions and the Acceptable Use Policy, except that entities with annual revenues of at least US$10,000,000 (counted with affiliates under common Control) must obtain a paid commercial licence; the 2.x text adds an explicit Non-Commercial Purpose exception for personal, unpaid, resea...
articleWhat does the Tencent Hunyuan Community License (HunyuanVideo) restrict by territory, user count, derivative models and outputs, and what notices must accompany redistribution?
Problem. HunyuanVideo (and models built on it) is common in ComfyUI video stacks, but its licence differs sharply from Apache. Conditions. tencent/HunyuanVideo on Hugging Face, revision 6204ad6a (modified 2025-03-06), files LICENSE and Notice; licence card name tencent-hunyuan-community; release date of the model licence Dec 3, 2024; GitHub licence metadata for Tencent-Hunyuan/HunyuanVideo reads NOASSERTION. Method. Territory: the agreement states it does not apply in the European Union, United Kingdom and South Korea, and that use of the Works or any Output outside the Territory is unlicensed (section 5c). Scale: if the licensee's products exceeded 100 million monthly active users in the month before the version release date, a separate licence must be requested and no rights apply until granted (section 4). Output and derivatives: you must not use the Works or any Output or results to improve any other AI model (section 5b); Model Derivatives include distillation and training on s...
articleWhat licence covers Wan 2.1, Wan 2.2 and Wan2.2-Animate weights and generated outputs, and what duties does the model card add?
Problem. Deciding whether Wan weights used in ComfyUI workflows can be used and redistributed commercially. Conditions. Hugging Face repos Wan-AI/Wan2.1-T2V-14B (revision a064a6c7, LICENSE.txt present), Wan2.2-I2V-A14B (revision 206a9ee1) and Wan2.2-Animate-14B (revision cb93a225); the VACE-14B card also tags Apache-2.0 (not read in full). Method. The card front matter says license: apache-2.0 for all three revisions read, and the Wan2.1 repository carries the standard Apache License 2.0 text as LICENSE.txt. Each card's License Agreement section states that the models are licensed under Apache 2.0, that the authors claim no rights over generated contents, and that the user remains accountable and must not share content that violates law, causes harm to individuals or groups, disseminates personal information intended for harm, spreads misinformation or targets vulnerable populations. Practical matrix: code Apache-2.0 (GitHub Wan-Video/Wan2.1 and Wan2.2 report Apache-2.0); weights Ap...
articleWhich older ComfyUI motion-control node packs (DragNUWA, InversedNoise) are stale or risky to install, and what do their dependency pins and weight downloads imply?
Problem. Tutorials still point to old trajectory-drag and unsampling nodes; an agent must judge installability and risk. Conditions. chaojie/ComfyUI-DragNUWA (commit bd9f53fe, last commit 2024-01-29, MIT) and logtd/ComfyUI-InversedNoise (commit acd77730, last commit 2024-03-31; no LICENSE file found at that commit). Method. DragNUWA's requirements pin old packages (for example kornia==0.6.9, opencv-python==4.6.0.66, pytorch-lightning==2.1.0, black==23.7.0) and install clip from a git URL, so installing into a modern ComfyUI environment can downgrade shared packages; weights are a .pth from Google Drive (a pickle-family checkpoint) or a pruned fp16 safetensors from a third-party Hugging Face repo, with Chinese-mirror wget guidance. It provides trajectory drawing, camera/object motion brushes and optical-flow workflows. InversedNoise's README calls itself experimental/WIP for DDIM/sample inversion (unsampling) in image-to-image and offers no further documentation. Failure modes. Depen...
articleWhat constraints does ComfyUI's Wan2.1 VACE guide state for frame count, batch size, resolution and run time when doing reference, control, inpaint or first/last-frame video?
Problem. Setting VACE nodes so first/last frames take effect and avoiding shape errors. Conditions. docs.comfy.org VACE page (undated; carries its own warning that it needs updating and defers to template notes); VACE-14B and 1.3B models, Apache-2.0 per the page. Method. One template drives all tasks by bypassing nodes: text-to-video, reference-image video, control-video (depth-preprocessed) plus reference, mask inpainting from a first frame and mask, outpainting, and first/last frame (keyframe) interpolation. Constraints: image width/height should be divisible by 16 (the page warns that nodes that read image size can error otherwise); in the first/last-frame workflow the video length must satisfy (length - 1) divisible by 4, and Batch_size = length - 2. Models: wan2.1_vace_14B_fp16.safetensors (about 32 GB) or the 1.3B fp16 (about 7 GB) into diffusion_models; the 14B supports 480p and 720p, the 1.3B only 480p. The page reports on a 4090: 720x1280 for 81 frames about 40 minutes, 640...
articleWhat does AnimateDiff-Evolved support (context windows, ControlNet, IPAdapter, camera control), where do model files go, and what version of ComfyUI does it require?
Problem. Building controllable SD1.5/SDXL animation workflows in ComfyUI. Conditions. Kosinkadink/ComfyUI-AnimateDiff-Evolved, commit 92576512 (pushed 2026-07-28), version 1.6.0, Apache-2.0 code, registry requires-comfyui >=0.3.68, no pip dependencies. Method. Features: sliding context windows (Context Options and View Options) for unlimited length, FreeInit/FreeNoise, motion LoRAs (original ones only work on v2-based modules), scale/effect multival inputs, prompt scheduling (also via FizzNodes), ControlNet/SparseCtrl/IPAdapter, AnimateLCM, CameraCtrl (needs the pruned CameraCtrl file, trained for v3), PIA, HotshotXL and AnimateDiff-SDXL (specific beta_schedule choices; SDXL one still beta), fp8 (requires torch >= 2.1 and changes outputs), and per-frame GLIGEN. Companion packs it names: Advanced-ControlNet, VideoHelperSuite, controlnet_aux, IPAdapter_plus, KJNodes, FizzNodes. Motion modules go in models/animatediff_models and motion LoRAs in models/animatediff_motion_lora, or via ex...
articleWhat do ComfyUI's Wan2.2 Animate and Fun Camera guides require (custom nodes, model pairs, speed LoRA) and what do they say about the speed/dynamics trade-off?
Problem. Reproducing character animation/replacement or preset camera moves with Wan2.2 in ComfyUI. Conditions. docs.comfy.org pages (undated); require a recent ComfyUI (templates ship with it). Method. Animate has two modes: Mix (replace a character in a video with the reference-image character) and Move (animate the reference character with the input video's motion, like Wan2.2 Fun Control). It needs two custom-node packs, ComfyUI-KJNodes and comfyui_controlnet_aux (the DWPose Estimator turns the input video into pose and face control videos), models wan2.2_animate_14B_bf16 or Kijai's fp8-scaled file, clip_vision_h, wan_2.1_vae, umt5_xxl fp8 scaled and an optional LightX2V 4-step LoRA. Fun Camera uses a pair of high-noise and low-noise 14B models (fp8_scaled) plus matching high/low 4-step LoRAs, with preset Pan Up/Down/Left/Right and Zoom In/Out and combinations. The page reports on an RTX 4090D 24GB at 640x640, 81 frames: fp8_scaled about 536 s first run and about 513 s second; w...