Video generation

Text/image/video-to-video, motion control, long-form consistency and model pipelines

• articleHow do you build a consistent multi-character cast with LoRAs (dataset, captions, training steps) for image and video models, per a July 2026 local workflow?
  Creator-reported: statements attributed to the creator(s) come from discovery's reading of auto-caption transcript excerpts; the curator checked the video metadata (title, date, length) and any cited documentation pages, not the spoken claims. Nothing was tested; promotional or sponsored framing applies where stated. Mickmumpitz (2026-07-17) gives five LoRA rules: (1) a made-up trigger word absorbs everything not captioned; (2) vary what should generalize (include wide shots, lighting, poses, clothing), or the character only exists as trained (e.g. close-ups); (3) decide what stays fixed versus editable; (4) captions decide editability: captioned clothes stay changeable but must then be re-prompted; (5) captions are reverse prompts, so match the target model's prompt style (long natural language for Krea 2; structured JSON with bounding boxes for Ideogram 4). Pipeline: an edit-model "character creator" generates a varied synthetic dataset from one or two images (add close-ups of det...
• articleHow can you build a whole AI short shot-by-shot locally with consistent characters and voices (LTX 2.3 + FLUX.2 Klein), and what licences apply?
  Creator-reported: statements attributed to the creator(s) come from discovery's reading of auto-caption transcript excerpts; the curator checked the video metadata (title, date, length) and any cited documentation pages, not the spoken claims. Nothing was tested; promotional or sponsored framing applies where stated. Mickmumpitz (2026-05-04): per shot, FLUX.2 Klein generates a start frame from up to 4 references (character face close-up + slightly turned full-body image auto-merged into a sheet; environments/assets too), then LTX 2.3 generates video with audio; an "LTX ID" LoRA plus a voice sample keeps the character voice consistent; each shot group can reference previous shots and color-match the first shot; a shot duplicator and assembler build the edit, and swapping the reference images and voice sample re-casts the whole film without touching prompts (keep character description vague in prompts for this). Paid version adds a 360-degree environment and crop group for consistent ...
• articleHow does a local generative video upscaler (Wan 2.2 tiled) compare with Topaz and FlashVSR, and what settings matter?
  Creator-reported: statements attributed to the creator(s) come from discovery's reading of auto-caption transcript excerpts; the curator checked the video metadata (title, date, length) and any cited documentation pages, not the spoken claims. Nothing was tested; promotional or sponsored framing applies where stated. Mickmumpitz (2025-11-25): Wan 2.2 low-noise model in 81-frame batches with overlap blending and tiling; FP8 for 24 GB+ GPUs, otherwise GGUF sized to VRAM (Q5 for 12 GB; he uses Q8); LightX2V low-noise LoRA for fewer steps; optional realism LoRA; an ESRGAN-style pre-upscaler; auto-captioned prompt. Settings: target 2K as a good default; a quality pass adds ~30% quality for ~2x time (skip at 4K); "creativity" (denoise) low for fidelity, higher for degraded footage (risk visible tiles/subject change); save frames as fallback because stitching very long 4K outputs can crash ComfyUI. ~10 minutes for 81 frames; RunPod RTX 5090 ~$0.89/h. His comparisons: his 2K beat Topaz 4K o...
• articleHow can you do markerless AI performance capture (turn an actor into a creature) with masked video-to-video, and what goes wrong?
  Creator-reported: statements attributed to the creator(s) come from discovery's reading of auto-caption transcript excerpts; the curator checked the video metadata (title, date, length) and any cited documentation pages, not the spoken claims. Nothing was tested; promotional or sponsored framing applies where stated. Mickmumpitz (2026-10-07) uses MiniMax H3's reference variant in ComfyUI: feed the original performance video, lock audio to the source, and in-paint only inside a mask (auto-mask from text such as "head, helmet, phone, cable" or a manual Resolve/After Effects matte) so the camera stays locked and the result composites over the plate. Identity is fixed with a character sheet; expressive emotion sheets gave much better facial performance than neutral sheets ("keep the character's identity, look, and art style from picture number one"). Findings: mask size drives head size; full-body masks enable whole-body transformation; partial masks spanning head and lower body produce...
• articleWhat practical rules help AI-rendered animation keep continuity and correct actions (guide frames, prompts, start/end frame fixes)?
  Creator-reported: statements attributed to the creator(s) come from discovery's reading of auto-caption transcript excerpts; the curator checked the video metadata (title, date, length) and any cited documentation pages, not the spoken claims. Nothing was tested; promotional or sponsored framing applies where stated. From the Bare Minimum short (V2, LTX 2.3): describe shot events in order, then expressions; specify handedness explicitly (a wrong-hand reach was fixed only by prompting "reaching... with her left hand"); unseen props (a newspaper) become gibberish when interpolated from a distant guide frame - paint a corrected frame in Photoshop and add it as another guide frame; when start and end frames generated separately drift in minor details, LTX warps between them - composite the start and end frames together manually so shared details match; add a second guide frame early when one is not enough; dialogue audio should be clean dialogue, at least as long as the clip, or the mod...
• articleHow do you "AI-render" a 3D layout locally (3D passes -> video model), and how did the best local approach change from Feb to Aug 2026?
  Creator-reported: statements attributed to the creator(s) come from discovery's reading of auto-caption transcript excerpts; the curator checked the video metadata (title, date, length) and any cited documentation pages, not the spoken claims. Nothing was tested; promotional or sponsored framing applies where stated. Three Mickmumpitz videos trace one pipeline across 2026. Feb (V1): Inner-Reflections' VACE + SkyReels V3 R2V merge (Apache-2.0 card) reads control passes (VACE) and reference images (SkyReels); export an outline pass (Blender Workbench flat black + white outlines, or Freestyle as a render pass) and a normalized/inverted Z depth pass with a contrast curve; outlines keep composition but let the model invent detail, depth follows geometry more tightly, blend both if needed; keep control-net strength low for start-frame image generation (Z-Image Turbo + control nets); free workflow ~120 frames at 24 fps; paid version chains 81-frame chunks with 11 overlapping start frames f...
• articleHow can you puppeteer AI video with physical props or rough animation (Time-to-Move), and how reliable is it?
  Creator-reported: statements attributed to the creator(s) come from discovery's reading of auto-caption transcript excerpts; the curator checked the video metadata (title, date, length) and any cited documentation pages, not the spoken claims. Nothing was tested; promotional or sponsored framing applies where stated. Mickmumpitz (2025-12-17): Time-to-Move (TTM, Apache-2.0 repo) is training-free and works with diffusion video models (he uses Wan 2.2; paper also names CogVideoX and SVD). Make a control video by moving toys/cutouts on a stick or by rough After Effects/Blender animation; TTM's "dual-clock denoising" uses lower noise where the character moves (follow the motion) and higher noise elsewhere (clean background). His ComfyUI prep: SAM 3 masks the character (add negative clicks on the stick), Qwen Image Edit removes the stick in start/end frames (also fix hovering by masking larger and prompting "stand on the table... do not change the look"), then TTM with start+end frames fo...
• articleHow do you build a local AI VFX shot pipeline with mask + driving video + control nets + reference images (Wan VACE/Phantom merge), and what fails?
  Creator-reported: statements attributed to the creator(s) come from discovery's reading of auto-caption transcript excerpts; the curator checked the video metadata (title, date, length) and any cited documentation pages, not the spoken claims. Nothing was tested; promotional or sponsored framing applies where stated. Mickmumpitz (2026-03-30) uses a community merge of Wan 2.1 VACE (in/outpainting guided by depth, Canny edges, pose) with Phantom (up to four reference images for identity), published as Inner-Reflections/Wan2.1_VACE_Phantom. Building blocks: (1) black-and-white mask (only white is regenerated), (2) driving video = modified plate + control signal, (3) reference images (one = start frame; more = style freedom), (4) detailed prompt (LLM-"proofed" but without camera description). Controls: tracking markers drawn as white squares let the model reproduce camera motion; Canny keeps exact detail, depth allows more new detail; OpenPose plus a back-view reference kept a creature ...
• articleHow can you generate an explorable 3D Gaussian-splat world from one image or prompt locally, and which world-generation approaches failed on consumer hardware (per Mickmumpitz, Aug 2026)?
  Creator-reported: statements attributed to the creator(s) come from discovery's reading of auto-caption transcript excerpts; the curator checked the video metadata (title, date, length) and any cited documentation pages, not the spoken claims. Nothing was tested; promotional or sponsored framing applies where stated. Mickmumpitz (2026-08-25) reports weeks of R&D: Apple SHARP (single-image to splat in one pass) stitched across 360-degree slices produced seams and fell apart off-axis, and its licence did not suit commercial users; Insta360 UniSarp (360 to 3D) degrades quickly when moving; HunyuanWorld 2.0 needs two large models resident and targets 4 GPUs; NVIDIA Lyra ran but needs 91 GB of checkpoints, Linux-only, 6-minute first start; World Labs Marble is fast and crisp but restricts movement area and lacks working reflections. Chosen base: Skywork Matrix-3D (MIT): 360 panorama -> depth (MoGe) -> rough mesh -> render camera paths -> Wan 2.1 video model + a lightweight LoRA in-paints...
• articleWhat frame counts, resolution rules and frame rates do the READMEs of LTX-Video, CogVideoX, Mochi, Wan and HunyuanVideo require, so a shot of a given duration can be specified correctly?
  Partially verified: the curator re-read none of the cited sources; S12, S17, S6, S8, S19, S4, S3 rest on the producer's pinned reading. Frame counts have model-specific forms: Wan 4n+1 (defaults 81 and 121); LTX-Video 8n+1 with sizes divisible by 32 (padding otherwise); CogVideoX 8N+1 with N<=6 (default 49) for the original models and 16N+1 with N<=10 (default 81) for CogVideoX1.5; HunyuanVideo 129 in every listed setting; Mochi: README example uses 480x848 at 31 frames, rule not stated in the README. Frame rates stated: Wan 16 fps (24 for TI2V-5B), CogVideoX 8 fps (1.5: 16 fps), Wan2.2 TI2V-5B 24 fps. Duration = frames / fps, e.g. CogVideoX 49/8 = about 6 s as the README says.
• articleWhat do the Wan2.1 and Wan2.2 READMEs and configs state about memory, offloading flags, supported sizes, frame counts and frame rate for each variant?
  Partially verified: the curator re-read none of the cited sources; S1, S2, S6, S3, S4, S5 rest on the producer's pinned reading. Wan2.1: T2V-1.3B supports 480P (832*480); 14B supports 480P and 720P; OOM is handled with --offload_model True and --t5_cpu; the time/peak-memory table is an image. Wan2.2 (commit 1ea34ff48f87168174e12956e200b1d908b1c5ff): A14B variants state at least 80GB VRAM for the 1280*720 command, TI2V-5B states at least 24GB at 1280*704 with offload, dtype conversion and T5 on CPU; defaults are 81 frames at 16 fps (A14B family) and 121 frames at 24 fps (TI2V-5B). The Wan2.2 SUPPORTED_SIZES list in code does not include sizes beyond those in the README. The efficiency table with per-GPU seconds and peak GB is a PNG, so the actual numbers are not in text.
• articleWhat rate, concurrency and spend limits do the services publish, and how should a batch of shots be scheduled against them?
  Partially verified: the curator re-read none of the cited sources; S11, S13, S2, S9, S16, S18, S20 rest on the producer's pinned reading. Published limits are of different kinds. Runway: per-model concurrency by tier (example: Tier 3 allows 5 concurrent Gen-4.5), no requests-per-minute cap, excess tasks queue as THROTTLED, a rolling daily generation cap with 429, and a monthly spend cap. Luma legacy Build tier: Ray 10 concurrent generations and 20 create requests per minute, $5,000 per month. Gemini: RPM/TPM/RPD plus IPM for image models, per project, with spend limits per rolling 10 minutes ($10/$50/$200 by tier); video-model values are only in AI Studio. OpenAI tiers cap monthly usage ($100 to $200,000); model limits are in the console. fal: 2 concurrent requests for new accounts rising to 40. Replicate: 600 prediction requests per minute. Several numeric tables are JavaScript or console only.
• articleAnswers "how do procedural systems keep feet in contact and the body supported?" from engine documentation. Plant (contact) settings (R1): speed_threshold - "Bone is considered planted below this speed" (speed from an FK speed curve or graph root motion); unalignment_speed_threshold - speed at...
  Answers "how do procedural systems keep feet in contact and the body supported?" from engine documentation. Plant (contact) settings (R1): speed_threshold - "Bone is considered planted below this speed" (speed from an FK speed curve or graph root motion); unalignment_speed_threshold - speed at which the foot becomes fully unplanted, the range between the two approximating the foot's roll phase (the docs themselves note this "feels innaccurate most of the time"); distance_to_ground - within this distance of the plant plane the bone is planted and fully aligned; unplant_radius and unplant_angle - linear and angular deviation that release the lock; replant_radius_ratio/replant_angle_ratio - below these fractions of the unplant limits the foot replants (hysteresis); min/max_extension_ratio bound how far the leg chain may extend; separating_distance keeps the feet apart; ankle_twist_reduction limits slope alignment twist; adjust_heel_before_planting; reconstruct_world_plant_from_velocity...
• articleAcross China's Measures, Korea's Article 31, California's AI Transparency Act and New York's synthetic-performer ad law, which media, actors and label forms does each text address?
  Scope: synthesis bound to the dated legal-text observations C2-B60-K4-06, -07 and -08; it is not additional evidence. Curator-checked synthesis across China, California and Korea texts: media scope, label form and addressed actors differ as offered; technical specifics are delegated outside the statute in China (national standard) and Korea (decree). The cross-text term mapping is synthesis, not any source's statement.
• articleWhich Odyssey statements about Odyssey-2 Pro, Max and Odyssey-3 are documented capabilities, which are vendor benchmark claims with stated conditions, and which are marketing wording?
  A scoped ledger. Documented capability (SDK docs): text/image input, 150 s streams, 429 limits. Benchmark claims with conditions: Odyssey-3 Pro 66.1 on Physics-IQ Verified is the best-of-8 entry (the prompt-enhanced entry is 63.4); Max's "highest physics score" uses a vendor-chosen comparator set and partly the vendor's own measurements, and the table caption claims superiority over bidirectional video models that the next sentence says were excluded. Marketing or roadmap: "can run for minutes", robot and driving demonstrations (data sizes but no success rates in the text read). The blog's "quality settings" and "target simulation length" for Simulations are not fields in the documented options object.
• articleWhat resolution, frame rate and latency does Odyssey state for Odyssey-2 Pro and Odyssey-3, and which figures are documented API behaviour versus marketing or benchmark settings?
  For Odyssey-2 Pro the docs give 1280x704 / 704x1280 (with "may vary by model") and a 50 ms frame interval; the launch blog says 720P at 22 FPS. These do not state the same quantities (704 vs 720 lines; 50 ms = 20 per second vs 22 FPS) and no measurement protocol is given. For Odyssey-3 the only resolutions found are benchmark settings (832x480 and 1280x720); no frame rate or latency appears on either Odyssey-3 launch page (searched in extracted text).
• articleHow long can an Odyssey interactive session run, and what concurrency, idle, queue and quota limits does the documentation state?
  The docs state per-stream 150 s and per-connection 60 min defaults, a 15-minute idle timeout, plan-dependent concurrent streams (Free example: 5), a 10-job simulation queue returning 429 when full, and non-refreshable 10-minute browser tokens. A restart after the 150 s lease costs a documented 1-2 s visual gap unless two clients roll over. Marketing wording ("can run for minutes", "120+ seconds of generation") is looser than these figures. All limits are for Odyssey-2 Pro; no limit for Odyssey-3 or Max was found.
• articleWhat input types does the documented Odyssey API accept (text, image, actions) and what does it return?
  In the JavaScript SDK pages read, input is a text prompt plus an optional start image (25 MB cap, resized to 1280x704 or 704x1280) and further text "interact" prompts; scripted simulations add timestamps. No keyboard, gamepad or camera-pose action type appears in those pages. Outputs are a WebRTC video stream, optional broadcast to spectators, and per-stream recordings (MP4, JSONL event log, JPEG thumbnail, preview MP4). A default prompt-expansion step is documented.
• articleHow does a developer actually get access to Odyssey Systems models (Odyssey-2 Pro, Odyssey-2 Max, Odyssey-3), and which of them does the self-serve developer API document?
  At the captures read (2026-10-22), Odyssey documents a self-serve developer API (JavaScript and Python SDKs, API key from a developer portal) only for Odyssey-2 Pro. Odyssey-2 Max is stated to be in private beta for selected partners, and Odyssey-3 is a research preview in a signed-in app plus a contact route. The "Get API Access" link on the Odyssey-3 launch page leads to the same developer portal, whose documentation index names Odyssey-2 Pro only, so no documented API for Odyssey-3 was found.
• articleIn ComfyUI's cache, exactly what happens when a node's IS_CHANGED / fingerprint_inputs returns NaN, raises, or returns a changing value, and does it affect downstream nodes?
  At the pinned commit the fingerprint value is one component of each node's signature, and the signature of a node also contains its ancestors' fingerprints. A NaN, a fresh float NaN produced by an exception, or any value that differs between prompts therefore gives a different cache key, so the node and everything downstream of it is not found in the output cache and runs again. The docs state this for NaN (NaN is unequal to anything); the code and a maintainers' comment agree, but the mechanism is Python equality/hash of nested frozensets, which was read, not run. One edge is inferred from code reading only: a NaN object that is the same object every run could compare equal by identity.
• articleHow does group offloading work in diffusers v0.41.0 for video models (block_level vs leaf_level, streams, disk), which combinations raise errors, and which numbers exist for it with hardware?
  Group offloading (ModelMixin.enable_group_offload, DiffusionPipeline.enable_group_offload, hooks.apply_group_offloading) moves groups of layers between CPU/disk and the accelerator. The source at the pin turns several documented cautions into hard errors: block_level needs num_blocks_per_group; record_stream needs use_stream; use_stream needs a non-CPU backend with streams; Accelerate hooks on the same module conflict; model/sequential CPU offload on a pipeline that has any group-offloaded component raises; disk offload with torchao tensors raises. Documented softer cautions: weight-dependent device casting may clash, streams need about 2x CPU memory of the model, block_level with streams is forced to 1 block per group, VAEs with streams and tiling need a dummy forward pass. Timing/memory numbers linked by the guide are a Flux table (no GPU named) and a Wan table (no GPU, size or resolution named), both from a proof-of-concept PR; no video-model benchmark with named hardware was fou...
• articleWhat changed in ComfyUI v0.39-v0.42 (5-19 Oct 2026) that matters to AI video/image production?
  From the GitHub release notes. Camera/shot control: CameraAngle node outputting camera_info and a shot prompt, CreateCameraInfo node + CameraInfoState widget (v0.40.0). Reference/latents: ReferenceMultiLatent node (v0.41.0), latent-geometry nodes, LatentOperationBlend with start/end percent. Colour pipeline: ImageSDRToHDR node (v0.41.0); Save EXR gains 16-bit float and it becomes the default for "exr", EXR compression, EXR previews in assets; higher-quality Save Video encoding defaults (v0.39.0). Local models: MiniMax H3 support work (LightVAE, LynnReal light VAE, tiled VAE blending, single images decoded as first latent), FLUX 3 Image (v0.41.0), MOSS-image (v0.42.0; v0.42.1 fixes crashes in its template), Qwen 3.x/3.5 speedups and low-VRAM fixes, native LoRA stack loader nodes. Operations/security: "governance enforcement for custom nodes" (PR #16167, v0.40.0); new --offline and --disable-partner-nodes flags, --disable-api-nodes deprecated (v0.39.0); cryptography dependency removed...
• articleWhich of the multi-shot and identity-consistency systems and benchmarks in these papers are actually released, as what (code, weights, data), and under which stated terms?
  Released: HoloCine (14B full and sparse weights as .safetensors on Hugging Face, code, CC BY-NC-SA 4.0), StoryMem (code, Hugging Face weights, S-Lab licence non-commercial; Hugging Face metadata says CC BY-NC 4.0), and OpenS2V-Eval (code Apache-2.0). Partly released: MuSS (data-construction code and YouTube ID lists only). Not found released: EntityBench data/code, EntityMem, PersonaShot, MuSS checkpoints. Both HoloCine and StoryMem require separately downloaded Wan2.x base weights (HoloCine README names .pth files, a pickle-capable format) whose terms were not read here. None of the licences were interpreted for output use.
• articleWhich systems claim to steer a running world model or long video with text, how was that steering actually tested, and what did the tests show?
  Four measured protocols for text steering exist in 2025-2026 preprints, plus one system (Zing-0.5) that shows it only qualitatively, and each measures something different. Hunyuan-GameCraft-2 uses a VLM judge on six interaction dimensions over 100 test images against image-to-video baselines. Incantation uses blinded human ratings (three annotators, 400 clips) of whether an entity performed the prompted action. LongLive uses CLIP score per 10 s segment over 160 videos with six prompts each. WorldSonus compares a mid-stream sound prompt switch against a no-switch hold with identical visuals (audio only). Zing-0.5 measures only navigation (WBench) and shows text events as a qualitative session whose lasting consequences the authors say are not established. No source tests a text event in a released interactive model with persistence over later actions.
• articleHow does WorldMem store and retrieve earlier frames so a generated Minecraft or RealEstate10K world stays consistent after the camera leaves and returns, what revisit numbers does it report, and what does it cost?
  WorldMem keeps every generated frame as a token-level memory unit with a 5-number pose and a timestamp, picks up to a handful of units per step by field-of-view overlap and time difference (greedy, with a similarity filter), and lets the generating tokens cross-attend to them with pose/time embeddings. Its revisit tests compare regenerated frames against ground-truth frames with PSNR, LPIPS and reconstruction FID: 100 frames generated after a 600-frame memory bank on a custom Minecraft set, and five start=end trajectories over 100 RealEstate10K scenes. These are paper_result numbers on simulator poses; the paper's own tables disagree with each other and with the repository's expected-results block on some cells.
• articleWhat does WorldMemBench's fixed-reference multi-revisit protocol measure, what did it find about decay across returns, axis switches and off-screen events, and how reliable are its checks?
  WorldMemBench (260 cases: 200 static with human, object and environment targets, 60 dynamic with off-screen events) compares every prescribed return with the same first-visit reference over translation loops and yaw/pitch cycles, giving a score per revisit and counting a missing target as zero. On nine open video world models, the static leader (HY-WorldPlay) initiates only 31.7% of prescribed events, scores decay from the first to the third return on all eight metrics, and switching from a yaw cycle to a pitch cycle costs more. The paper adds pose-qualified subsets to separate memory from bad camera execution, and a context-policy study on LingBot World v2. Human agreement is 78.5-92.5% for static metrics but model-level correlation varies from 0.301 to 0.988.
• articleWhich of the explicit-memory world-model papers and revisit benchmarks can I actually download or run, and what do their licence texts, metadata and gates say per component?
  Observed on 2026-10-22: WorldMem has code (non-commercial licence text) and public ungated checkpoints with no licence field; SPMem has Apache-2.0 code and Apache-labelled checkpoints; VMem has MIT-labelled code that vendors a CC BY-NC-SA component and weights behind a Hugging Face access gate; Context-as-Memory has a released dataset but no code I could find; WorldTrace has no code I could find. The two newest benchmarks (R2M-Bench, WorldMemBench) release MIT code and data, but WorldMemBench's dataset card says its third-party images and the generated videos keep their original rights, and its examples set is labelled CC BY-NC by metadata. Nothing here was run, so 'released' means files are listed, not that they work.
• articleHow does VMem index past views with surfels, what is its cycle-trajectory revisit test, and which parts of its release are open, gated or conflicting?
  VMem keeps the past views as images but indexes them through a coarse surfel model: each surfel stores which views saw it, new target cameras render the surfels to vote for the top-K most relevant past views, and an image-set generator (Stable Virtual Camera, SEVA) generates the next views from those. Its main contribution to evaluation is a cycle-trajectory protocol (go out, return along the same path, score every 10 frames of the return) with LPIPS/PSNR/SSIM plus rotation and translation distance of camera poses re-estimated from the generated frames. The tables contain rows that are identical across methods, which should be checked before reuse. Code is MIT-labelled but vendors a CC BY-NC-SA component, and the weights sit behind a Hugging Face access gate.
• articleIf I run the open Oasis 500M code, what resolution, frame rate, context length and action vector does it actually use, and which of those does the project text state?
  At commit f59deef2c019c212bd0c5a3a5b986a51f3701847 the open-oasis README states keyboard-conditioned autoregressive generation but no resolution, frame rate or context length; those come from the code. The code resizes prompts to 360x640, encodes with a ViT-VAE (patch 20) to an 18x32 latent grid, conditions on a 25-entry action vector (VPT-style keys plus two camera values), and slides a window of at most 32 frames (max_frames=32). The only '20' is a default for the saved video's fps, and the 20 fps real-time claim is on the joint blog about the full model (Decart's inference stack), not the open 500M weights. The weights repo is gated and was not opened.
• articleHow does Physics-IQ test physical understanding in video models, which models and versions were scored, what were the results, and what do the authors say they do and do not show?
  Physics-IQ (Motamed et al., Google DeepMind) gives a model the first 3 seconds (or the last frame) of a real, static-camera physics event and asks it to continue for 5 seconds, then compares to the real continuation with four motion-based metrics, combined and normalised so two real takes of the same scene score 100. At v3 the best of eight configurations (VideoPoet, multiframe) scores 29.5 and Sora (image-to-video) 10.0; visual realism judged by Gemini 1.5 Pro was not significantly correlated with physics score (r = -0.46, p = .249). The authors call the metrics proxies, say they are on the conservative side (they penalise camera moves and shot cuts), and say results do not show that future models cannot learn physics.
• articleGiven a scene description, what check can an agent run so every line shows something that can be seen (1914 practice), and what in that check is only the lead's proposal, including any use for text-to-video prompts?
  Agent check assembled from Phillips (1914 practice, each step tagged with its source) plus lead-proposed steps (tagged PROPOSAL). Steps: strike 'registers <emotion>' lines; replace each with a restrained visible sign; add a short beat only where the feeling needs room; use caption/close-view/vision-scene for dreams (period devices); write sounds only when they bear on the action, as source plus reaction; give each line an action verb and only visible objects (outside Chapter V); finish with a reader test (editor/director/actor in 1914; a second reader or agent listing what they would see in the PROPOSAL). Use for text-to-video prompt writing is a hypothesis only; no source inspected documents it, and Phillips does not address AI.
• articleWhere is full vs limited range recorded in a file (MP4 nclx, MOV nclc, H.264/HEVC VUI, codec atoms), and what goes wrong when it is wrong?
  Range can live in several places. In FFmpeg movenc.c (pin 9047fa1b) MP4/AVIF colr nclx has a range bit, while MOV colr nclc has none; the H.264/HEVC bitstream filters set video_full_range_flag in VUI; DNxHD in MOV gets a separate ACLR atom with a full/legal range marker. Filters (setparams, scale) can mark or convert range. A practitioner says Rec.709 should never be full range and reports crushed/clipped results when it is; BT.709-6 as read defines 219/224 quantisation but the word "full range" does not occur in its text (searched, two spellings).
• articleWhat do players assume when an MP4 has no colour tags (including files from video-generation tools), and what should an agent do about it?
  Players assume different things for untagged video. Apple documented (2011) that untagged QuickTime media is treated as SMPTE-C; Chromium (commit 44e7da44) guesses BT.709 with limited range when nothing is set; a user report shows mpv showing bt.709 for a stream ffprobe called unknown. No inspected source says what any video-generation tool (e.g. Sora, Veo, Kling, Runway) writes: that part is a hypothesis and must be checked per file with ffprobe/mediainfo.
• articleWhy does the same Rec.709-tagged file look different in QuickTime/Safari, Chrome and VLC, and what are the numbers (1.96, 2.2/sRGB, 2.4)?
  Rec.709 (BT.709-6) specifies the camera-side encoding function; the display-side reference is BT.1886 (gamma 2.404), and BT.709-6 says production judges the look on that reference. Players then differ in which display curve they apply to a 1-1-1 file: Chromium (commit 44e7da44) renders BT709-tagged video with the sRGB curve in its generic path and carries a separate Apple curve with exponent 1.961 for macOS decode; Apple documents ColorSync playback lowering contrast for brighter surround. Practitioner posts quote 1.95, 1.96, 1.976 and 2.4 for the "right" figure and disagree on which is the specification.
• articleWhat do the colour-tag numbers (primaries, transfer, matrix, range; e.g. 1-1-1 for Rec.709) mean, and which of them does a file really need?
  A video colour tag is up to four independent facts: colour primaries, transfer characteristics, matrix coefficients (the three H.273 / ISO 23091-2 code points, which Apple writes as an nclc triple in that order, 1-1-1 for HD Rec.709) and a separate full/limited range flag. Code 2 means unspecified in each. The code values here are read from the FFmpeg header that says it matches H.273 and from Apple TN2227; the H.273 PDF itself was over the fetch cap, so the numbers are not compared to the standard text.
• articleDoes any video-generation work document explicit controls for over-the-shoulder shot/reverse-shot dialogue and the 180-degree rule, and what do its authors say still fails?
  ShoulderShot (arXiv 2508.07597v2) documents a pipeline for OTS dialogue: a finetuned Flux-dev generates one dual-shot image in which character A is always on the left and B on the right (its implementation of the 180-degree rule), each crop is animated with a finetuned Wan I2V model as an 8-second looping video, and the loops are lip-synced and assembled. Training pairs were mined by detecting an A-B-A shot alternation in dialogue footage. The authors state precise eye contact is not solved. ShotDirector (arXiv 2512.10286v1) claims editing-pattern-aware prompting and camera pose control; only its abstract was read.
• articleWhich classic craft concepts (key, breakdown, in-between, spacing, holds) do the AI in-betweening tasks actually correspond to, and is any production use documented?
  Editor's mapping (synthesis, not stated by any source): the papers' "given frames" are Lutz's extremes/keys; the generated frames are in-betweens; ToonComposer's optional middle sketch plays the role of a middle (breakdown-like) key; spacing is controlled only implicitly (sketch positions, FPS conditioning, text) and holds and exposure (on twos) are not modelled in either ToonCrafter or ToonComposer (no hit for hold, twos, spacing, timing, breakdown or exposure in either paper body). No production use is documented in the sources read: ToonComposer thanks two studios for permission to use footage in evaluation; ToonCrafter calls itself a research exploration. Production use is therefore a hypothesis here.
• articleWhat does ToonComposer (2025) claim for AI "post-keyframing" from sparse keyframe sketches, what was measured on real artist sketches, and what are its failure modes and licences?
  ToonComposer (arXiv v1, Aug 2025) merges in-betweening and colouring into one step on a Wan2.1 video model: a colour reference frame plus one or more keyframe sketches (any frame position) and optional region masks produce the clip. Evaluation: synthetic benchmark from cartoon films used with permission, PKBench (30 scenes, start/end sketches drawn by professional artists) and a 47-person preference study (70.99% aesthetic, 68.58% motion for ToonComposer). Stated problems: baselines distort faces and are sensitive to bold line styles; blank sketch regions render flat unless region masks are used; compute cost (README: 61 frames at 480p needs ~57 GB VRAM). Licence statement is MIT including weights.
• articleWhat does ToonCrafter claim and measure for generative cartoon in-betweening (inputs, frame counts, metrics, user study, failure modes) and what are the code/weights/data terms?
  ToonCrafter (arXiv v1, May 2024; SIGGRAPH Asia 2024 journal track per README) generates intermediate frames between two cartoon images with a fine-tuned image-to-video diffusion model, optionally guided by sparse sketches. Released model: 16 frames at 512x320 from a start and end image plus text. On its own 1K cartoon test set it reports FVD 43.92 vs 98.96 for the next best (SEINE) and a 24-person study with 68.98% preference for motion quality; it does not win LPIPS (0.1733 vs 0.1702 FILM). Stated failure modes: baselines fail on dis-occlusion and non-linear motion; unadapted priors leak live-action content; compressed latents cause flicker/detail loss; the maintainers state the success rate is not guaranteed.
• articleWhat do TikTok and Meta say creators must label as AI at upload, which labels do the platforms add themselves, and which parts are announced but unverified?
  TikTok (Support page via the IA capture of 2026-06-27; Newsroom post of 2025-11-19): creators are encouraged to label completely generated or significantly edited content and are required to label realistic AI content; routes include captions, hashtag stickers or the "AI-generated content" post setting. Misleadingly labelling unaltered content is a Terms violation. TikTok may also apply an automatic label (TikTok AI effects, uploaded Content Credentials) which the creator cannot remove, and says detection models back the rule. Invisible watermarks and a viewer AI-content control were announced in November 2025 and not observed. Meta: 2024 posts say Meta would require its disclosure tool for photorealistic video and realistic audio (penalties possible), and adds "AI info" labels when it detects industry-standard indicators or when people self-disclose; from September 2024 labels for content only edited by AI moved to the post menu. Meta's current Transparency Center page rendered no ...
• articleHow should an agent read FID/FVD tables on nuScenes for driving world models, given that one paper reports two sets of numbers for the same method and later papers copy different ones?
  The same method (DriveDreamer) has two published nuScenes results: FID 52.6 / FVD 452.0 in arXiv v1 and FID 14.9 / FVD 340.8 in v2, with the DriveGAN baseline also changing (73.4 / 502.3 to 27.8 / 390.8). Vista's Table 2 reuses the v1 pair; DriveDreamer-2's Table 1 uses the v2 pair. Vista says no prior model was publicly accessible, so cross-paper comparisons are of reported numbers, each under its own sampling, resolution and conditioning protocol. Downstream detection/tracking gains (mAP, NDS, AMOTA with generated training data) are an additional, task-based check used by DriveDreamer and DriveDreamer-2. A ranking claim from a table is only as good as the protocol match.
• articleHow do the driving and robot world-model sources measure multi-view, temporal, physics and control-following consistency, and which of those measurements are numbers versus figures or company claims?
  The sources use five families of checks: distribution distances (FID, FVD, FDD, FVMD), epipolar Sampson error within a view over time (TSE) and between views (CSE), pose re-estimation from the generated clip to check commanded paths (Cosmos TAE/TFE and camera-path errors, Vista IDM trajectory difference), detector/segmentation agreement with the conditioning boxes (GAIA-2), and human preference tests (Vista, Cosmos instruction tests). GAIA-2 provides no multi-camera metric in its text; Cosmos does (TSE/CSE, Table 24 and Predict2.5 Table 14) and notes the real-video reference values. Wayve's GAIA-3/4 pages describe LiDAR alignment, a three-level fidelity scheme and unpublished correlation claims; these are company statements without a linked report. Numbers from different reports are not comparable even for similar model names.
• articleWhich control inputs (ego trajectory, steering/speed, agent boxes, map, weather text, camera layout, robot actions, camera pose) do the driving and robot world models accept, and how is each encoded?
  The driving systems expose a small set of structured controls: ego motion (speed + curvature in GAIA-1/2; angle/speed, trajectory, command or goal point in Vista; yaw + velocity in DriveDreamer; 64-point trajectory in Cosmos), other agents (3D boxes in GAIA-2 and DriveDreamer; LLM-written trajectories in DriveDreamer-2), scene attributes (categorical metadata, CLIP or T5 text), and the camera rig (intrinsics/extrinsics in GAIA-2; per-view captions and view embeddings in Cosmos). Robot systems use gripper-frame action vectors (Cosmos on Bridge, 7-D) or whole-body low-level trajectories (1XWM). Camera-pose control with Plucker rays exists in Cosmos but was trained on static real-estate-style scenes. Most control interfaces were trained on proprietary data, and several are demonstrated only in figures.
• articleWhen driving and robot-learning papers say "world model", what exactly do they mean (inputs, outputs, intended use), and where does the word claim more than the evidence tests?
  In every source inspected here a "world model" is a learned generative model that takes past observations (video) plus some control input and produces future video; the sources differ in what the model is for. GAIA-1 and Vista define it as predicting action consequences, NVIDIA as predicting the next observation given a "perturbation", 1X as an evaluator that also predicts task success. Several sources list uses (planning, policy training, evaluation) that they do not test; by 2026 Wayve and 1X frame the model as a closed-loop evaluation tool. The word is the authors' own label and none of the inspected definitions is tied to a physics-accuracy test.
• articleHow far can I trust a MEt3R-style score that rests on a learned reconstruction backbone: what do controlled-corruption, injected-warp and human-ranking tests say, and what do COLMAP-based scores do better or worse?
  Three independent tests probe MEt3R from outside. SysCON3D (v1) finds that learned backbones (DUSt3R, MASt3R, VGGT, Fast3R) produce geometry for unrelated or noise inputs, MEt3R scores such sets as more consistent than real ones in its harness (negative effect sizes, 23% overall win rate), and its rank correlation with humans on eight sparse-view NVS methods is -0.10 to 0.26 against 0.98 for a COLMAP-based score. GeCo's injected-warp test finds MEt3R near chance at spotting a deformed frame. The SGC paper, in a different setup (text/video generators, 11 entities), reports MEt3R at Spearman 0.864 with humans. The sources agree that MEt3R is no stand-alone gate; they disagree on how well it ranks generators.
• articleWhat does WorldScore's DROID-SLAM reprojection-error '3D Consistency' score measure and miss, what do named closed and open video models score on it, and how much has it been validated?
  WorldScore's 3D Consistency runs DROID-SLAM on a generated static-world clip and averages the reprojection error of co-visible pixels between consecutive frames, then maps it to 0-100 against an empirical worst case (a frame-interpolated clip between unrelated images). Reported scores: Gen-3 68.31, Hailuo 67.18, DynamiCrafter 72.90, CogVideoX-I2V 86.21, 3D scene generators LucidDreamer 90.37 and WonderWorld 86.87. Its authors check it with human pairwise tests (91.7% and 97.3% agreement for 30-point bucket gaps); an independent injected-warp test found it good on scene-level but weak on object-level deformation. In code it assumes a fixed lens and cannot score dynamic subjects.
• articleHow is epipolar (Sampson) error computed on a generated clip, what does it say about geometry versus texture, where does it break for moving subjects, and what do papers report for named models?
  Epipolar error needs no learned reconstruction: match SIFT features between sampled frames, fit a fundamental matrix with the normalized 8-point algorithm in RANSAC, and average the Sampson distance of the matches to their epipolar lines; lower means the pair obeys one rigid camera motion. The Epipolar-Geometry paper reports 0.190 for a Wan2.1-1.3B baseline versus 0.131 after preference finetuning on Sampson-ranked pairs, with held-out-frame splat fidelity and human 'consistent' rates moving the same way (54.1% to 71.8%). It assumes a static scene (the authors say moving objects corrupt it), can be fooled by repetitive texture or a near-static clip, and GeoFlow shows a masked, short-stride variant for dynamic clips.
• articleWhat does a MEt3R score for a generated clip actually measure, how is it computed, what values do real and generated clips get, and what should I not read into it?
  MEt3R (arXiv v2) scores a pair of frames: DUSt3R reconstructs a pose-free pointmap pair, upscaled DINO features of both frames are projected into the first frame's camera, and the score is one minus the mean cosine similarity over the overlap (0 to 2, lower is better; no camera poses needed, so it works on generated video). In the paper, real videos sit slightly above zero, SVD scores 0.032, Ruyi-Mini-7B 0.047 and I2VGen-XL 0.050 on 48-frame RealEstate10K-conditioned clips, but those clips did not share a camera path and SVD moved least. It measures feature agreement after a learned reconstruction, not image quality and not (in this paper) agreement with human raters; every test in the paper is a static scene.
• articleCamI2V's README ranks 'MotionCtrl' and 'CameraCtrl' below CamI2V: are those the released MotionCtrl and CameraCtrl checkpoints, and what protocol do the numbers come from?
  The CamI2V README's comparison table is the CamI2V authors' own measurement (README-reported, not independently reproduced) at 256x256 and 50k steps, in which 'MotionCtrl' and 'CameraCtrl' are their re-implementations on DynamiCrafter, labelled 'reproduced non-official'. These are different artefacts from the official CameraCtrl (AnimateDiffV3, 256x384) and MotionCtrl (VideoCrafter/SVD) checkpoints, so the ranking does not rank the official releases. For the official CameraCtrl paper-style numbers on RealEstate10K with SVD and AnimateDiff builds, see the earlier batch candidate C3-B062-W13-03, which documents that RotErr/TransErr values from different papers are not comparable across estimators and normalisations. An agent choosing a checkpoint for camera accuracy should not use the README table as evidence about the official CameraCtrl or MotionCtrl releases.
• articleHow many frames and what resolution can each camera-pose-conditioned checkpoint produce in one pass at the pinned revisions, and what is documented about longer clips?
  Single-pass limits are short: CameraCtrl 16 frames at 256x384 (AnimateDiffV3 build); MotionCtrl 16 frames at 256x256 (VideoCrafter main branch; its UI text says 14, a documentation inconsistency within one file) and 14 frames on SVD; CamI2V 16 frames at 256x256 or 512x320; Stable Virtual Camera 21 frames per pass at 576x576 default, with the authors reporting 80 to 134 frame sequences via chunking; GEN3C 121 frames per pass (121*N-1 autoregressive), example at 704x1280 while its model card says 720*1080; ViewCrafter 25 frames at 576x1024 (or 16 frames, or 25 at 320x512); Cosmos-Predict2.5 agibot 93 frames. Timings and memory in READMEs are the authors' statements (A100 40G for ViewCrafter; ~43GB for GEN3C with offloading) and were not reproduced. A 5 to 10 second previs shot therefore needs chunking or autoregression for every system except possibly GEN3C and SVC, and only SVC and GEN3C document how to do so.

More