Video generation

Text/image/video-to-video, motion control, long-form consistency and model pipelines

• articleHow do I keep screen direction, eyelines and action continuous across separately generated shots, using the 180-degree rule, eyeline match and match-on-action?
  Problem. Each generated clip is independent, so left/right placement, gaze and mid-action poses drift between shots. Conditions. Multi-shot dialogue or action built from separate generations. Craft statements (sources). 180-degree rule: keep the camera on one side of an imaginary axis so the first character stays frame right of the second; crossing it reverses left/right and may disorient viewers; a moving character's path also sets a line (exit frame left, enter frame right). Eyeline match: set off-camera eyelines so cut-together singles look at the right person; matching close-ups use the same lens, matching camera height and distance, with the partner equidistant on the opposite side so A looks off right and B looks off left. Match on action: pick up a motion where the cut left it. Editing manual: reject a cut where the actor faces a different direction. Method (inference, my own mapping): (1) draw a floor plan and fix the axis before generating; (2) write the screen position and...
• articleShould I generate an AI plate on green or matte it afterwards, and which green-screen craft problems (spill, compression, soft edges) carry over?
  Problem. Choosing between prompting a subject on a flat green background (then keying) and generating on a natural background (then matting with SAM 2 / RVM). Conditions. Compositing generated plates into other footage. Craft statements (Wikibooks Chroma Key, 2014 page). Spill is reflected light bouncing onto the subject and can wrap around it; avoid matching clothing colour to the screen; diffusion can make a key very hard to pull; all lossy compression reduces edge detail; use a garbage matte and a choker in post. Method (inference, my own mapping to AI plates): (1) green-prompted plates inherit video-codec loss and model-invented green bounce on skin, so expect spill and soft edges the key cannot recover; (2) matting a natural-background plate with a learned alpha (RVM for people; SAM 2 mask plus refinement otherwise) avoids spill but depends on the matte model handling hair and motion blur; (3) if you do key, prompt for a flat, evenly lit screen, no green in costume, low contras...
• articleHow do I use RIFE and FILM to raise frame rate of an AI clip (exp, scale, times_to_interpolate), and what do they not fix?
  Problem. Generated clips are often short and low-fps; interpolation adds frames but does not repair identity changes. Conditions. Local RIFE (MIT) or FILM (Apache-2.0). Method. RIFE: python3 inference_video.py --exp=1 --video=video.mp4 (exp=1 gives 2X; exp=2 gives 4X, so exp = log2 of the factor), optional --fps; for very high resolution such as 4K set --scale=0.5; if disordered patterns appear try --scale=2.0 (this controls the processing resolution for the optical flow model). FILM: --times_to_interpolate N yields (2^N + 1)*(num_frames - 1) output frames and the tool targets near-duplicate frames with large motion; frames must sort naturally by name. Inference: for 24 to 48 fps use RIFE exp=1 or FILM times_to_interpolate=1 per frame pair; interpolation synthesises in-betweens from two neighbours, so it cannot remove a face change that already exists between them. Failure modes. RIFE flow errors on fast motion show as 'disordered pattern' (scale parameter); see the cycle's existing...
• articleWhich documented diffusers mechanisms reduce temporal flicker in AnimateDiff-style video (video-to-video strength, FreeInit, FreeNoise), and what do they cost?
  Problem. Frame-to-frame flicker and drift in diffusion video. Conditions. diffusers AnimateDiff pipelines (SD 1.5 class). Method. (1) Video-to-video: AnimateDiffVideoToVideoPipeline takes video, prompt, guidance_scale, num_inference_steps and strength; the doc examples use strength=0.5 and, with ControlNet conditioning frames, strength=0.8 with controlnet_conditioning_scale=0.75 and 10 steps at guidance_scale=2.0. Inference (not stated in the doc): lower strength keeps more of the source video's frames and therefore its motion and stability. (2) FreeInit: refines the initial noise iteratively to improve temporal consistency without training; enable with pipe.enable_free_init(method='butterworth', use_fast_sampling=True) and disable with pipe.disable_free_init(). It costs extra sampling passes (num_iters). (3) FreeNoise: noise rescheduling, temporal attention over sliding windows and weighted averaging of latent frames to generate longer videos from short-video models, with enable_fr...
• articleHow do MotionCtrl and CameraCtrl differ from text camera prompts: what do they condition on, what can be controlled independently, and which backbones are supported?
  Problem. Words like 'dolly in' give no exact trajectory. Pose-conditioned models take a numeric camera path. Conditions. Research checkpoints you run yourself (not hosted prompt UIs). Method. MotionCtrl: motion conditions are camera poses and object trajectories, treated as appearance-free; camera motion and object motion can be controlled independently or together; the repo script sets condtype to 'camera_motion', 'object_motion' or 'both'; released on VideoCrafter, SVD and AnimateDiff branches; extra test poses in dataset/camera_poses; object trajectories can be drawn with HandyTrajDrawer or extracted with ParticleSfM. CameraCtrl: a plug-and-play camera pose control module trained on top of a video diffusion model, leaving other base modules untouched; main branch is on AnimateDiff V3 with an svd branch; trajectories are per-frame camera poses (the README visualises each frame as a small tetrahedron for camera position and orientation); paper reports that training videos with dive...
• articleWhich camera moves does Google's Veo prompt guide expose in text, how do they map to dolly/truck/pedestal/zoom/crane grammar, and what does the guide say is unsupported?
  Problem. Text-only camera control needs words the model was tuned on, and the craft meaning of each word matters. Conditions. Veo/Gemini video models via the Google Cloud prompt guide (no seed, parameters or pose inputs on this page). Method. The guide lists these moves with definitions: static, pan (rotate horizontally from a fixed position), tilt (rotate vertically), dolly in/out (camera physically moves toward/away), truck left/right (camera physically moves sideways), pedestal up/down (vertical move, level perspective), zoom (lens focal length changes, camera does not move), crane (vertical or sweeping arcs), handheld, whip pan, arc shot, plus lens effects: wide-angle, telephoto, rack focus, fisheye and vertigo effect (dolly zoom). Craft mapping (inference): pan/tilt change view direction only, so parallax between foreground and background should stay fixed; dolly/truck/pedestal/crane translate the camera, so parallax and occlusion should change; zoom changes magnification witho...
• articleFor speech and video datasets, how often does the dataset licence disagree with the terms of the underlying sources, and where can I see the per-dataset records?
  Corrected 2026-10-23 - see C3-B643-05; the original statements below are kept for history and are superseded where the correction says so. The Data Provenance Initiative's 2025 audit (arXiv 2412.17847v2) reports, for 3916 datasets, that 25%, 33% and 32% of text, speech and video datasets carry non-commercial licences but that most of their source content carries non-commercial restrictions through source terms (author-reported); its released AMI record lists the dataset licence as CC BY 4.0 while the GitHub licence inferred from a converter repository (AMICorpusXML) is MIT, and Papers with Code reports Unspecified.
• articleWhich VBench-2.0 dimensions can I run on my own clips (for example previs shots), what do they need, and which only work on the standard prompt suite?
  At the pinned repo commit, only five of the 18 dimensions accept custom videos: Human Anatomy, Human Identity, Human Clothes, Diversity (needs 20 clips per prompt) and Multi-View Consistency. The other 13 (including Camera Motion, Instance Preservation, Motion Rationality and all Controllability and Physics state-change dimensions) assert-fail on custom input and need the standard prompt suite, which means sampling the model under test on fixed prompts (3,860 videos for a full run by my count). Requirements are heavy (CUDA 11.8, flash-attention build, several third-party weights, some from Google Drive/Dropbox).
• articleWhat does VBench-2.0 actually measure (five categories, 18 dimensions), how is each dimension scored, and how is a single Total Score built?
  VBench-2.0 (arXiv v2) scores generated video on 18 dimensions grouped into Human Fidelity (3), Creativity (2), Controllability (7), Physics (4) and Commonsense (2). Scoring mixes generalist judges (LLaVA-Video-7B captions or yes/no questions, Qwen2.5-7B-Instruct as text judge) with specialist detectors (ViT human-anomaly detectors, ArcFace, CoTracker heuristics, a Qwen2.5-VL-3B LoRA anomaly detector). The paper reports per-dimension scores for four systems and no aggregate; the equal-weight Total Score is defined only in the repository README and used by the leaderboard code.
• articleWhat control signal does GameNGen condition on, and what resolution, context and speed does the paper state?
  GameNGen's ICLR version (arXiv v2) says the action set is 'key presses' of DOOM and that each action is embedded into a single token; the paper text never lists the individual actions (no hit for mouse, keyboard or discrete in the whole PDF text), so the exact action set is not stated. Stated numbers: 320x240 frames padded to 320x256, context of 64 frames plus 64 actions (a little over 3 seconds), 20 fps with four DDIM steps on a TPU-v5 (50 fps with distillation), trained on RL-agent play where each agent action is held for 4 frames. Table 2 shows PSNR rising from 20.94 at one history frame to 22.36 at 64.
• articleWhat do the authors of video-to-audio models say the models get wrong, so I know what not to ask of them?
  Documented failure modes cluster: speech and lip sync (MMAudio 'unintelligible mumbles'; Google V2A 'uncanny lip-syncing'; Movie Gen does not support voice), dense, small or occluded motion (Movie Gen: tap dance, footsteps), unfamiliar sound concepts (MMAudio: 'RPG firing'), unrequested music (MMAudio), dependence on clean input video (Google V2A), and long-form or precise-timestamp reasoning (ThinkSound's MLLM). These come from the model authors' own statements; none was reproduced.
• articleI have a silent or AI-generated video clip and want a sound track: what kinds of video-to-audio systems exist, what do they take in and put out, and which of them can I actually run?
  Primary sources split into measured open systems (MMAudio, HunyuanVideo-Foley, ThinkSound: video plus optional text in, audio out, weights on Hugging Face), a measured closed system (Meta Movie Gen Audio, 13B, 48 kHz, up to 30 s per pass plus extension, described as research-only), and vendor showcases (Google V2A blog, Veo 3 joint audio) that give no sample rate, duration or sync metric. Sample rate and clip length differ by system; only the open ones can be inspected or run, and their licences differ (see G033 candidates).
• articleWhat acceptance check could an agent run before relying on a streaming video or world model for a long take (proposal, not a tested protocol)?
  No source gives a ready acceptance test. Combining measures the papers use gives a proposal: generate at the longest length you need, score per-segment brightness/imaging quality and a motion measure, compute first-vs-last delta, run a copy-first-clip control, and separately test whether the model returns to an earlier place (memory), which the papers say these methods do not provide. This is synthesis, untested.
• articleAI music video pipeline (Suno + Seedance audio-sync): feed vocal-only stems or the model lip-syncs the instruments; fix drift with a dedicated lip-sync pass and edit to the track
  Creator-reported: statements attributed to the creator(s) come from discovery's reading of auto-caption transcript excerpts; the curator checked the video metadata (title, date, length) and any cited documentation pages, not the spoken claims. Nothing was tested; promotional or sponsored framing applies where stated. R2 (Apr 2026): lyrics drafted with an LLM, song made in Suno v5.5 with a structured prompt (concept, voices, chorus/outro, BPM, language) and "exclude styles" used as a negative prompt. Characters: character sheets, then many photos per character (33, 41 and 57) to train a custom character model on the host platform. Lip-sync shots framed as close wide-angle (8 mm-style) singing-to-camera. Key failure: when the full mix is uploaded, Seedance lip-syncs instruments too, and it can alter or invent lyrics in the audio track; fix by exporting Suno stems (Pro tier) and uploading the vocal stem only, then running a separate lip-sync pass with the same vocal stem. Dance shots c...
• articleSeedance multi-shot practice from a heavy user: timeline prompts as shot lists, explicit rules for failure points, at most ~3 characters, draft at 480/720p, and edit across several generations
  Creator-reported: statements attributed to the creator(s) come from discovery's reading of auto-caption transcript excerpts; the curator checked the video metadata (title, date, length) and any cited documentation pages, not the spoken claims. Nothing was tested; promotional or sponsored framing applies where stated. Dankieft (Daan) on Seedance 2.0/2.5 via Higgsfield. Timeline prompting (R1): a scene header (subject, wardrobe, environment, mood, music, colour logic, style) followed by time-stamped shots works as a shot list - one 15 s generation can return ~10 shots, but over-detailed beats get skipped or rushed, so cut detail per beat; he drafts the prompt with an LLM using a reusable instruction file, then edits it ("Don't trust AI blindly"). When a detail fails twice, add an explicit rule (e.g. which card is chosen and who returns it) plus a negative prompt; add a "logic"/one-continuous-take instruction or the model invents camera cuts. Glitches rise above about three characters ...
• articleSame creator, two video-model shootouts six weeks apart, opposite verdicts on Gemini Omni Flash: test conditions (resolution cap, start-frame use, prompt specificity) decide rankings more than the models
  Creator-reported: statements attributed to the creator(s) come from discovery's reading of auto-caption transcript excerpts; the curator checked the video metadata (title, date, length) and any cited documentation pages, not the spoken claims. Nothing was tested; promotional or sponsored framing applies where stated. Two Higgsfield-hosted comparisons by Youri van Hofwegen. Aug 2026 (R2): five models at each model's MAX settings (Seedance 2.0 native 4K 15 s, Sora 2 Pro Max 12 s 1080p, Kling 3.0 4K, Happy Horse 1080p, Gemini) on a talking-head lip-sync test and a breakdance motion test, scored 1-10. Verdict: Seedance top on both; "Gemini is the real surprise" top-two while cheapest (30 credits vs Kling 90, Happy Horse 68, Sora 153, Seedance 330 - Seedance partly because of native 4K); Kling 3 lip sync "pretty bad" (5/10); Sora bottom. Sep 2026 (R1): four models BLIND-labelled A-D, all capped at 10 s, 16:9, 720p with audio "otherwise this becomes a test of which model has the biggest r...
• articleHow a top AI-tools channel ranks video generators (Nov 2025-Jul 2026): same-prompt rounds, audio as a scoring factor, and "cost per second of usable video" rather than credits per generation
  Creator-reported: statements attributed to the creator(s) come from discovery's reading of auto-caption transcript excerpts; the curator checked the video metadata (title, date, length) and any cited documentation pages, not the spoken claims. Nothing was tested; promotional or sponsored framing applies where stated. Youri van Hofwegen's ranking format (condensed reads; videos use OpenArt or Higgsfield as the test host and link to them). Method: identical prompt across models per round (universal realism scene, multi-character fight, dialogue), audio quality counts in the score (poor audio drops a model even if visuals are good), and a final VALUE round. His key value rule: compare how much usable video each generation yields, not credits per generation - a cheaper call that returns a shorter clip can cost more per second (R1); in Jan 2026 he contrasted a top realism model costing ~3,000 OpenArt credits with a near-equal alternative at ~400 credits per 10 s (R3); in Nov 2025 he rate...
• article"Face swap any video" tutorials and permanent digital personas: what the workflow enables and the consent/disclosure lines it crosses
  Creator-reported: statements attributed to the creator(s) come from discovery's reading of auto-caption transcript excerpts; the curator checked the video metadata (title, date, length) and any cited documentation pages, not the spoken claims. Nothing was tested; promotional or sponsored framing applies where stated. Youri van Hofwegen (2026-02-19, R4) demonstrates replacing his own face in recorded video with a generated person (Nano Banana Pro for the identity frame, then a video tool with lip sync), auto-captioning, and then making that synthetic person "perform" trending dances and scripted ads so a "permanent digital influencer" can work "for free". Assessment: swapping one's OWN performance onto a fictional face is legal in many places but still realistic synthetic media that platforms require to be disclosed (TikTok C3-B398-01; YouTube altered/synthetic disclosure), and EU AI Act Art. 50 may require visible deep-fake disclosure for professional/monetised use (C3-B426-01). Swa...
• articleRunning synthetic "AI influencer" accounts (Feb-Jun 2026 creator practice): consistency methods, a local dashboard, and the disclosure obligations creators tend to skip
  Creator-reported: statements attributed to the creator(s) come from discovery's reading of auto-caption transcript excerpts; the curator checked the video metadata (title, date, length) and any cited documentation pages, not the spoken claims. Nothing was tested; promotional or sponsored framing applies where stated. Dan Kieft says he runs about five AI-influencer pages, some with millions of impressions a month, and built a free local dashboard that stores nine personas (each in a niche, e.g. a yoga instructor) and one-click generates character sheets, close-ups and "feature" sheets, then scenes for posting (R3). Consistency methods (R2): detailed prompts for skin texture and imperfections to avoid a plastic look; generate a background grid with the character as reference and upscale a chosen panel; edit with "keep the exact same person, face, skin texture, expression, pose, background, proportions and camera framing" and change only clothes or pose; pose transfer by giving two ref...
• articleA blind creator test (Jul 2026): Seedance 2.0 4K won most rounds, Grok Imagine best dialogue, Kling 3.0 "80% of the quality at a quarter of the cost"
  Creator-reported: statements attributed to the creator(s) come from discovery's reading of auto-caption transcript excerpts; the curator checked the video metadata (title, date, length) and any cited documentation pages, not the spoken claims. Nothing was tested; promotional or sponsored framing applies where stated. Dan Kieft (Higgsfield-sponsored, 2026-07-05). A teammate generated clips so he ranked them blind: Seedance 2.0 at 4K high bitrate, Kling 3.0 at 4K high bitrate, Grok Imagine at 720p, Google Omni at native 720p (upscalable). Tests: image-reference POV dialogue, text-to-video, motion control (Grok lacks it), editing (Omni expected to win; he ranked Seedance first, Omni second, Kling third), dialogue/lip sync (Grok first, Seedance second, Kling third), multi-reference (Seedance first). Overall Seedance took the most golds, but he notes it is among the most expensive; Kling 3.0 offers 4K and 60 fps cheaply - "80% of the quality with 1/4 of the cost" - and some shots were ea...
• articleDirecting AI camera moves with a Blender previs clip: an agent builds the 3D blockout and camera, then the render is fed to Seedance as a "camera movement reference only"
  Creator-reported: statements attributed to the creator(s) come from discovery's reading of auto-caption transcript excerpts; the curator checked the video metadata (title, date, length) and any cited documentation pages, not the spoken claims. Nothing was tested; promotional or sponsored framing applies where stated. Dan Kieft (2026-08-05). Problem: complex camera choreography is hard to describe, and re-prompting never reproduces a lucky shot. Method: (1) make a cinematic start frame (any image model); (2) give the frame and a written camera plan (e.g. 15 s: push in on one person, move to another, pass through a window to reveal a food stand) to Claude connected to Blender via MCP - Claude builds a rough scene and animated camera (took about 5+ minutes on his PC); preview, then Render > Render Animation; (3) in Seedance 2.0/2.5 attach the start frame plus the Blender render and begin the prompt with "video 1 as a camera movement reference only", then describe action/dialogue. His A...
• articleHow do I apply Murch's priority order to choosing among AI-generated takes and cut points, and which failures can I accept?
  Problem. Generated shots rarely satisfy every continuity criterion; reviewers need a rule for what to give up. Conditions. Multiple candidate takes or cut points for the same beat; labelled inference throughout, built on Murch's list (C3-B597-01). Method [inference]. (1) Emotion: pick the take whose performance or atmosphere is right for the moment, even if later criteria are flawed; reject takes that are clean but emotionally flat. (2) Story: the shot must show the story information (who, what changed); (3) Rhythm: choose the in and out point where the cut feels right, which is a free choice with generated footage because you can trim a clip anywhere inside it. (4) Eye-trace: check that the viewer's eye lands where the next shot expects (face, hand, object) in the first frames after the cut; generated clips often start with the subject not yet in position. (5) Planarity: screen direction and eyeline across the cut (see C3-B571-12 for the 180-degree rule, eyeline match and match-on-...
• articleWhat do open AI video tools (Wan2.2 save_video, diffusers export_to_video) actually write to the file: bit depth, encoder, colour tags, resolution?
  Problem. A generated clip looks different after export (banding, size change, shifted levels) and nobody knows what the tool did. Conditions. Python tools that write MP4 through imageio/ffmpeg. Method (read from code). Wan2.2 wan/utils/utils.py save_video: tensor clamped to value_range (-1,1), made into a grid per frame, multiplied by 255 and cast with .type(torch.uint8), written with imageio.get_writer(codec='libx264', quality=8); fps default 30 in the signature. diffusers export_utils.export_to_video: float numpy frames become (frame*255).astype(np.uint8); imageio.get_writer with quality default 5 (variable bit rate; 10 highest), optional bitrate, and macro_block_size default 16, where imageio tells ffmpeg to scale the image up to the next size divisible by that number; set None or 1 to disable, with the warning that many players cannot decode odd sizes. Observations: (a) 8-bit output, so banding risk (C3-B586-01/02); (b) float to uint8 is a cast, not documented as rounded or dith...
• articleWhich input modes (first frame, first and last frame, reference images/video/audio, performance video) does each model accept through Runway's API, and which combinations are rejected?
  Problem. Continuity workflows (keyframes, references, motion transfer) depend on which inputs a model accepts; mixing modes silently fails or errors. Conditions. Runway Dev API reference fetched 2026-10-23; per-model request bodies are discriminated unions keyed on model, so fields valid for one model are invalid for another. Method (documented fields). Gen-4.5 image_to_video: one first-frame image, ratio list including 1280:720, 720:1280 and 1584:672, duration 2-10 s, outputFormat options (C3-B586-04). Veo 3.1 (via Runway): first frame, or first plus last; last-frame-only is not supported; duration 4, 6 or 8; ratios 1280:720, 720:1280, 1080:1920, 1920:1080; audio boolean default true. Kling 3.0 Pro: first frame, optionally last; when both are given the output aspect ratio follows the first frame; multiPrompt up to 6 segments with total 3-15 s; cfgScale 0-1 default 0.5; audio default true. Seedance 2.5, Wan 3.0 and Hailuo 3: promptImage uses position first/last for keyframe mode or ...
• articleHow can I measure and match colour and brightness between separately generated shots using FFmpeg (signalstats, colorbalance/colorcorrect/colortemperature, normalize, colormap)?
  Problem. Generated shots of the 'same' scene differ in exposure, white balance and saturation, and there is no colour chart in frame. Conditions. Two or more SDR clips decoded to the same colour space and range; a reference frame you accept as the look. Method. (1) Measure each clip. signalstats exports per-frame metadata keys such as lavfi.signalstats.YMIN and reports for Y, U, V: min, 10th-percentile (LOW), average, 90th-percentile (HIGH), max; SATMIN..SATMAX; HUEMED and HUEAVG; and YDIF/UDIF/VDIF (difference to the previous frame). Use YLOW/YAVG/YHIGH for black point, mid and white point, and SATAVG and HUEAVG for colour (values are in the file's coded range 0-255 as the doc states). The doc describes the filter as aimed at problems of digitised analog video, so use its numbers as relative comparisons only. (2) Correct. colortemperature (1000 to 40000 K, default 6500, mix, pl preserve lightness); colorcorrect (red/blue shift for shadows rl/bl and highlights rh/bh, saturation -3 t...
• articleWhich benchmark checks whether a video or world-generation model follows a camera path and keeps a scene consistent, and does it measure physical laws?
  Partially verified: the curator re-read none of the cited sources; S3, S10, S11 rest on the producer's pinned reading. WorldScore (Stanford; arXiv v2 Nov 2025) evaluates 'next scene' generation: given a scene image, a next-scene prompt and a camera layout (a trajectory plus one of 8 text camera moves), it scores controllability (camera, object, content), quality (3D, style, photometric consistency, subjective quality) and dynamics (motion accuracy, magnitude, smoothness). It tested 20 models, including 13 video models; video models had weak camera control (best video 40.22 on camera controllability) and 3D scene generators beat them on static worlds. It does not define a physical-law metric, so it informs camera and consistency questions only.
• articleThe diffusers memory guide says the Wan VAE does not support tiling or slicing, but the class has enable_tiling: how can a newcomer tell which statement is out of date without running anything?
  Partially verified: the curator re-read none of the cited sources; S1, S2, S6, S4, S12, S13, S14, S16, S17 rest on the producer's pinned reading. A newcomer can compare dates: in diffusers, the two Wan warnings in the memory guide were added on 2025-05-01, and the Wan VAE tiling/slicing code was merged on 2025-05-05, so the code is newer than the warnings; the guide's tiling sentence has not changed since. The class defines enable_tiling itself and gets enable_slicing from a shared mixin, and slicing only helps when decoding more than one video at once. Treat the warning as likely stale but unconfirmed until a short run on your own hardware, because several merged bug-fix PRs show Wan2.2 tiling had errors that were later fixed and two related PRs are still open.
• articleMy GPU cannot hold a video pipeline in diffusers: which memory-saving calls exist, what does each one cost, and what do the Wan, LTX-Video, HunyuanVideo and CogVideoX pages tell me to try first?
  Partially verified: the curator re-read none of the cited sources; S1, S14, S2, S8, S9, S10, S11, S25, S27, S26, S28, S34, S35 rest on the producer's pinned reading. At diffusers v0.41.0 (tag commit 086bf9578c0f4acbc66e48cd1e7cc26befd9e10f) the docs and docstrings describe three offloading levels: enable_model_cpu_offload (whole components move; smaller saving, much faster), enable_sequential_cpu_offload (submodule level; largest saving, 'extremely slow') and group offloading (block_level or leaf_level; stated to sit between the two). Each of the four video pipeline pages gives one example with a stated approximate VRAM figure (Wan ~13GB, LTX-Video ~10GB, HunyuanVideo ~14GB, CogVideoX-5B ~16GB), but each example combines different options, no sentence names a GPU or a peak-memory metric, and only the LTX example sets its own resolution (Wan and HunyuanVideo use pipeline defaults for size), so the four numbers are not comparable and are not a feasibility test for a given machine. No ...
• articleIf I only have a text prompt, what do MAV3D and 4D-fy actually show, how were they evaluated, which one can I run, and how good is the motion?
  Corrected 2026-10-23 - see C3-B588-07; the original statements below are kept for history and are superseded where the correction says so. Partially verified: the curator re-read none of the cited sources; S1, S3, S4, S19, S8, S16, S9, S5 rest on the producer's pinned reading. Two papers cover text-to-4D with score distillation. MAV3D (Meta, January 2023) optimises a six-plane dynamic NeRF against a text-to-video model in three stages (about 6.5 h on 8 A100) and shows an R-Precision of 83.7 against adapted internal baselines of 6.2-10.6; no code is available. 4D-fy (arXiv v2, May 2024) blends 3D-aware, image and video diffusion priors, takes about 23 h on one 80 GB A100, and ships a repository with run instructions. In 4D-fy's own user study (26 evaluators, 28 prompts) users prefer 4D-fy for appearance (92%) and structure (89%) but only 41% prefer it for motion, and the motion difference is the one comparison that is not statistically significant. 4D-fy says motion is limited to sim...
• articleWhat representation does each of Marble, HunyuanWorld 1.0, WonderWorld and WonderJourney actually output (splats, meshes, panoramas, depth layers, point clouds, video), and is each a paper result, a demo or a released capability?
  Corrected 2026-10-23 - see C3-B588-01 and C3-B598-01; the original statements below are kept for history and are superseded where the correction says so. The four systems output different things. Marble (World Labs) is a hosted product whose docs list Gaussian splats (SPZ/PLY), triangle meshes (GLB), a 360 panorama and rendered video. HunyuanWorld 1.0 builds layered meshes from a generated panorama plus depth; its paper also describes a 3DGS and an HDRI-sky option, but the released demo code at the pinned commit writes only per-layer vertex-coloured PLY meshes (optional Draco). WonderWorld produces Fast Gaussian Surfels (saved as .ply and .splat by its code); WonderJourney produces a sequence of coloured point clouds that its code renders to video. RTFM, also from World Labs, deliberately has no explicit 3D representation. Labels differ by component, see systems[].
• articleWhy do AI-generated clips show banding in skies and fog, and when should I use FFmpeg gradfun versus deband versus a 10-bit master?
  Problem. Smooth gradients (sky, fog, walls, vignettes) in generated clips show contour bands, usually worse after delivery encoding. Conditions. 8-bit delivery. BT.709 8-bit coding assigns black 16 and nominal peak 235 (219 steps); 10-bit assigns 64 and 940 (876 steps), so 10-bit has four times the steps over the same range (arithmetic from the table). Open tools quantise float frames to uint8 before encoding (see P-L1-4-03). Method. (1) Decide where the bands enter: at the 8-bit cast/export, or at lossy compression. (2) For a playback-only copy, FFmpeg gradfun=strength[:radius]: strength range .51 to 64 (default 1.2) is both the maximum change per pixel and the flat-region threshold; radius 8 to 32 (default 16); a larger radius gives smoother gradients but protects pixels near detail. The FFmpeg doc says it is designed for playback only and must not be used before lossy compression because compression loses the dither and brings the bands back. (3) deband filter: replaces banded pi...
• articleAccording to ZEISS's lens-division paper, what actually controls depth of field (f-number, imaging scale, format), and how should that change how I prompt for 'shallow depth of field' and lens length?
  Problem. Prompts mix 'shallow depth of field', 'f/1.4' and '85mm' as if they were interchangeable. Conditions. Any photoreal image/video prompt that names lens or aperture. Source statements (H. H. Nasse, Carl Zeiss Camera Lens Division, 2010). Depth of field depends on the acceptable blur and so rests on arbitrary specifications; it does not stop sharply but changes continuously. In object space it is (almost) independent of focal length when compared at the same imaging scale (subject size on the sensor), meaning different focal lengths shot from correspondingly different distances give similar depth of field; this stops holding at very large distances. Differently, perspective and the maximum blur of the distant background differ: background blur is lower for shorter focal lengths than for longer ones. Describing a lens by its 35 mm-equivalent focal length is inconsistent unless the aperture is converted too; equivalent f-numbers scale with format size, so small formats have litt...
• articleWhat licences and access terms apply to Marble, HunyuanWorld 1.0, WonderWorld and WonderJourney (code, weights, outputs), and what hardware does running each require?
  Corrected 2026-10-23 - see C3-B588-03; the original statements below are kept for history and are superseded where the correction says so. Four different regimes. Marble is a hosted service under World Labs Terms of Service (last updated 2026-01-21): Free-account output is retained by World Labs with a personal non-commercial licence to the user; paid-account output is owned by the user including commercial purposes, with a training licence over user content that paid users may opt out of. HunyuanWorld 1.0 is under a Tencent community licence that is expressly limited to a Territory excluding the EU, UK and South Korea, includes a 1M monthly-active-user trigger and a bar on using outputs to improve other AI models. WonderJourney code is MIT. WonderWorld has no licence file at the pinned commit. Nothing here is legal clearance.
• articleWhat scores do named systems (Sora 2, Veo 3.1, Gen-3, Hailuo, Cosmos, SVD, LTX-Video and others) get under each geometric-consistency metric, and which of those numbers can legitimately be compared?
  Corrected 2026-10-23 - see C3-B588-06; the original statements below are kept for history and are superseded where the correction says so. GeCo-Eval (80 static-scene prompts, 4 videos each, with motion columns) is the only inspected benchmark that scores Sora 2 and Veo 3.1; it puts Sora 2 and Veo 3.1 close together (object-centric fused 0.069 vs 0.090, indoor 0.169 vs 0.233, outdoor 0.265 vs 0.217; lower is better) with Veo moving more. WorldScore gives Gen-3 68.31 and Hailuo 67.18 on its 0-100 reprojection axis; SGC scores Cosmos 0.0722 and Latte 0.3226; the MEt3R paper has SVD at 0.032. Metrics, scales, prompts, clip lengths and motion differ, so the numbers form a table of separate experiments, not a ranking; the only visible overlap is model family names.
• articleWhich primary pages support the Vista (32 GB, Apache-2.0) and Cosmos-Predict2.5 multiview (8x80 GB) hardware and licence statements previously cited only to the Apache licence text?
  Correction to C3-B144-W29-05 (nature: citation does not support the claims; facts hold at other primary pages). Original summary cites only the generic Apache-2.0 text, which cannot support product-specific claims on hardware, weights or availability. Audit result: Vista's SAMPLING.md suggests GPUs with a minimum of 32 GB VRAM for sampling (with a --low_vram option for GPUs under 80 GB); Cosmos-Predict2.5's multiview inference guide requires at least 8 GPUs with 80 GB each; Vista's README states its repository content is under Apache-2.0 and the Hugging Face model record tags apache-2.0. The 'not found' claims for GAIA, DriveDreamer weights and 1XWM, and the Vista-versus-SVD licence tension, were not re-verified. Replace the single Apache URL with these pages when citing. Version/date: fetched 2026-10-23.
• articleWhere does Mochi 1's Apache-2.0 licence actually appear? Correction: the cited README lines do not state it; the LICENSE file (Apache-2.0 text) and the Hugging Face card (license: apache-2.0) do.
  Problem. The earlier finding says Mochi's Apache 2.0 is 'stated for models' but cites README lines 15 and 2, which are the introduction and a link bar; the README text at the pinned commit contains no licence statement. Method. Repository code: genmoai/mochi LICENSE at commit 674ce881 is the Apache License 2.0 text. Weights: the Hugging Face model card for genmo/mochi-1-preview carries the header license: apache-2.0 (read from the main branch on 2026-10-23, revision not recorded). Impact. The conclusion (Apache-2.0 for code and the published weights) stands; only the citation was wrong. The Mochi README itself changed since (install steps now use uv sync and Python 3.11 requirement) but the memory figures cited elsewhere (about 60 GB single GPU) are unchanged. Limitations. Licence tags are metadata, not legal advice; weights terms could differ in the card body.
• articleHow long does fal retry a failed webhook? Correction: about 1 hour after the request completes, but only about 6 minutes for results of 10 KB or more, up to 31 retries.
  Problem. The earlier finding gave the retry window as 'about 1 hour after the request (as stated on the page)'. The page as fetched on 2026-10-23 is more specific. Method. Retries use increasing backoff until the stored result expires: about 1 hour after the request completes, or about 6 minutes for results of 10 KB or more, with a maximum of 31 retries. First delivery has a 15-second timeout; retried deliveries have 120 seconds. Redirects (3xx) are a permanent failure. Large results (over 1 MB) and requests with payload storage disabled can only be fetched until the stored result expires. Failure modes. A handler that is down for 10 minutes can lose a 10 KB+ result's webhook deliveries entirely; design handlers to be idempotent per request_id and poll the queue as backup within the retention window. Limitations. Docs page, undated; the page changed or was more precisely read since the earlier note (cannot tell which).
• articleWhat licence applies to the StereoCrafter code and weights, and can I use them commercially?
  Correction (changed or newly resolved). C3-B581-06 (round 3) said the StereoCrafter licence file was not found on the repository main branch, so reuse rights were unverified. The Hugging Face model repository for TencentARC/StereoCrafter carries a LICENSE and a NOTICE. The LICENSE states the project is fine-tuned with the assistance of Stable Video Diffusion, which is under a Non-Commercial Community License, and adds that users agree to use StereoCrafter only for academic, research and education purposes, refraining from any commercial or production purposes. The NOTICE lists the Stable Video Diffusion model as a third-party component under its own licence. So StereoCrafter is non-commercial; do not use it to produce client or production stereo conversions. Limitations. The GitHub main branch still returned 404 for LICENSE, LICENSE.md and LICENSE.txt, so the licence text was read from the Hugging Face repository. Version/date. Fetched 2026-10-23; HF model last modified 2024-12-27.
• articleWhat do the cited Diffusion4D and 4D-fy papers report about generation time, and where do the MAV3D 6.5-hour and Diffusion4D 8-minute figures come from?
  Correction (unsupported by the cited source). C3-B168-W33-03 and W33-04 give Diffusion4D an '8 minutes' text-to-4D time and MAV3D about 6.5 hours on 8 A100. In the cited arXiv pages (Diffusion4D 2405.16645v1, MAV3D 2301.11280v1) the text extracted says only that Diffusion4D generates 4D assets 'within just several minutes' and contains no A100 or hours figure for MAV3D. The 4D-fy time is supported: the three optimisation stages take roughly 2, 2 and 19 hours (23 hours total) on an A100 with about 80 GB of VRAM. Treat '8 minutes' and '6.5 h on 8 A100' as unverified (tables or later versions may carry them). Version/date. Fetched 2026-10-23.
• articleWhich cited benchmark papers contain the SGC scores for Cosmos and Latte, and what do the cited papers actually report for Sora 2 and Veo 3.1?
  Correction (unsupported by the cited sources). C3-B139-W28-07 reports 'SGC scores Cosmos 0.0722 and Latte 0.3226'. Searching the four cited papers (MEt3R 2501.06336, WorldScore 2504.00983, 2511.18991, GeCo 2512.22274) for 0.0722 and 0.3226 found neither value, and none of them evaluates Cosmos or Latte under an SGC score in the text extracted. The other numbers were reproduced: GeCo-Eval evaluates Sora 2 and Veo 3.1 (object-centric fused 0.069 and 0.090), the MEt3R paper lists 0.032, and WorldScore's 3D Consist column gives 68.31 for Gen-3 and 67.18 for Hailuo. Remove or re-source the SGC figures. Version/date. Fetched 2026-10-23.
• articleConversational creative agents inside tools (Artlist agent, Flova skills, Lovart brand kits, Combos.fun game agent): what they automate and where humans still steer
  Creator-reported: statements attributed to the creator(s) come from discovery's reading of auto-caption transcript excerpts; the curator checked the video metadata (title, date, length) and any cited documentation pages, not the spoken claims. Nothing was tested; promotional or sponsored framing applies where stated. Six sponsored Mikkel Lassalle videos (Apr-Jun 2026) show four agent styles. Artlist AI agent: describe a goal in plain language, it picks a model (e.g. suggests 4K-capable models when asked), generates, and proposes next-step edits (moonlight, deeper shadow contrast, anamorphic flare) that you accept or override; same chat then animates a 5 s clip. Flova: "skills" (script-driven, story-driven, reference video, remix, music video, product commercial) wrap a pipeline; his manual path was Nano Banana Pro frame with "use the reference as the absolute identity source" -> GPT Image 2 character sheet -> Seedance 2.0 with both references, 15 s. Lovart: one prompt yields a brand...
• articleThe invented-brand product spot: packshot with rendered label text, then a 15 s multi-cut commercial (Apr-Jun 2026 creator recipe)
  Creator-reported: statements attributed to the creator(s) come from discovery's reading of auto-caption transcript excerpts; the curator checked the video metadata (title, date, length) and any cited documentation pages, not the spoken claims. Nothing was tested; promotional or sponsored framing applies where stated. Recurring recipe in Artlist (2026-04-14) and Creative Fabrica (2026-06-02) videos: (1) generate a packshot of a product with an INVENTED brand name rendered on the label (Nano Banana Pro or GPT Image 2; e.g. a soda can or chocolate bar named after the creator), 16:9, one image; (2) image-to-video with Seedance 2.0: "Fast multiple cuts and cinematic camera movement. Premium beverage commercial. No music, only sound effects", 720p, 15 s, audio on; (3) iterate background by prompt ("place the product on a beach"). He reports good product and label consistency. Version-claim drift to note: in April he said Seedance 2.0 combines "up to 12 references"; later videos cite 50 re...
• articleAgentic assistants driving Blender, DaVinci Resolve and After Effects for AI film pre-production and editing (GPT-6 Astra, Sept 2026)
  Creator-reported: statements attributed to the creator(s) come from discovery's reading of auto-caption transcript excerpts; the curator checked the video metadata (title, date, length) and any cited documentation pages, not the spoken claims. Nothing was tested; promotional or sponsored framing applies where stated. Dan Kieft (Higgsfield-sponsored, 2026-09-23) on OpenAI's GPT-6 Astra (launched early Sept 2026 per secondary news listings R4/R5, emphasising computer use; primary OpenAI page not read). Uses: (1) ChatGPT + Blender MCP builds a 3D set (a bar) from a prompt and references; he places cameras for shot angles, renders previs frames, has GPT Image 2.5 make them photoreal, then uses them as Seedance 2.5 references - more control than describing shots in text; Astra also built a web viewer of the set so non-Blender teammates can orbit/walk it. (2) Recreating rooms from photos to plan a studio layout. (3) Thumbnail ideation with own portrait, channel examples and inspiration, a...
• articleSmall production tips from editor-integrated workflows: "no music, only sound effects", layered timelines, and AI dialogue re-language with lip sync
  Creator-reported: statements attributed to the creator(s) come from discovery's reading of auto-caption transcript excerpts; the curator checked the video metadata (title, date, length) and any cited documentation pages, not the spoken claims. Nothing was tested; promotional or sponsored framing applies where stated. From CapCut PC (2026-07-04) and Lumina (2026-08-18) videos: (1) end video prompts with "No music, only sound effects" so generated SFX are usable while you keep control of the score in the edit; (2) layer in a multi-track editor - generated clips on the main track, generated stills as overlays, music/sound design below, captions/motion graphics on top; simple keyframed fades between generated clips; (3) prompt vocabulary he uses for animated frames: ambient shadows, cinematic close-up portrait, shallow depth of field, emotional storytelling, photoreal CGI, highly detailed animated film frame; (4) Seedance 2.5 video editing on Lumina can change a clip's dialogue language...
• articleWhat free AI video tiers actually give you (Jan 2026 creator test) and why a "backed by data" title needs checking
  Creator-reported: statements attributed to the creator(s) come from discovery's reading of auto-caption transcript excerpts; the curator checked the video metadata (title, date, length) and any cited documentation pages, not the spoken claims. Nothing was tested; promotional or sponsored framing applies where stated. Youri van Hofwegen (2026-01-24) tests three no-cost options: a no-login generator with separate action and camera prompt fields (camera field noticeably improves motion; free tier limited to its basic model), Grok Imagine (continuous variations as you scroll, edit-by-instruction, presets incl. an adult "spicy" mode, text-to-video), and Meta AI create (variety/weirdness/stylization sliders - stylization helps avoid the glossy 3D look - restyle, animate, lip sync). Shared free-tier limits he observed: queue waits of 5-10 min for a 5 s clip, weak prompt adherence needing rerolls, older/lower-quality models with morphing faces; he attributes this to compute cost. Packaging ...
• articleBuilding a fictional brand campaign end to end in an editor (CapCut PC, Sep-Oct 2026): logo, products, model/product sheets, 30 s generation, scene split and chat-based edits
  Creator-reported: statements attributed to the creator(s) come from discovery's reading of auto-caption transcript excerpts; the curator checked the video metadata (title, date, length) and any cited documentation pages, not the spoken claims. Nothing was tested; promotional or sponsored framing applies where stated. Mikkel Lassalle CapCut videos (2026-09-22, 2026-10-15) plus an Artlist Seedance 2.5 video (2026-08-11). Pipeline: invent the brand and make a logo with GPT Image 2 (his preference for logos and character sheets); develop products and style frames in a design workspace with Nano Banana Pro (his preference for cinematic frames); create one character sheet for the model and one for the product and tag both in the prompt; add camera/style cues (e.g. "ARRI Alexa 35 aesthetic"); generate up to 30 s with Seedance 2.5 (still capped at 720p in August per him; start/end frames also available); import into the editor, use "split scenes" to auto-cut the generation into shots, reord...
• articleCreator head-to-head (Aug 2026): Seedance 2.5 vs Seedance 2.0 vs MiniMax H3 vs Flux 3 video on camera moves, acting, dialogue, music and multi-reference
  Creator-reported: statements attributed to the creator(s) come from discovery's reading of auto-caption transcript excerpts; the curator checked the video metadata (title, date, length) and any cited documentation pages, not the spoken claims. Nothing was tested; promotional or sponsored framing applies where stated. Dan Kieft (OpenArt link, 2026-08-21), each prompt run up to 3 times, best shown, 15 s clips. Specs as he states them on OpenArt then: Seedance 2.0 up to 15 s, 480p-4K (3,000 credits 1080p/15 s, 6,000 4K); Seedance 2.5 up to 30 s, 720p only at the time, up to 50 references (30 images, 10 videos, 10 audio, 30 s total), 1,955 credits/15 s; MiniMax H3 up to 15 s, 2K, 15 references, 1,350 credits/15 s, runnable locally; Flux 3 up to 20 s, 1080p, ~3,000 credits/15 s. His verdicts: complex camera moves (crash zoom, whip pan, earth zoom-out) - Seedance 2.5 best, Flux 3 worst; simple cinematic shot - Seedance 2.0 4K looked best; complex two-angle dialogue shot - Seedance 2.5 bes...
• articleA four-stage control ladder for AI video: one-line prompt, LLM-expanded multi-shot prompt, edited storyboard grid, then saved character/location assets
  Creator-reported: statements attributed to the creator(s) come from discovery's reading of auto-caption transcript excerpts; the curator checked the video metadata (title, date, length) and any cited documentation pages, not the spoken claims. Nothing was tested; promotional or sponsored framing applies where stated. Youri van Hofwegen (Higgsfield link, 2026-08-11) holds model (Seedance 2.0), 1080p, 16:9 and the idea constant and changes only workflow: (1) one plain sentence - fast for testing, but camera, sound and look are the model's choices; (2) an LLM (Claude) writes a detailed three-shot prompt - deliberate cuts and camera, though a requested dialogue line was dropped; (3) a three-panel storyboard image from GPT Image 2, fixing a wrong detail (paved vs dirt road) by editing only the affected panels, then giving the storyboard plus an LLM prompt written from it - output followed the board shot-for-shot; (4) saved assets: a character sheet made from a COLLAGE of real photos (mul...
• articleDecart: priced, API-accessible real-time models - Oasis 3 Preview (text-prompted driving world model) and Lucy 2.5 live video editing at about $0.02 per second
  Decart's API platform (docs fetched 2026-10-23) is one of few places where a world model is a paid, documented API rather than a demo. Oasis 3 Preview (released June 2026, R3): set a scene with a text prompt, then send chunks of exactly four [throttle, steering] actions in [-1, 1] and receive four generated frames for each of three camera streams (left_forward, front, right_forward) over a stateful gRPC session; a new prompt resets the world context; Python SDK decart-oasis; a browser playable preview exists; the docs pitch it as a learned driving simulator for RL, with a PPO Colab (R1). It is driving-specific (vehicle controls, three car-mounted views), not a general walkable world. Pricing (R2): pay-as-you-go, no subscription; Oasis 3 Preview $0.02 per second (a 60 s session = $1.20); Lucy 2.5 realtime video editing (add/replace/remove objects, swap characters, change backgrounds live) $0.02/s at 720p or $0.04/s in fast mode, $0.04/s for batch video; Lucy Restyle 2 $0.01/s; image ...
• articleOpen interactive world models you can run or train: Matrix-Game 3.5 (720p, long-horizon memory, first/third person, 40 GB GPU) and ForgeWM (fully open recipe, real-time keyboard/mouse control, trained on 8 GPUs)
  Two open releases extend the open world-model landscape (C3-B399-01). Matrix-Game 3.5 (Riemann Dynamics; HF weights apache-2.0, July 2026; repo Sep 2026) (R1, R2): memory-augmented interactive world model for 720p long-horizon camera-controllable video in first-person and third-person modes. "Patch Memory" lifts past observations into a 3D memory with geometric alignment and visibility-aware retrieval so revisited places stay consistent; static scene and dynamic subjects are separated, with third-person mode taking 0-4 protagonist reference crops for identity. Built on Wan2.2-TI2V-5B with Depth-Anything-3; 5B base (bidirectional) checkpoints plus a three-step distilled first-person causal checkpoint for real-time streaming. Needs one NVIDIA GPU with >= 40 GB VRAM (704x1280 peaks around 40 GB), Linux, >= 64 GB RAM. ForgeWM (arXiv Aug 2026) (R3, R4): an end-to-end open recipe (bidirectional SFT -> teacher-forced causal AR -> consistency distillation -> on-policy DMD) producing 1/2/4-s...

More