3D and worlds

Modeling, texturing, geometry, reconstruction, materials, scenes and rendering

• articleHow should an agent read FID/FVD tables on nuScenes for driving world models, given that one paper reports two sets of numbers for the same method and later papers copy different ones?
  The same method (DriveDreamer) has two published nuScenes results: FID 52.6 / FVD 452.0 in arXiv v1 and FID 14.9 / FVD 340.8 in v2, with the DriveGAN baseline also changing (73.4 / 502.3 to 27.8 / 390.8). Vista's Table 2 reuses the v1 pair; DriveDreamer-2's Table 1 uses the v2 pair. Vista says no prior model was publicly accessible, so cross-paper comparisons are of reported numbers, each under its own sampling, resolution and conditioning protocol. Downstream detection/tracking gains (mAP, NDS, AMOTA with generated training data) are an additional, task-based check used by DriveDreamer and DriveDreamer-2. A ranking claim from a table is only as good as the protocol match.
• articleHow do the driving and robot world-model sources measure multi-view, temporal, physics and control-following consistency, and which of those measurements are numbers versus figures or company claims?
  The sources use five families of checks: distribution distances (FID, FVD, FDD, FVMD), epipolar Sampson error within a view over time (TSE) and between views (CSE), pose re-estimation from the generated clip to check commanded paths (Cosmos TAE/TFE and camera-path errors, Vista IDM trajectory difference), detector/segmentation agreement with the conditioning boxes (GAIA-2), and human preference tests (Vista, Cosmos instruction tests). GAIA-2 provides no multi-camera metric in its text; Cosmos does (TSE/CSE, Table 24 and Predict2.5 Table 14) and notes the real-video reference values. Wayve's GAIA-3/4 pages describe LiDAR alignment, a three-level fidelity scheme and unpublished correlation claims; these are company statements without a linked report. Numbers from different reports are not comparable even for similar model names.
• articleWhich control inputs (ego trajectory, steering/speed, agent boxes, map, weather text, camera layout, robot actions, camera pose) do the driving and robot world models accept, and how is each encoded?
  The driving systems expose a small set of structured controls: ego motion (speed + curvature in GAIA-1/2; angle/speed, trajectory, command or goal point in Vista; yaw + velocity in DriveDreamer; 64-point trajectory in Cosmos), other agents (3D boxes in GAIA-2 and DriveDreamer; LLM-written trajectories in DriveDreamer-2), scene attributes (categorical metadata, CLIP or T5 text), and the camera rig (intrinsics/extrinsics in GAIA-2; per-view captions and view embeddings in Cosmos). Robot systems use gripper-frame action vectors (Cosmos on Bridge, 7-D) or whole-body low-level trajectories (1XWM). Camera-pose control with Plucker rays exists in Cosmos but was trained on static real-estate-style scenes. Most control interfaces were trained on proprietary data, and several are demonstrated only in figures.
• articleWhen driving and robot-learning papers say "world model", what exactly do they mean (inputs, outputs, intended use), and where does the word claim more than the evidence tests?
  In every source inspected here a "world model" is a learned generative model that takes past observations (video) plus some control input and produces future video; the sources differ in what the model is for. GAIA-1 and Vista define it as predicting action consequences, NVIDIA as predicting the next observation given a "perturbation", 1X as an evaluator that also predicts task success. Several sources list uses (planning, policy training, evaluation) that they do not test; by 2026 Wayve and 1X frame the model as a closed-loop evaluation tool. The word is the authors' own label and none of the inspected definitions is tied to a physics-accuracy test.
• articleHow far can I trust a MEt3R-style score that rests on a learned reconstruction backbone: what do controlled-corruption, injected-warp and human-ranking tests say, and what do COLMAP-based scores do better or worse?
  Three independent tests probe MEt3R from outside. SysCON3D (v1) finds that learned backbones (DUSt3R, MASt3R, VGGT, Fast3R) produce geometry for unrelated or noise inputs, MEt3R scores such sets as more consistent than real ones in its harness (negative effect sizes, 23% overall win rate), and its rank correlation with humans on eight sparse-view NVS methods is -0.10 to 0.26 against 0.98 for a COLMAP-based score. GeCo's injected-warp test finds MEt3R near chance at spotting a deformed frame. The SGC paper, in a different setup (text/video generators, 11 entities), reports MEt3R at Spearman 0.864 with humans. The sources agree that MEt3R is no stand-alone gate; they disagree on how well it ranks generators.
• articleWhat does WorldScore's DROID-SLAM reprojection-error '3D Consistency' score measure and miss, what do named closed and open video models score on it, and how much has it been validated?
  WorldScore's 3D Consistency runs DROID-SLAM on a generated static-world clip and averages the reprojection error of co-visible pixels between consecutive frames, then maps it to 0-100 against an empirical worst case (a frame-interpolated clip between unrelated images). Reported scores: Gen-3 68.31, Hailuo 67.18, DynamiCrafter 72.90, CogVideoX-I2V 86.21, 3D scene generators LucidDreamer 90.37 and WonderWorld 86.87. Its authors check it with human pairwise tests (91.7% and 97.3% agreement for 30-point bucket gaps); an independent injected-warp test found it good on scene-level but weak on object-level deformation. In code it assumes a fixed lens and cannot score dynamic subjects.
• articleHow is epipolar (Sampson) error computed on a generated clip, what does it say about geometry versus texture, where does it break for moving subjects, and what do papers report for named models?
  Epipolar error needs no learned reconstruction: match SIFT features between sampled frames, fit a fundamental matrix with the normalized 8-point algorithm in RANSAC, and average the Sampson distance of the matches to their epipolar lines; lower means the pair obeys one rigid camera motion. The Epipolar-Geometry paper reports 0.190 for a Wan2.1-1.3B baseline versus 0.131 after preference finetuning on Sampson-ranked pairs, with held-out-frame splat fidelity and human 'consistent' rates moving the same way (54.1% to 71.8%). It assumes a static scene (the authors say moving objects corrupt it), can be fooled by repetitive texture or a near-static clip, and GeoFlow shows a masked, short-stride variant for dynamic clips.
• articleWhat does a MEt3R score for a generated clip actually measure, how is it computed, what values do real and generated clips get, and what should I not read into it?
  MEt3R (arXiv v2) scores a pair of frames: DUSt3R reconstructs a pose-free pointmap pair, upscaled DINO features of both frames are projected into the first frame's camera, and the score is one minus the mean cosine similarity over the overlap (0 to 2, lower is better; no camera poses needed, so it works on generated video). In the paper, real videos sit slightly above zero, SVD scores 0.032, Ruyi-Mini-7B 0.047 and I2VGen-XL 0.050 on 48-frame RealEstate10K-conditioned clips, but those clips did not share a camera path and SVD moved least. It measures feature agreement after a learned reconstruction, not image quality and not (in this paper) agreement with human raters; every test in the paper is a static scene.
• articleIf I track inventory, location, character knowledge and consequences across many turns of an interactive story, which failure modes have these two papers measured and which are only proposals?
  Documented use (measured in the papers, on non-story environments): predicted-state quality drops with horizon when predictions are fed back (S2), a model that predicts from raw history can claim an object is unavailable or return the previous room (S2 Appendix G examples, selected), a learned simulator can let weak agents "win" it could not win for real (S1 Table 5), and one world model lost room connectivity in a looping TextWorld run (S1 Appendix H, one trajectory). Not measured: character knowledge or belief differences between characters, NPC dialogue, long narrative sessions (the longest horizon is 20 actions in Jericho; trajectories average 28.6-52 steps), or any per-category error rate for inventory. Everything about running such a tracker for authored fiction is a hypothesis.
• articleWhat exactly did arXiv 2610.06637 (v1) define and measure for long-horizon textual world modelling: environments, horizons, models, metrics, headline numbers, ablations, limits, release?
  arXiv 2610.06637 (Wang, Gao, Yao, Dong, Walia, Das; UIC and Intuit AI Research; v1 submitted 2026-10-05 16:25:44 UTC; CC BY 4.0) fine-tunes Qwen3-4B-Instruct-2507 world models that keep a JSON "textual state" and predict a whole k-action sequence in one call as sparse set/remove edits that code replays (latent-SR). It evaluates horizons 1-10 (ScienceWorld, CEO-Bench) and 1-20 (Jericho) with a fixed-target protocol against raw-history recursive/direct baselines and GPT-5.2. latent-SR has the best averages on every metric (ScienceWorld Fact-F1 92.44 vs 88.94 for obs-direct; CEO-Bench SE-score 87.35 vs 81.85; Jericho BLEU 24.5 and Score MAE 1.728), with gains growing at long horizons, but obs-recursive is better on Jericho at h1 and h5. The abstract's "nearly 30%" Jericho error drop holds against obs-direct at h10 (28.9%) but not against obs-recursive (12.2%), which is the lower-error raw-history baseline there. No code, data or weights release is stated.
• articleHow many frames and what resolution can each camera-pose-conditioned checkpoint produce in one pass at the pinned revisions, and what is documented about longer clips?
  Single-pass limits are short: CameraCtrl 16 frames at 256x384 (AnimateDiffV3 build); MotionCtrl 16 frames at 256x256 (VideoCrafter main branch; its UI text says 14, a documentation inconsistency within one file) and 14 frames on SVD; CamI2V 16 frames at 256x256 or 512x320; Stable Virtual Camera 21 frames per pass at 576x576 default, with the authors reporting 80 to 134 frame sequences via chunking; GEN3C 121 frames per pass (121*N-1 autoregressive), example at 704x1280 while its model card says 720*1080; ViewCrafter 25 frames at 576x1024 (or 16 frames, or 25 at 320x512); Cosmos-Predict2.5 agibot 93 frames. Timings and memory in READMEs are the authors' statements (A100 40G for ViewCrafter; ~43GB for GEN3C with offloading) and were not reproduced. A 5 to 10 second previs shot therefore needs chunking or autoregression for every system except possibly GEN3C and SVC, and only SVC and GEN3C document how to do so.
• articleFor each camera-pose-conditioned checkpoint, what exactly is the pose convention (world-to-camera vs camera-to-world, OpenCV vs OpenGL, relative to first frame, intrinsics, units or scale)?
  Conventions differ and must be matched per system. CameraCtrl, MotionCtrl and CamI2V take world-to-camera 3x4 matrices (RealEstate10K style, OpenCV camera axes) and all re-express them relative to the first (conditioning) frame, so absolute world axes and origin drop out but the camera axis convention and handedness do not. Stable Virtual Camera takes camera-to-world matrices in the OpenGL convention and internally flips two columns to OpenCV. GEN3C works in world-to-camera with OpenCV intrinsics and scene-relative distances (scaled by an estimated centre depth). ViewCrafter is parametrised by spherical offsets about a point at the depth of the image centre and needs a hand-estimated elevation. No inspected source gives a metric unit: MotionCtrl uses a unitless 'speed', CamI2V a trace_scale_factor, SVC a camera_scale (with a documented single-view scale ambiguity), GEN3C and ViewCrafter a depth-relative scale. Cosmos-Predict2.5's loader inverts file extrinsics before use, so even wi...
• articleWhat is inside a RealEstate10K-style camera pose file, and how do CameraCtrl and CamI2V turn it into the per-pixel Plucker embedding the model sees?
  A RealEstate10K-style pose file is plain text: one row per frame holding a timestamp (or frame path), four normalised intrinsics (fx, fy, cx, cy), two unused zeros and a 3x4 world-to-camera matrix. CameraCtrl and CamI2V both read columns 1-4 and 7 onward, invert the matrix to camera-to-world, shoot one ray per pixel using a pinhole model with x right, y down, z forward, and encode each ray as a 6-vector (moment, direction). That 6-channel image per frame is the 'camera input' the network sees. CamI2V's single-image demo path replaces the file intrinsics with a default field of view (fx = 0.5 * max(H, W)), so intrinsics in a hand-written pose file may be ignored there. This is code reading at pins; no file was parsed by running code, except line/column counts computed with standard-library scripts (synthesis).
• articleWhich open-weight camera-controllable video or novel-view generators exist, and for each one does it take an arbitrary per-frame camera path, or only presets?
  At the pinned revisions, all seven inspected systems take a camera trajectory as data; five accept free per-frame poses: CameraCtrl (16-pose RealEstate10K-style txt), MotionCtrl (one 12-float RT row per frame), CamI2V (RealEstate10K-style rows), Stable Virtual Camera (transforms.json, OpenGL convention, any number of target cameras) and GEN3C (per-frame world-to-camera matrices via NPZ or its GUI; the single-image CLI has presets only). ViewCrafter is different: it takes three short offset sequences (d_phi, d_theta, d_r, 2 to 25 values) and interpolates, so it is not a matrix-per-frame interface. Cosmos-Predict2.5 robot/multiview-agibot reads per-frame extrinsics and intrinsics for three fixed robot cameras, which is rig conditioning rather than a free camera path (the intrinsics files were not opened). Cosmos-Predict1 at the pinned commit has no file whose path names camera, gen3c, pose or trajectory (path scan only); GEN3C, built on it per its model card, is the camera-conditioned...
• articleAcross later driving video/world-model papers with nuScenes generation tables, which FID/FVD values for DriveDreamer and DriveGAN does each table print, and which arXiv version of each paper was read?
  Of seven later tables that print DriveDreamer FID/FVD, five use the arXiv v1 pair 52.6/452.0 (DrivingDiffusion v1, Drive-WM v1, Panacea v1 with columns swapped, GenAD v1 and v2 with FVD rounded to 452, UniMLVG v1-v3), one prints the v2 pair 14.9/340.8 alongside another pair (DriveDreamer-2 v1 and v2), and two (DriveDreamer-2 v1/v2 and Delphi v1/v3) print 26.8/353.2, a pair found in neither DriveDreamer v1 nor v2 HTML text. DriveGAN is printed only in Drive-WM, GenAD and UniMLVG, always at the v1 pair 73.4/502.3 (GenAD 502); none of the tables read uses the v2 pair 27.8/390.8. MagicDrive-V2/MagicDriveDiT and DiVE print no DriveDreamer FID/FVD (MagicDrive-V2 lists only DriveDreamer's resolution and frame count). DrivingDiffusion v1 (2023-10-11) predates DriveDreamer v2 (2023-11-27); Panacea v1 (11-28) and Drive-WM v1 (11-29) were posted one and two days after v2 yet print the v1 pair; the papers do not say whether v2 was seen.
• articleBefore I compare nuScenes FID/FVD scores of two driving world-model papers, what exactly should I check, and which concrete mismatches have already happened with DriveDreamer baselines?
  Six checks, each with a documented case in the papers read: (1) pin the arXiv version of the citing and the cited paper, because DriveDreamer v1 and v2 print different FID/FVD for the same method; (2) read the row condition, not only the method name, because DriveDreamer-2 quotes three different DriveDreamer pairs in one table; (3) check the metric protocol (resolution, clips, frames, FVD feature extractor), which differs across papers; (4) check whether a baseline row is quoted or recomputed (Delphi marks recomputed rows with a star and gets MagicDrive FID 46.18 against the quoted 16.20); (5) check column order and rounding (Panacea lists FVD before FID; 16.96 appears as 16.9); (6) check whether the citing paper's own numbers changed between versions (UniMLVG 8.8/60.1 in v1-v2, 5.8/36.1 in v3). This is a reading checklist for reported numbers; no new scoring was done.
• articleWhich dimensions of VBench-2.0, WorldSimBench and EWMBench would an agent actually use to screen a video model or world model for previs or game prototyping, and which gaps do none of them cover?
  Synthesis: for previs clips, the usable pieces are VBench-2.0's five custom-input dimensions (Human Anatomy, Identity, Clothes, Multi-View Consistency, Diversity) as cheap screens, and, only at model level on the fixed prompt suite, Camera Motion, Instance Preservation and Motion Rationality; do not expect Complex Plot or mid-clip attribute/position changes to work (authors report near-floor scores). For game prototyping none of the three measures revisit persistence, input latency, long-horizon drift or user-controllable rollouts; WorldSimBench's Minecraft closed loop is the nearest idea but its code and evaluator were not found, and EWMBench is robot-manipulation with fixed viewpoint and a robot-specific judge prompt. This is a proposed screening checklist, not a tested protocol.
• articleHow does EWMBench score models, which systems lead and by how much, and what bias, scale or protocol risks do the paper and repo leave open?
  EWMBench reports a domain-adapted lead (EnerVerse_FT 4.70, LTX_FT 4.55) over Kling (3.87), Hailuo (3.41) and open models (2.97-3.29) on seven models. The 'Avg.' and 'Overall' columns are plain sums (I verified to 4 decimals), so the 'overall' mixes components of very different range: the semantic Diversity term spans 0.06 while Dyn spans 0.57. Semantic scores barely separate EnerVerse_FT (2.09), Kling (2.04) and Hailuo (2.02). The human study ranks four models with no annotator count or correlation statistic. Motion best-of-three is selected using the ground-truth trajectory, and the judge prompt is robot-specific (declares human hands abnormal). Weights and data are CC BY-NC-SA 4.0 per cards and README, but the repo has no LICENSE file.
• articleWhat is EWMBench, what exactly does it measure (scene, motion, semantics), what inputs does it need, and how do I run it?
  EWMBench (AgiBot, arXiv v2) scores image-text-to-video models on robot manipulation: given an initial scene image, an instruction and optionally a 6D trajectory, the model generates frames that are compared with ground-truth AgiBot-World episodes. It measures scene consistency (fine-tuned DINOv2 patch similarity), end-effector motion (Hausdorff, normalised DTW and a velocity/acceleration Wasserstein term from a fine-tuned YOLO-World detector), and semantics (a Qwen2.5-VL caption compared by BLEU and CLIP score, a diversity term, and a logic check). It is robot-specific, fixed-viewpoint and needs a ground-truth trajectory per case.
• articleHow are WorldSimBench scores produced and validated, which systems lead and by how much, and which of the paper's own claims do its tables fail to support?
  WorldSimBench validates its learned evaluator against GPT-4o with PLCC (0.60 AD, 0.43 RM in-distribution; zero-shot 0.34-0.49 AD, 0.44-0.47 RM) and accuracy for OE, but gives no evaluation-set sizes, annotator counts or agreement. The explicit and implicit modes disagree: by my computation OE rank agreement across seven models is Spearman 0.04 (0.21 on the five text-conditioned models). The authors admit inflated scores in robot manipulation and Velocity, and several statements in the text do not match the paper's own tables (GPT-4o negative-correlation attribution, 'AD variation is minimal', Open-Sora-Plan's trajectory lead). All results are for eight older open models fine-tuned on scenario data, so 'who leads' says nothing about current commercial systems.
• articleWhat is WorldSimBench: how does it define a 'world simulator', what are its two evaluation modes and three scenarios, and can I get its code, data or evaluator?
  WorldSimBench (arXiv v1, Oct 2024) sorts predictive models into stages S0-S3 by output modality and calls S3 'actionable video' models World Simulators. It evaluates them two ways: Explicit Perceptual Evaluation (a LoRA-tuned video LLM scores generated clips on scenario-specific dimensions, trained on a 35,701-video human-feedback set) and Implicit Manipulative Evaluation (generated video is converted to actions by video-to-action models and run closed-loop in Minecraft/MineRL, CARLA and CALVIN). It evaluated eight fine-tuned open video generators; no code, dataset or evaluator release could be located (homepage buttons have no links; GitHub and Hugging Face name searches found nothing).
• articleHow good is the evidence that VBench-2.0 scores track human judgment, which systems does it say lead and by how much, and what judge-bias, saturation or protocol risks do the authors and repo admit or leave?
  The human-alignment evidence is a model-level comparison over only four systems per dimension. I recomputed the table's 'Correlation' row as Pearson r over those four points (matches to within rounding), while the figure caption says Spearman; with n=4 rank agreement is far weaker on some dimensions (0.4 for Human Anatomy and Human Interaction, average ranks, no significance test). Several dimensions have tiny score spreads between systems, and the authors admit VLM judges are unreliable for Human Anatomy and Motion Rationality. Between arXiv v1 and v2 the Instance Preservation method and values changed for the same four systems (about 71-76% to 82-94%). Leader statements: per Table II no system leads everywhere; there is no paper aggregate, and leaderboard Total Scores were not retrievable.
• articleWhat do the open-weight licences of LTX-2.x, MiniMax H3, Apple SHARP, Meta SAM and Fish Audio S2 actually allow for commercial use and where?
  Licence texts read 2026-10-23 (not legal advice): LTX-2.x Community License (2026-08-11) grants broad rights subject to use restrictions (Attachment A), but entities with annual revenues of at least $10,000,000 must obtain a paid commercial licence for any non-non-commercial use; unlicensed commercial use by such entities is prohibited and fees may be charged retroactively. MiniMax H3 Community License (2026-08-02) is territorially limited: the "Applicable Territory" is worldwide EXCLUDING the European Union, the United Kingdom, the Republic of Korea and the United States, so the open-weights grant does not cover users there (MiniMax invites licence requests); also prior written authorization if commercial products/services exceed US$20M yearly revenue, and "MiniMax H3" must be displayed in commercial UIs. Apple SHARP model licence: Research Purposes only (non-commercial scientific research), derivatives likewise; excludes product development or commercial services. Meta SAM License...
• articleWho owns 3D models generated with Meshy and Hyper3D Rodin, what do they cost, and what are the retention/training terms?
  Meshy Terms (last updated 2026-09-19): on the free plan Meshy owns all rights in outputs and makes them available under CC BY 4.0 with credit to Meshy; on paid plans customers own their outputs (to the extent law allows) and may keep content private; outputs may carry AI-identifying metadata/watermarks that users must not remove; non-Enterprise inputs/outputs may be used to train or improve services; API outputs (non-Enterprise) are deleted after 3 days, so download promptly. Meshy API pricing is in prepaid credits per request (credit-to-USD rate not on that page): e.g. Image/Text-to-3D preview 20 credits on meshy-7.1/meshy-6 (5 on meshy-6-lite or the smart-topology t2/t2.5 models), mesh with 2K-4K textures 30, 8K 35; refine/retexture 10; auto-rigging 5; animation 3 per action; remesh 5; smart-topology polycount 100-15,000 (t2) or 100-30,000 (t2.5). Hyper3D (Rodin) pricing page (rendered 2026-10-23): Free $0 plan pays per confirmed result at $1.5 per direct credit, with export from ...
• articleDo interactive world models make sound today, and which audio-capable ones are only a paper, a demo, or something I can actually download?
  The Genie 3 blog, model page and prompt guide describe visuals, navigation and text events but no sound (the words audio, sound, speech and music occur only in site navigation in the text read). Veo 3 and 3.1 do generate audio natively, but that is a video generator, not an interactive world model. Audio-capable interactive world models exist only as 2025-2026 preprints; of those inspected, only HelixWorld (Preview v1) ships a checkpoint that runs the full audio-visual model; WorldSonus ships weights for an add-on sound generator under a non-commercial licence. All performance numbers are the authors' own.
• articleWhat exactly does R2M-Bench compute for revisit memory (pair mining, MemoryGain, NMR), how well does it track humans and resist the slow-motion shortcut, what did it find, and what can be reused?
  R2M-Bench defines revisit memory as behaviour: after a commanded return the output should recover the earlier scene more than comparable non-revisit moments in the same video. For each revisit pair it samples a gap-matched non-revisit pair and a short-range pair from the same rollout, reports MemoryGain (revisit minus baseline) and a Normalized Memory Ratio (gain divided by short-range minus baseline) over five families (appearance, scene identity, object identity, local geometry, persistent state). On 100 scenes x 3 trajectories, DreamX-World-Memo leads (Overall NMR 0.706); raw similarity tracks humans better (0.594) than Overall NMR (0.547) but is coupled to low motion. The benchmark team and the top model share an organisation. Code and 100 reference images are MIT-labelled; you generate the videos.
• articleHow can an agent test whether a generated world 'remembers' a place it left: what revisit protocol, which measures, which failure cases, and what can be reused with original synthetic camera paths?
  The inspected papers converge on one shape of test: script a camera path that leaves a start view and returns to it (out-and-back, square loop, left-right yaw cycles, A-B-C-D-A loops), find the frame where the camera is back, and compare it with the first-visit frame. Measures range from PSNR/SSIM/LPIPS and DINO/CLIP/place-descriptor similarity to feature-match inlier ratios, depth-order correlation, pose-recovery error and VLM checklists. Three cautions recur: absolute similarity is inflated by a rollout that barely moves (R2M-Bench), the camera may not actually have returned (WorldExam, WorldMemBench), and one return is not enough (WorldMemBench found decay by the third). The reuse proposals below are labelled proposals; nothing was executed.
• articleBesides retrieving raw frames, how do a fused point-map spatial memory (SPMem) and an addressable compressed KV-cache memory (WorldTrace) keep revisits consistent, and what revisit tests and costs do they report?
  SPMem (Wu et al.) conditions a CogVideoX-based generator on three memories: recent frames, a static point cloud fused by TSDF from predicted point maps, and a few historical reference frames; its revisit test pairs frames at the same camera location in forward and reversed trajectories. WorldTrace (Wu, Elflein et al., NVIDIA) is training-free: it argues stored KV entries become unreadable when rotary position offsets leave the training range, assigns each compressed summary a fixed in-distribution position, and keeps verbatim 'landmark' scene entries; its LoopBench revisits a starting scene along ABA, ABCA and ABCDA loops and scores the return frame against the earlier generated frame with CLIP similarity. Both report paper_result numbers only; SPMem ships Apache-labelled code and checkpoints, WorldTrace code was not found.
• articleHow does Context-as-Memory pick which earlier frames to condition on, how does it test scene consistency, and what can an outside agent obtain from it?
  Context-as-Memory (CaM) stores past frames unprocessed, concatenates retrieved ones with the frames being predicted along the frame axis, and retrieves by camera-trajectory field-of-view overlap (then drops adjacent duplicates and optionally adds far-in-space/time frames). It evaluates with PSNR/LPIPS against ground truth and a stronger 'history context' comparison where a camera rotates n degrees and returns and the new frames are compared with earlier generated frames. Reported on an Unreal Engine 5 set of 100 long videos with a 1B internal base model; code was 'Coming Soon' on the project page and the GitHub path was not found, while the dataset is on Hugging Face without a licence field.
• articleWhat did each of these open interactive world models actually evaluate (metrics, baselines, raters, test sets), and what do the evaluations not support?
  The evaluations are author-run and use different protocols. Matrix-Game 1.0 uses its own GameWorld Score on Minecraft plus a double-blind human comparison against Oasis and MineWorld. Matrix-Game 2.0 reports the same score against YUME on wild scenes and is lower on motion smoothness, object consistency and scenario consistency. Matrix-Game 4.0's report and card give latency and throughput but no image-quality metric, and the card says it claims no benchmark. Hunyuan-GameCraft reports FVD, relative pose error, image quality and a 30-evaluator ranking on 150 images and 12 action signals, compared with camera-control baselines and Matrix-Game 1.0. LingBot-World 1 uses VBench on 100 generated videos over 30 s; LingBot 2.0 relies on comparison figures and an hour-long session. None measures persistence or revisit consistency quantitatively.
• articleWhat terms govern the non-text guardrail parts shipped in the Cosmos-Transfer1-7B snapshot: SigLIP so400m, the Resnet50_Final.pth face-blur model, NVIDIA's safety_filter.pt and blocklists, and nltk_data (wordnet, punkt_tab)?
  SigLIP carries an Apache-2.0 declaration in its card header (no licence file in the repo). The face-blur file Resnet50_Final.pth shares its name with a weights file that the upstream Pytorch_Retinaface project distributes (that repository is MIT-licensed; NVIDIA code cites it, but the origin of the bundled bytes is not verified); neither that licence nor its README states terms for the weights or the WIDERFACE training data, and the bundle file was not compared to upstream. NLTK data is governed per package: wordnet carries a Princeton permission notice, while punkt_tab is listed by the NLTK maintainers as an unclarified licence. NVIDIA's own safety_filter.pt and blocklist files fall under the NVIDIA Open Model License metadata of Cosmos-Guardrail1 (card unreadable). NVIDIA's ATTRIBUTIONS.md lists neither the SigLIP weights, the face-blur weights nor the NLTK data.
• articleIs the Hugging Face snapshot nvidia/Cosmos-Transfer1-7B@ef4dc8e3 the same release as GitHub nvidia-cosmos/cosmos-transfer1@5005e823, and where do they differ?
  Metadata does not show them to be one release. They are different artefact kinds (weights repo vs code repo) with no cross-reference by commit or tag, and 5005e823 (2026-06-07, a README-redirect merge) postdates every HF weight file (latest 2025-07-11) and the HF head (2025-11-19, README only). They are compatible by file name: all eight checkpoint paths the code names exist in the HF snapshot. They differ on guardrails: the code at 5005e823 loads Cosmos-Guardrail1 and Llama-Guard-3-8B from separate repos, while the HF snapshot bundles a guardrail/ folder (Cosmos-Guardrail1 contents plus an Aegis/LlamaGuard-7b folder) that predates GitHub commit b7b20fec, which replaced Aegis with Llama Guard 3.
• articleHow does TripoSG's MIT declaration sit alongside the Tencent agreements and the BRIA licence that its own files and NOTICE point to, and where do the texts conflict?
  At commit fc5c40990181e2a756c4e0b1c2f4d6b5202faf8c TripoSG declares MIT at the root while one file states that its use is subject to the Tencent Hunyuan Community License Agreement (with the agreement text embedded), a second licence file in triposg/ is the FlashVDM agreement, and NOTICE adds BRIA and Tencent code. The Tencent template lets modifiers add other licence terms 'except' where its sections 3 and 5 apply, and both agreements restrict territory and other-model improvement; whether and how those survive in an MIT-labelled release is not stated by any of the texts. Separately, FlashVDM's own source files name a Non-Commercial agreement while its LICENSE and NOTICE name the Community agreement, and RMBG-1.4's licence URL is dead. No conclusion on legal effect is drawn.
• articleIs TripoSG just MIT, or does its code include third-party parts under other agreements, and which files come from where?
  At commit fc5c40990181e2a756c4e0b1c2f4d6b5202faf8c TripoSG's root licence is MIT, but its NOTICE lists four third-party sources: BRIA RMBG-1.4 (licence bria-rmbg-1.4), Diffusers (Apache 2.0), HunyuanDiT (Tencent Hunyuan Community License) and FlashVDM (Tencent Hunyuan FlashVDM Community License). In the tree, RMBG is the file scripts/briarmbg.py; HunyuanDiT is declared in the header of triposg_transformer.py, which embeds the agreement text; FlashVDM's agreement is a second licence file in triposg/, and the flash decoder functions in inference_utils.py and attention_processor.py share much text with FlashVDM's files. The README has no licence section.
• articleHow do the papers measure quality degradation over rollout time (curves, drift deltas, judges), what do the numbers show, and where are the metrics unreliable?
  Four measurement styles appear: (1) an imaging-quality-versus-time curve over 30 s (CausVid Fig. 8); (2) a first-5-s versus last-5-s imaging-quality delta, "ΔDriftQuality", on 30 s videos (Rolling Forcing); (3) whole-video VBench-Long dimensions at 50/75/100 s plus a Gemini-2.5-Pro "visual stability" score (Self-Forcing++); (4) VBench dimensions at 50/250 s with an explicit warning that copied clips fool consistency metrics (SVI). The authors of Self-Forcing++ and SVI both say standard image/consistency metrics can reward degraded or static video, so motion (dynamic degree) must be read with them.
• articleWhat training and inference remedies do the primary papers use against error accumulation in long autoregressive video, how does each work in steps, and what does each cost?
  Seven remedy families appear: (1) noise the past latent at sampling (Diffusion Forcing); (2) distil a causal student from a bidirectional teacher with DMD (CausVid); (3) train on self-generated rollouts with KV cache (Self Forcing); (4) roll out beyond the teacher horizon and distil on re-noised windows (Self-Forcing++); (5) train-long-test-long with a frame sink (LongLive); (6) rolling-window joint denoising with an attention sink (Rolling Forcing); (7) error recycling (Stable Video Infinity). The papers disagree on whether an attention sink works alone (LongLive says only after long tuning; Rolling Forcing ablates it as helpful in its own recipe).
• articleHow long have these autoregressive video systems been shown to run, and which lengths are measured in a paper versus only shown in a figure or demo?
  Measured (table) lengths in the primary papers are 30 s (CausVid, LongLive, Rolling Forcing), 50/75/100 s (Self-Forcing++) and 50/250/300 s (Stable Video Infinity). Longer claims are qualitative figures or demos: 255 s (Self-Forcing++ Fig. 6), 240 s (LongLive appendix + demo page), over 2 min (Rolling Forcing Fig. 4), 1000 frames at 128 px (Diffusion Forcing, qualitative) and 8-14 min (SVI README). Self Forcing itself reports no quality measure beyond 5 s and states that degradation remains past the training length.
• articleWhere does the shared state of a multiplayer world model live, and what are the basic design families the primary papers use to keep several players consistent?
  The eight papers fall into four placements of the shared state: inside one joint network (Solaris, MIRA, Gamma-World, ShareVerse) coupled by attention over all views or via hub tokens; in an explicit global state encoder (MultiWorld); in an explicit typed state with separate renderers (MASS); or exchanged between per-player clients as player and scene state (WorldCast). MetaWorld couples two denoising branches by cross-attention at every layer. Which design is better is not settled: papers compare only against their own retrained baselines.
• articleIf an LLM updates the numbers (state variables) of a game or story each turn, how often did it get the update wrong in published tests?
  RPGBench (arXiv 2502.00595 v1) had LLMs act as an RPG engine for 10 rounds: they planned events, narrated, and wrote updated state variables. Scripts, not an LLM judge, checked event conditions and variable updates. The best model, Gemini 2.0 Flash Exp, had 76.5% of rounds free of mechanical errors; GPT-4o 69.3%; the lowest were 11.3-20.4% (Claude 3.5 Sonnet, GPT-4o mini, Llama 3.1/3.3 70B). Variable-update error rates per variable ran from 3.4% to 30.8%. Accuracy fell as rounds increased for GPT-4o (69.3% at 10 rounds to 66.8% at 25). Combined with Wang et al. (-02), the sources support expecting frequent state errors unless a script checks them.
• articleCan I get a free-viewpoint 4D scene from one ordinary (phone) video, and how do I tell whether a monocular dynamic-scene result is real?
  The primary sources read say: not reliably. The DyCheck paper (arXiv 2210.13445v1) shows that common monocular benchmarks effectively leak multi-view signal through teleporting cameras or quasi-static scenes, and that with realistic camera motion and complex motion published NeRF-era methods lose roughly 1-2 dB (masked PSNR, no multi-view cues) and 4-5 dB (complex motion) per the abstract. The 4D-GS paper's own appendix reports overfitting to training views on monocular iPhone-style input and failure on large motion, and Dynamic 3D Gaussians states it needs a multi-camera rig. A reader can check a monocular claim by looking at how the camera moves relative to the scene (EMF), whether metrics are co-visibility masked, and whether sequences are not quasi-static. DyCheck evaluated NeRF-era models (T-NeRF, NSFF, Nerfies, HyperNeRF), not Gaussian methods; no read source reports a Gaussian method on its masked metrics.
• articleWhat does it mean to reconstruct a dynamic (4D) scene with 4D Gaussian splatting or a dynamic NeRF, what do I have to capture, and what do I actually get back?
  A 4D scene method fits a per-scene model to video so that images can be rendered for a chosen camera and a chosen time. In the papers read, 4D Gaussian splatting keeps one set of 3D Gaussians plus a learned deformation (or native 4D primitives) queried by time; dynamic NeRFs (D-NeRF family) do the same with a canonical radiance field plus a deformation network. What you must capture depends on the method: Dynamic 3D Gaussians needs a synchronised multi-camera rig (27 training cameras in its experiments); 4D-GS and Real-time 4DGS are benchmarked both on 15-20 camera rigs (Neu3D) and on one- or two-camera and synthetic monocular sets; D-NeRF assumes one camera and one image per time instant. The output is a trained model that renders novel views at times inside the captured span (plus optional per-timestamp Gaussian exports), not a rigged or topology-stable mesh. Every number here is author-reported and nothing was run.
• articleHow comparable are the evaluation protocols (datasets, subsets, alignment, baselines) across TripoSR, SF3D, SPAR3D, TRELLIS, TRELLIS.2, Hunyuan3D and TripoSG?
  Protocols differ enough that absolute numbers should not be compared across papers. TripoSR, SF3D and SPAR3D share GSO and OmniObject3D but use different subsets (about 300 each; 278 and 308; about 250 each) and different GPUs; SF3D re-measured TripoSR at CD 0.111 / FS@0.1 0.645 on GSO while TripoSR reports CD 0.111 / 0.651 on its own subset. TRELLIS uses Toys4k with depth-unprojected points; TRELLIS.2 adds 90 curated Sketchfab assets and 100 AI-generated prompts; Hunyuan 2.0 anonymises several baselines; TripoSG evaluates shape-only with Normal-FID and an LMM judge. Only SF3D, SPAR3D and TripoSR share a metric and datasets closely enough for rough comparison, and even they were each run by the authors of the newer method.
• articleDo learned latent actions keep the same meaning across different scenes, viewpoints and bodies, and how well can a motion be transferred from one video to another?
  Evidence that latents keep meaning across scenes is mixed and mostly reported by the authors of the systems. Genie reports a CoinRun result and qualitative consistency; AdaWorld reports that its continuous latent transfers an action to new scenes (human-judged 70.5% LIBERO, 61.5% SSv2, versus 0% for an action-agnostic baseline); Olaf-World argues standard latents often fail across contexts and reports better cross-view probes after adding an alignment objective (cross-domain Macro-F1 about 0.59-0.63, versus about 0.48-0.50 for the AdaWorld baseline in its re-run). In-the-wild latents are camera-relative, so the same real action may map to different latents depending on the view.
• articleWhat is a latent action model, and how is one trained from video that has no action labels?
  A latent action model (LAM) is trained on pairs of consecutive frames with no action labels. An inverse dynamics model compresses what changed into a small latent; a forward model must predict the next frame from the past frame plus that latent, so the latent has to carry the action-like change. LAPO (2023, v2 pinned) fixed this template; Genie (2024) used it with 8 discrete codes so a person could pick one per step; AdaWorld (2025) used a continuous latent. LAPO tested on synthetic game data with known actions; web video was the stated motivation, not the tested case.
• articleBeginner: which question should an agent ask first to choose between a world model, a video generator, a game engine or a 3D scene representation, and which of these tradeoff statements come from primary sources?
  No inspected primary source gives a decision procedure; the decision rule below is my synthesis. Sources do state tradeoff facts. Game engine: GameNGen describes games as manually crafted systems running a game loop; the UE 5.1 in-camera VFX page (IA snapshot) describes real-time rendering with camera tracking. 3D Gaussian Splatting: input is images of a static scene; real-time rendering after training; memory and artifacts in unseen regions are limits; the paper says nothing on editing. Action-conditioned world models: GameNGen (20 FPS DOOM, ~3 s history), Genie 3 (24 fps, 720p, few minutes consistency), NWM (planning in known environments). Video generators: physics and consistency measured elsewhere (C3-B006-W3). Proposal: pick by what must stay fixed (see decision table in finding F1).
• articleCan I say current video models understand physics? What do the newer benchmarks support me saying, and what must I avoid claiming?
  After reading six 2025-2026 benchmark papers, a safe statement is: on each paper's own tests, current video generators score well below ceiling on many physical behaviours, do better on some (reflection, buoyancy, communicating vessels, sand-pile angles) than others (momentum, collisions, melting with state change, persistence after the camera looks away), and visual quality does not predict physical consistency. Unsafe statements: that any model 'has learned the laws of physics', that a score is comparable across benchmarks, or that open or closed models are generally ahead. The papers do not test training-distribution generalisation, so they neither confirm nor refute 'learn' in the mechanistic sense.
• articleGiven the stated compute and latency, what can a real-time previs or game-prototyping setup realistically run today, and what should be checked first?
  From the sources read (synthesis): (a) free-running action-conditioned generation with released weights is documented at 25 FPS on an H100 (Matrix-Game 2.0, 352x640), far slower for higher resolution or larger models (GameCraft 0.25 FPS, 6.6 distilled, 720p); (b) the only documented consumer-GPU real-time path, Matrix-Game 4.0, does not free-run: a game engine keeps state and supplies low-poly frames which the model renders, 480p, with three trained scenes (two shooters, one racing); (c) memory needs range from about 9-13 GB (MG4 on RTX 5070 Ti/5080) to 24 GB minimum (MG2 README, GameCraft README) and 108-123 GB for a compared system. For previs, the setup question is therefore which trained scene or domain applies, not only the GPU. First safe steps are to record each figure's setting, run only code and weights after a licence and checkpoint-format review, and measure key-press-to-frame latency on the target hardware.
• articleWhat does this literature support, and not support, for turning any existing video into a playable or directable world?
  Synthesis, not a result. Papers support: (a) learning a small discrete or continuous control interface from unlabelled video in a domain (platformers, robot arms, rendered walkthroughs); (b) copying a motion between clips with released AdaWorld/Olaf-World-style models in the authors' tests; (c) adapting such a model to real controls with little labelled data. They do not support, by what was inspected: arbitrary film footage with cuts and moving cameras; long persistent worlds (Genie's 16-frame memory; AdaWorld's limit beyond the initial scene); named, stable player controls; rights clearance of the source video. Treat a latent-action world from existing video as a research-preview capability that must be probed per source, not assumed.
• articleWhich of these open 3D generators can be used commercially, and what territory, user-count or revenue limits do their licence texts state?
  Three licence families appear. TRELLIS, TRELLIS.2, TripoSR and TripoSG state MIT for code and models (README/LICENSE text; Hugging Face metadata agrees), with listed exceptions for submodules or dependencies. Hunyuan3D 2.0 and 2.1 use dated Tencent community licences whose text excludes the European Union, United Kingdom and South Korea from the licensed Territory, requires a separate licence above 1 million monthly active users at the release date, and forbids using the works or outputs to improve another AI model. SF3D and SPAR3D use the Stability AI Community License (Last Updated July 5, 2024): free for research/non-commercial; commercial use needs registration and ends above USD 1,000,000 annual revenue. Texts address use, distribution and outputs differently; none of this is legal clearance.
• articleWhat failure modes (identity drift, geometry violations, flicker) do sources report when photoreal video is generated from engine renders or structural controls?
  Reported failure modes come from papers and vendor caveats, not studio accounts. RealMaster (research, GTA-V) reports identity drift and colour shifts in an Aleph baseline, flicker and identity fluctuation from multiple image-edit anchors, face-structure loss under depth conditioning, and temporal artifacts under fast motion. NVIDIA reports hallucinated or misoriented vehicles for Transfer1 AV variants and error accumulation over long clips, and Transfer1 papers show geometry errors for single controls (segmentation-only headings). Vendor docs warn that high control weight can cost quality and that image-sourced subjects may be inconsistent.

More