Virtual production and spatial media

Real-time scenes, cameras, capture, XR, volumetric and spatial storytelling

• articleHow does the VR180 format define projection meshes, per-eye UV adjustment by stereo mode, and camera-motion metadata?
  VR180 is a profile of Spherical V2 metadata: two projection meshes (one per eye; ~40x40 grids typical), per-eye UV adjustment by stereo mode, and optional camm camera-motion metadata. Mesh index bit widths are ceil(log2(count*2)) with zig-zag deltas (defined in the V2 RFC mesh box, cited by VR180).
• articleWhat does Epic's Live Link Face documentation say the app outputs (live data, recorded files, timecode, OSC) at UE 5.3?
  UE 5.3's Live Link Face guide says the iOS app streams ARKit facial animation to Unreal over the network (not USB); each recording saves a .mov reference video (JPEG-compressed frames, audio, timecode) and a .csv of raw ARKit curves importable via the Beta LiveLinkFaceImporter plugin into a Level Sequence; timecode comes from the system timer, an NTP server (default time.apple.com) or Tentacle Sync, and the app accepts OSC remote control.
• articleWhat do the publicly readable SMPTE ST 2059-1, ST 2059-2 and ST 12-1 documents define about synchronising video signals and time code?
  SMPTE ST 2059-1:2021 defines the SMPTE Epoch (1970-01-01 00:00:00 TAI, equal to the PTP epoch) and how interface signals and ST 12-1 time code are aligned to it; ST 2059-2:2021 profiles IEEE 1588 PTP (default domain 127) with a synchronization-metadata TLV for time-address flags, jam and leap-second data; ST 12-1:2014 defines the time address, drop-frame counting, LTC and VITC. The PDFs are freely readable at pub.smpte.org (SMPTE copyright).
• articleWhich parts of an IES file do Blender, Unity and Unreal actually read, ignore or reject when used for virtual lighting?
  The parsers differ in tolerance: Blender v5.2.2 skips TILT=INCLUDE tables (never applying tilt), ignores lumens and units, accepts types A/B/C, mirrors missing quadrants ('much more forgiving than the spec') and converts candela with a fixed 4*pi/177.83 factor; Unity recognises versions from IESNA91 to IES:LM-63-2019 but rejects any TILT other than NONE; Unreal documents candela brightness and clipping of profiles to the spot cone.
• articleWhat do Unity's URP and HDRP docs say about enabling HDR output, tonemapping modes, paper white and UI brightness for real-time output?
  Unity applies HDR output as tone mapping (including Rec.709-to-display-gamut conversion) followed by a transfer function; URP and HDRP offer Neutral and ACES modes, HDRP's ACES mode has fixed 1000/2000/4000-nit presets, paper white sets unlit/UI brightness, and the docs recommend a player calibration menu; HDRP has gamut and over-paper-white debug views.
• articleIn Godot, how do I enable HDR output and choose between tonemapping and the output-max-linear-value approach, and which tonemappers are compatible?
  Godot's HDR output guide lists enabling steps (no SDR-only environment features such as Filmic/ACES tonemapping or Soft Light glow), two approaches (tonemap scene lighting, or use get_output_max_linear_value only for direct presentation), notes that Filmic and ACES ignore the output maximum and stay SDR, supports iOS, Wayland, macOS, visionOS and Windows but not Android, X11 or web, and warns that many HDR screens apply their own tonemapping (prefer HGiG/clip modes during development).
• articleDoes OpenUSD's Hydra pipeline (hdParticleField, hdSt, hdEmbree) at v26.08 use ParticleField3DGaussianSplat sortingModeHint and projectionModeHint?
  At v26.08 no Hydra path read consumes the splat hints: the schema calls them advisory (sortingModeHint allows zDepth, cameraDistance and rayHitDistance), the usdImaging adapter only has a TODO to pass them, the hdsi conversion never mentions them, and the sample renderer sorts by view-space z regardless.
• articleI author a splat prim in USD: should I set sortingModeHint and projectionModeHint, and will Hydra renderers honour them at v26.11.0?
  The hints are advisory: at v26.11.0 they reach Hydra scene data, but Pixar's inspected consumers (hdPrman conversion, sample renderer) do not read them, so setting them is harmless and should not be expected to change output there; the docs now describe rayHitDistance and say rasterizers may treat it as cameraDistance.
• articleWhat changed in OpenUSD's Gaussian-splat particle-field path between v26.08 and v26.11.0, and does any renderer path now use the sorting or projection hints?
  At v26.11.0 usdImaging exports a Hydra particleField container with kernelType (hardcoded gaussianEllipsoid) and both hints, closing the v26.08 TODO; the new hd schema exposes only kernelType, and the hdsi conversion, hdPrman plugin, sample renderer and schema.usda are byte-identical to v26.08, so by code reading none of them consumes the hints.
• articleHow do depth-of-field and hyperfocal-distance calculators differ in their assumptions (circle of confusion, hyperfocal definition), and what does ZEISS say?
  ZEISS uses z = D/1500 (0.029 mm for 24x36) in its charts, gives hyperfocal approximately f'^2/(z'k) and treats the circle of confusion as a format property; a secondary source records a second hyperfocal definition differing by one focal length (H = f^2/(Nc) + f) and calls the 'Zeiss formula' D/1730 apocryphal. CoC choice and the +f term shift results: for 50 mm at f/2.8 on full frame, 35.8 m (0.025 mm, +f), 31.0 m (ZEISS D/1500) and 29.8 m (0.03 mm, +f), recomputed by the curator.
• articleIs there any numeric RGB vs RGBW vs RGBCA comparison (TM-30, SSI, CRI/TLCI, delta E) and does it cover skin tones or a camera?
  The only numeric RGBW-versus-RGBCA comparison found is ROE's November 2024 brochure: TM-30 Rf 80 for both, SSI 68 for both at 3200 K, and SSI 62 (RGBCA) vs 52 (RGBW) at 6500 K, stated as reference values that may vary. These are panel-spectrum metrics with no camera, delta E or skin-tone data.
• articleWhat does an ASC-published first-hand test report say about evaluating exposure and colour on an LED wall, including metering and camera sync?
  A first-hand ASC test of Kodak film and an Alexa 35 against an LED wall (D65 native, content at 3200 K and 5600 K) reports that light meters misread narrow-band LED light so a digital camera and calibrated monitor stay necessary, that wall colour must be calibrated to the shooting camera's colour matrix, and that cameras must sync to panel refresh (genlock for digital; a sync box for film).
• articleWhy can skin and costumes look wrong on an RGB LED volume, and what do RGBW and RGBCA panels claim to change (and what is only a vendor claim)?
  An ASC-reported 2021 test compared RGB, RGBW and RGBCA panels and a Kino Flo Mimik with a spectrometer and colour meter and found visible improvement over RGB (RGBCA and Mimik better than RGBW for warm and off-white content), without a numeric score; Brompton and ROE describe added white or cyan/amber emitters as broadening the spectrum, Brompton notes cameras see the white LED differently, and ROE claims better SSI and skin rendering without skin data.
• articleWhat have open papers measured about colour-rendition limits of RGB LED volumes for lit subjects (skin, clothing), and how far do 3x3 corrections get?
  Two open papers attribute LED-volume colour errors to narrow-band RGB emission seen through camera sensitivities (lighter skin pinker); a 3x3 pre-correction of panel content plus camera post-correction (and black-level subtraction for 4-10% panel albedo) improves skin and orange/yellow materials, but residual errors remain (sodium-vapour light; saturated reds and greens) that no 3x3 matrix can remove. Neither tests RGBW/RGBCA panels or diverse skin tones.
• articleWhat are storyboards, animatics, previs, techvis, stuntvis and postvis, and how does Epic's guide order them?
  Epic's Virtual Production Field Guide v1.3 defines visualization as prototype imagery conveying a shot's creative intent, lists pitchvis, previs, virtual scouting, techvis, stuntvis and postvis (techvis about physics and camera data, postvis merging live action with temporary effects), and calls storyboards and animatics precursors to previs. No Previsualization Society definition was available.
• articleWhat does Unreal Engine 5.8 documentation say about which USD features load through the USD Stage (prims, animation, blend shapes, materials, variants, payloads), and could the Unreal glTF importer documentation be obtained?
  Unreal's Beta USD Stage represents static and skeletal meshes, HISM, materials, lights, cameras, variants, animation and blend shapes as prims, loads .usd/.usda/.usdc/.usdz, supports payload loading, UsdPreviewSurface textures and PreviewSurface/DisplayColor materials (MDL via a schema), shows xform animation as Transform tracks and other animated values on a Time track in a Level Sequence, and needs FORCE_ANSI_ALLOCATOR=1 for runtime loading. Unreal glTF import docs were unreadable (producer).
• articleWhat do the SPZ v4 and PlayCanvas SOG formats specify for compression, structure and licensing, and how do viewers describe their support?
  SPZ v4 (MIT) is a 32-byte plaintext NGSP header plus independently ZSTD-compressed attribute streams (24-bit fixed-point positions, smallest-three 10-bit rotations); v1-3 gzip files are read-only. PlayCanvas's SOG stores quantised attributes as WebP images with log-domain positions/scales and gamma-space DC colour in a right-handed +Y-up frame, and PlayCanvas compares SOG (15-20x smaller than PLY, lossy), SPZ (about 10x, lossy) and lossless GLB with KHR_gaussian_splatting. Its SPZ script parser accepts only v4, up to 3 SH bands, ignoring the antialiased flag and vendor extensions.
• articleHow do USD layers/sublayers and Unreal Engine USD multi-user editing support several people working on the same scene, and what limits do the OpenUSD docs state?
  OpenUSD treats a layer as the atomic persistent unit, advises sublayering over referencing when namespaces match, and requires single-writer, no-concurrent-read editing of a stage's layers. Unreal's Beta USD multi-user editing syncs prim add/remove/rename, attribute and visibility edits, needs identical file paths on all clients (USD under project source control), and cannot undo prim deletion. No source read describes text-level .usda merging. OpenUSD code is under TOST 1.0 (Apache 2.0 with a different trademark section).
• articleHow does Godot 4.7.2 Movie Maker mode fix the time step and capture audio for offline cinematic capture, and what does it promise about identical output?
  Godot Movie Maker renders non-real-time with a fixed step (default 60 fps when --write-movie is used without --fixed-fps), so pacing is perfect regardless of hardware; audio is mixed mix_rate/fps samples per frame and the writer warns when the mix rate is not divisible by fps (audio drift); not meant for live gameplay capture.
• articleWhat concrete caption, audio-cue and locomotion-alternative guidance do Meta Horizon OS and Apple visionOS publish, and how do their precision and status differ from XAUR?
  Only Meta gives numeric caption parameters (about 1 m start, ~0.5 s roll-up, 32 characters x 2 rows, at most 4 speakers) and labels them evolving and non-blocking; Apple and XAUR stay at principle level; none cites measurements. Meta's optional head-leashed captions vs Apple's advice not to anchor content to the head is kept as a platform difference.
• articleWhat does the W3C XR Accessibility User Requirements Note ask for regarding captions, audio, locomotion and motion in immersive media?
  XAUR (W3C Note, 2021) states XR needs and requirements (captions with reflow, text descriptions of important audio, mono audio, motion-agnostic input, sickness-trigger avoidance) at requirement level; its one numeric value is the flicker limit wording (more than 3 times a second).
• articleWhat do the OpenXR face and eye tracking extensions define about blendshape counts, timing and ratification?
  At OpenXR-SDK 1.1.65, XR_FB_face_tracking2 (ext 288, rev 1, not ratified) exposes 70 expression weights, 2 confidence areas and a visual or audio data source, each result carrying an XrTime that is tracked or extrapolated; XR_FB_eye_tracking_social (ext 203) returns per-eye gaze with time; XR_HTC_facial_tracking defines 14 eye and 37 lip expressions.
• articleHow do the MPEG glTF extensions (MPEG_media, MPEG_buffer_circular, MPEG_accessor_timed, MPEG_texture_video, MPEG_scene_dynamic, MPEG_viewport_recommended) wire timed video, V3C and dynamic scene data into a glTF scene?
  The Khronos-hosted MPEG glTF extension READMEs (7cee0fcd) are based on ISO/IEC FDIS 23090-14 and define timed media through MPEG_media (alternatives with required mimeType, listed value video/mp4), a circular buffer (count default 2, minimum 2), a timed accessor that scenes with timed data shall use (suggested update rate default 25) and a video texture (format default RGB); each README states Copyright ISO/IEC 2022 with use subject to ISO's licence and IPR policy, not the repository licence.
• articleWhat do ITU-R BT.2100, BT.2408 and BT.2446 specify for PQ/HLG, HDR reference white and HDR-to-SDR conversion, and how do BT.2446 methods A, B and C differ?
  ITU-R BT.2100-3 (Feb 2025) defines PQ as an absolute EOTF up to 10,000 cd/m2 and HLG with a display-dependent system gamma of 1.2 at a 1,000 cd/m2 nominal peak, and sets HDR reference white (100% reflectance card) at 203 cd/m2; Report BT.2408-8 (2024) marks that level as 75% HLG / 58% PQ on a waveform and discusses conversion at a 1,000 cd/m2 reference condition; Report BT.2446-1 (2021) describes static HDR-to-SDR methods (Method B using a 291 cd/m2 nominal peak) and notes that adaptive or dynamic methods exist.
• articleWhat do the VES on-set data guide and the tracking tools say about survey measurements and witness cameras for camera tracking, and what do they leave unspecified?
  The VES guide (title page 1.1.0, 2026-05-12) lists witness cameras, lidar, photogrammetry, lens encoders and reference-camera timecode as items to capture, each with purpose and consumers, but gives no quantities (no camera counts, marker sizes or tolerances) in the sections read. Tool manuals say how survey data is consumed: Nuke ties sequences to a known 3D world via survey points; Blender sets scale from a distance in scene units; SynthEyes keeps "survey shots" (stills) distinct from set surveys. The term "witness camera" means extra video angles in the VES guide and unordered stills in the SynthEyes manual.
• articleHow should a lens grid be shot on set so a tracker or compositing tool can use it, and what goes wrong with a bad grid?
  A 2026-05-01 practitioner page gives a field checklist: rigid, flat, level grid; a long dolly track; the grid filling the whole frame; sharp focus; even light at about F11 for spherical lenses; and a slate with body, lens, actual focal length and focus distance. It quotes a supervisor against mixed-resolution grids and tape over intersections. Nuke’s grid-analysis page states it should be used when a grid exists and gives a recovery step when detection under-covers the image; Blender’s manual says it has no calibration tool and points to OpenCV.
• articleWhere do the lens-distortion models of matchmove and compositing tools (Nuke, SynthEyes, OpenCV, Blender/libmv, COLMAP, 3DE, OpenTrackIO) disagree, and what parameters must travel with a shot for the distortion to be reproduced?
  Reading pinned vendor docs and source, the models disagree in at least five ways: direction (Blender division applies x/(1+k r^2) from ideal normalised to image; Nuke Classic and COLMAP SIMPLE_DIVISION use the same form from distorted to ideal), the meaning of k4 (OpenCV: a rational denominator term; Blender Brown: an r^8 polynomial term), the roles of p1 and p2 (Blender Brown is swapped relative to OpenCV, COLMAP and Blender Polynomial, from code reading), the normalisation (OpenCV by z with focal in pixels; Nuke width/height/diagonal/maximum; Blender Nuke model by half the larger image side; OpenTrackIO offsets in mm), and pixel origin conventions. For the shot to be reproducible the record must carry model name, direction, coefficient order and units, normalisation, centre/offset, sensor size and resolution, focal length and the focus/zoom state; an STMap or projection screen is the model-free alternative with storage and precision costs. 3DEqualizer model equations were not read...
• articleWhat did Unreal Engine 5.1's documentation say about the OpenColorIO plugin and HDR display output for cinematics and virtual production?
  In Unreal 5.1 documentation, OCIO came through one OpenColorIO Configuration Asset used by the viewport/Play-in-Editor, Composure, Blueprints, nDisplay and Movie Render Queue, and the engine supported OCIO v1 configs only. Turning on the viewport OCIO display disables the Tone Curve, and MRQ's Color Output can apply an OCIO config and disable the Tone Curve, so whether written frames include the tone curve depends on that setting (see CA-B374 for MRQ file encodings). HDR display output was limited to 1000/2000-nit paths and D3D11 exclusive fullscreen. Confirm the engine release before relying on these limits.
• articleWhen lens metadata reports "focal length", which focal length is it, and what reference point do entrance-pupil and focus values use?
  Cooke distinguishes nominal, pinhole and effective focal length and warns that mixing them causes matchmove problems; positions are referenced to a nominal image plane 52 mm behind the PL flange; pinhole focal length is available as /i lens metadata on supporting lenses and otherwise must be measured by grid calibration or solved in a match-move. Record which focal length a metadata field carries.
• articleDo the Panasonic AW-UE100 and AW-UE160 public documents describe FreeD output field mappings and Camera ID the same way as the AW-UE150 manual?
  Panasonic's interface specifications for AW-UE150/HE145 (2022), AW-UE100 (2022, producer) and AW-UE160 (2025) list a settable Camera ID command (OSJ:F4) under tracking-data output, while the AW-UE150 operating manual (2025) says the FreeD Type D1 Camera ID field 'Always stores FFh'; the vendor documents conflict on whether Camera ID is configurable. None of the UE100/UE160 documents read gives a field-by-field FreeD mapping.
• articleHow do the original 3DGS PLY layout, Khronos KHR_gaussian_splatting and Niantic SPZ differ in coordinate system, colour (spherical harmonics), activation (scale/opacity) and rights, so that conversions between them are correct?
  The reference 3DGS code (54c035f7) writes PLY columns holding pre-activation parameters (log scale, pre-sigmoid opacity), so copying them into KHR_gaussian_splatting (linear scale, 0..1 opacity, degrees 0-3, required colour space) needs exp/sigmoid conversion; SPZ (MIT) stores RUB coordinates, names PLY as typically RDF and GLB as LUF, and supports SH degree 4, which KHR cannot carry without truncation. The three components carry different terms: 3DGS code research/non-commercial, SPZ MIT, KHR text Khronos copyright with a CC-BY-4.0 header.
• articleWhat files, metadata and HLS settings does Apple state for authoring and publishing Apple Immersive Video, and where do the stated bitrate tiers disagree?
  Apple describes Apple Immersive Video as captured at 90 fps with more than 50 MP per eye, targeting 60 pixels per degree and up to 230 degrees field of view; WWDC25 session 403 recommends MV-HEVC 4320x4320 per eye at 90 fps in P3-D65-PQ for HLS, with the AIME VenueDescriptor and APAC audio referenced from the playlist. The session's spoken tiers (25-100 Mbps average, 50-150 Mbps peak) conflict with its own summary (25-150 Mbps average); the conflict is preserved.
• articleWhich of the scene-graph systems have released code, under what stated licence, and what third-party models, keys or data does each need?
  At the observed commits, Hydra (BSD-style LICENSE), ConceptGraphs (MIT) have permissive code licences, while the 3DSceneGraph repository carries a non-commercial research-use software licence and its data sits behind a separate click-through agreement that was not accessed. ConceptGraphs code is MIT but running it needs third-party segmentation and vision-language checkpoints plus an OpenAI API key, whose terms are separate. A permissive code licence does not state anything about weights, data or outputs.
• articleWhat is lens metadata, which values do Cooke /i lenses and ARRI cameras give per frame, and should I rely on the 'focal length' value as a camera solve input?
  In the ARRI SDI Metadata white paper dated 25 July 2023, ALEXA 35 SUP 1.2 changes add Lens Effective Focal Length (LDI26) and Lens Entrance Pupil Position (LDI27) alongside the existing Lens Focal Length (LDI04, 1/1000 mm).
• articleMy stereo or immersive video file is going to Apple Vision Pro: which Apple movie profile (3D, Spatial Video, APMP for delivery, APMP for production) applies, and what does each allow?
  Apple's Movie Profiles for Spatial and Immersive Media (v0.9 Beta, 9 June 2025) distinguishes 3D/Spatial Video (rectilinear MV-HEVC) from APMP delivery and production; APMP for production permits other codecs and frame-packed video while delivery is MV-HEVC, with vexu boxes signalling projection, packing and disparity adjustment.
• articleWhat does Apple publish about MV-HEVC stereo video files and the spatial metadata that turns a stereo MV-HEVC video into 'spatial video'?
  Apple's HEVC Stereo Video Interoperability Profile is Version 0.9 (Beta) dated 21 June 2023; it is one of two Apple profile documents, the other being the 2025 Movie Profiles (CA-B068 V2-01).
• articleI have splat values from a typical training export (log scales, logit opacity): which values must I convert before putting them in a glTF KHR_gaussian_splatting file or a USD ParticleField3DGaussianSplat prim?
  Both glTF KHR_gaussian_splatting and OpenUSD ParticleField3DGaussianSplat store linear scale and linear 0..1 opacity; the OpenUSD example converter applies exp() to PLY log scales and a logistic sigmoid to PLY logit opacity and treats rot_0 as the quaternion real part; glTF marks out-of-range opacity invalid and defines diffuse colour from SH0 with a +0.5 bias at render time.
• articleWhich controls and evaluations from driving and robot world models could carry over to media previs or virtual camera planning, and what has not been shown?
  SYNTHESIS, not a measured result. Transferable pieces: (1) camera-pose conditioning from a single image with Plucker rays (Cosmos; joystick-style paths shown qualitatively, scored by pose re-estimation on static scenes); (2) multi-view synchronised re-rendering along arbitrary camera trajectories (Predict2.5 robot multiview; cross-view Sampson error 19.73 px vs 26.61 px baseline); (3) re-driving a logged scene while holding everything else fixed ("world-on-rails", GAIA-3/4 company pages) and embodiment (rig) transfer; (4) edit-by-partial-noising for weather/time-of-day and box-guided agent inpainting (GAIA-2, qualitative); (5) language -> trajectories (DriveDreamer-2). Transferable evaluations: re-estimate the camera path from the output and compare with the commanded path (pose success, rotation/translation error, Vista IDM), Sampson cross-view error with a real-footage floor. Not shown by any source read: film-style subjects (actors, close-ups), cuts, lens language, or non-static ...
• articleWhich control inputs (ego trajectory, steering/speed, agent boxes, map, weather text, camera layout, robot actions, camera pose) do the driving and robot world models accept, and how is each encoded?
  The driving systems expose a small set of structured controls: ego motion (speed + curvature in GAIA-1/2; angle/speed, trajectory, command or goal point in Vista; yaw + velocity in DriveDreamer; 64-point trajectory in Cosmos), other agents (3D boxes in GAIA-2 and DriveDreamer; LLM-written trajectories in DriveDreamer-2), scene attributes (categorical metadata, CLIP or T5 text), and the camera rig (intrinsics/extrinsics in GAIA-2; per-view captions and view embeddings in Cosmos). Robot systems use gripper-frame action vectors (Cosmos on Bridge, 7-D) or whole-body low-level trajectories (1XWM). Camera-pose control with Plucker rays exists in Cosmos but was trained on static real-estate-style scenes. Most control interfaces were trained on proprietary data, and several are demonstrated only in figures.
• articleWhen driving and robot-learning papers say "world model", what exactly do they mean (inputs, outputs, intended use), and where does the word claim more than the evidence tests?
  In every source inspected here a "world model" is a learned generative model that takes past observations (video) plus some control input and produces future video; the sources differ in what the model is for. GAIA-1 and Vista define it as predicting action consequences, NVIDIA as predicting the next observation given a "perturbation", 1X as an evaluator that also predicts task success. Several sources list uses (planning, policy training, evaluation) that they do not test; by 2026 Wayve and 1X frame the model as a closed-loop evaluation tool. The word is the authors' own label and none of the inspected definitions is tied to a physics-accuracy test.
• articleHow many frames and what resolution can each camera-pose-conditioned checkpoint produce in one pass at the pinned revisions, and what is documented about longer clips?
  Single-pass limits are short: CameraCtrl 16 frames at 256x384 (AnimateDiffV3 build); MotionCtrl 16 frames at 256x256 (VideoCrafter main branch; its UI text says 14, a documentation inconsistency within one file) and 14 frames on SVD; CamI2V 16 frames at 256x256 or 512x320; Stable Virtual Camera 21 frames per pass at 576x576 default, with the authors reporting 80 to 134 frame sequences via chunking; GEN3C 121 frames per pass (121*N-1 autoregressive), example at 704x1280 while its model card says 720*1080; ViewCrafter 25 frames at 576x1024 (or 16 frames, or 25 at 320x512); Cosmos-Predict2.5 agibot 93 frames. Timings and memory in READMEs are the authors' statements (A100 40G for ViewCrafter; ~43GB for GEN3C with offloading) and were not reproduced. A 5 to 10 second previs shot therefore needs chunking or autoregression for every system except possibly GEN3C and SVC, and only SVC and GEN3C document how to do so.
• articleFor each camera-pose-conditioned checkpoint, what exactly is the pose convention (world-to-camera vs camera-to-world, OpenCV vs OpenGL, relative to first frame, intrinsics, units or scale)?
  Conventions differ and must be matched per system. CameraCtrl, MotionCtrl and CamI2V take world-to-camera 3x4 matrices (RealEstate10K style, OpenCV camera axes) and all re-express them relative to the first (conditioning) frame, so absolute world axes and origin drop out but the camera axis convention and handedness do not. Stable Virtual Camera takes camera-to-world matrices in the OpenGL convention and internally flips two columns to OpenCV. GEN3C works in world-to-camera with OpenCV intrinsics and scene-relative distances (scaled by an estimated centre depth). ViewCrafter is parametrised by spherical offsets about a point at the depth of the image centre and needs a hand-estimated elevation. No inspected source gives a metric unit: MotionCtrl uses a unitless 'speed', CamI2V a trace_scale_factor, SVC a camera_scale (with a documented single-view scale ambiguity), GEN3C and ViewCrafter a depth-relative scale. Cosmos-Predict2.5's loader inverts file extrinsics before use, so even wi...
• articleWhich open-weight camera-controllable video or novel-view generators exist, and for each one does it take an arbitrary per-frame camera path, or only presets?
  At the pinned revisions, all seven inspected systems take a camera trajectory as data; five accept free per-frame poses: CameraCtrl (16-pose RealEstate10K-style txt), MotionCtrl (one 12-float RT row per frame), CamI2V (RealEstate10K-style rows), Stable Virtual Camera (transforms.json, OpenGL convention, any number of target cameras) and GEN3C (per-frame world-to-camera matrices via NPZ or its GUI; the single-image CLI has presets only). ViewCrafter is different: it takes three short offset sequences (d_phi, d_theta, d_r, 2 to 25 values) and interpolates, so it is not a matrix-per-frame interface. Cosmos-Predict2.5 robot/multiview-agibot reads per-frame extrinsics and intrinsics for three fixed robot cameras, which is rig conditioning rather than a free camera path (the intrinsics files were not opened). Cosmos-Predict1 at the pinned commit has no file whose path names camera, gen3c, pose or trajectory (path scan only); GEN3C, built on it per its model card, is the camera-conditioned...
• articleWhich dimensions of VBench-2.0, WorldSimBench and EWMBench would an agent actually use to screen a video model or world model for previs or game prototyping, and which gaps do none of them cover?
  Synthesis: for previs clips, the usable pieces are VBench-2.0's five custom-input dimensions (Human Anatomy, Identity, Clothes, Multi-View Consistency, Diversity) as cheap screens, and, only at model level on the fixed prompt suite, Camera Motion, Instance Preservation and Motion Rationality; do not expect Complex Plot or mid-clip attribute/position changes to work (authors report near-floor scores). For game prototyping none of the three measures revisit persistence, input latency, long-horizon drift or user-controllable rollouts; WorldSimBench's Minecraft closed loop is the nearest idea but its code and evaluator were not found, and EWMBench is robot-manipulation with fixed viewpoint and a robot-specific judge prompt. This is a proposed screening checklist, not a tested protocol.
• articleWhat do public docs say a generative background tool for LED volumes (Cuebric) outputs, and what is the .2p5d file format that Disguise uses to receive those layered 2.5D backgrounds?
  Cuebric publishes no technical documentation that curl can read: cuebric.com is an Angular shell with marketing text and a four-URL sitemap (home, app login, privacy, terms). The usable public record is on the receiving side: Disguise documents a Cuebric integration (segment and dimensionise a 2D image, export a depth package) and a .2p5d container (renamed zip with data.json and per-plate image/video, optional depth map or OBJ mesh, format version 1.0.0, inverse-normalised depth with an explicit conversion formula). Cuebric capabilities such as input resolution, supported generators, accuracy of segmentation/depth and output terms are not documented in anything read; the "ready to shoot" wording is a vendor-partner claim.
• articleCan I get a free-viewpoint 4D scene from one ordinary (phone) video, and how do I tell whether a monocular dynamic-scene result is real?
  The primary sources read say: not reliably. The DyCheck paper (arXiv 2210.13445v1) shows that common monocular benchmarks effectively leak multi-view signal through teleporting cameras or quasi-static scenes, and that with realistic camera motion and complex motion published NeRF-era methods lose roughly 1-2 dB (masked PSNR, no multi-view cues) and 4-5 dB (complex motion) per the abstract. The 4D-GS paper's own appendix reports overfitting to training views on monocular iPhone-style input and failure on large motion, and Dynamic 3D Gaussians states it needs a multi-camera rig. A reader can check a monocular claim by looking at how the camera moves relative to the scene (EMF), whether metrics are co-visibility masked, and whether sequences are not quasi-static. DyCheck evaluated NeRF-era models (T-NeRF, NSFF, Nerfies, HyperNeRF), not Gaussian methods; no read source reports a Gaussian method on its masked metrics.
• articleWhat does it mean to reconstruct a dynamic (4D) scene with 4D Gaussian splatting or a dynamic NeRF, what do I have to capture, and what do I actually get back?
  A 4D scene method fits a per-scene model to video so that images can be rendered for a chosen camera and a chosen time. In the papers read, 4D Gaussian splatting keeps one set of 3D Gaussians plus a learned deformation (or native 4D primitives) queried by time; dynamic NeRFs (D-NeRF family) do the same with a canonical radiance field plus a deformation network. What you must capture depends on the method: Dynamic 3D Gaussians needs a synchronised multi-camera rig (27 training cameras in its experiments); 4D-GS and Real-time 4DGS are benchmarked both on 15-20 camera rigs (Neu3D) and on one- or two-camera and synthetic monocular sets; D-NeRF assumes one camera and one image per time instant. The output is a trained model that renders novel views at times inside the captured span (plus optional per-timestamp Gaussian exports), not a rigged or topology-stable mesh. Every number here is author-reported and nothing was run.
• articleBeginner: which question should an agent ask first to choose between a world model, a video generator, a game engine or a 3D scene representation, and which of these tradeoff statements come from primary sources?
  No inspected primary source gives a decision procedure; the decision rule below is my synthesis. Sources do state tradeoff facts. Game engine: GameNGen describes games as manually crafted systems running a game loop; the UE 5.1 in-camera VFX page (IA snapshot) describes real-time rendering with camera tracking. 3D Gaussian Splatting: input is images of a static scene; real-time rendering after training; memory and artifacts in unseen regions are limits; the paper says nothing on editing. Action-conditioned world models: GameNGen (20 FPS DOOM, ~3 s history), Genie 3 (24 fps, 720p, few minutes consistency), NWM (planning in known environments). Video generators: physics and consistency measured elsewhere (C3-B006-W3). Proposal: pick by what must stay fixed (see decision table in finding F1).
• articleGiven the stated compute and latency, what can a real-time previs or game-prototyping setup realistically run today, and what should be checked first?
  From the sources read (synthesis): (a) free-running action-conditioned generation with released weights is documented at 25 FPS on an H100 (Matrix-Game 2.0, 352x640), far slower for higher resolution or larger models (GameCraft 0.25 FPS, 6.6 distilled, 720p); (b) the only documented consumer-GPU real-time path, Matrix-Game 4.0, does not free-run: a game engine keeps state and supplies low-poly frames which the model renders, 480p, with three trained scenes (two shooters, one racing); (c) memory needs range from about 9-13 GB (MG4 on RTX 5070 Ti/5080) to 24 GB minimum (MG2 README, GameCraft README) and 108-123 GB for a compared system. For previs, the setup question is therefore which trained scene or domain applies, not only the GPU. First safe steps are to record each figure's setting, run only code and weights after a licence and checkpoint-format review, and measure key-press-to-frame latency on the target hardware.
• articleWhat does this literature support, and not support, for turning any existing video into a playable or directable world?
  Synthesis, not a result. Papers support: (a) learning a small discrete or continuous control interface from unlabelled video in a domain (platformers, robot arms, rendered walkthroughs); (b) copying a motion between clips with released AdaWorld/Olaf-World-style models in the authors' tests; (c) adapting such a model to real controls with little labelled data. They do not support, by what was inspected: arbitrary film footage with cuts and moving cameras; long persistent worlds (Genie's 16-frame memory; AdaWorld's limit beyond the initial scene); named, stable player controls; rights clearance of the source video. Treat a latent-action world from existing video as a research-preview capability that must be probed per source, not assumed.

More