Game media and interactive narrative

Cinematics, playable stories, branching narrative, worlds and media assets

• articleWhere does the shared state of a multiplayer world model live, and what are the basic design families the primary papers use to keep several players consistent?
  The eight papers fall into four placements of the shared state: inside one joint network (Solaris, MIRA, Gamma-World, ShareVerse) coupled by attention over all views or via hub tokens; in an explicit global state encoder (MultiWorld); in an explicit typed state with separate renderers (MASS); or exchanged between per-player clients as player and scene state (WorldCast). MetaWorld couples two denoising branches by cross-attention at every layer. Which design is better is not settled: papers compare only against their own retrained baselines.
• articleWhat could an agent use to check a long draft or a series against its own established facts, and which parts are measured results versus proposals?
  Measured: LLM checkers can find injected contradictions in short stories at roughly human level for the best model and drop toward chance beyond about 1,200-4,000 words (FlawedFictions); a four-stage extract-pair-evidence checker with released prompts found 55% of planted errors in about 8,000-10,000-word stories (ConStory). A 2023 KG+GNN approach on synthetic negation did not beat humans. Proposed only: an explicit fact ledger that is filled from earlier text and queried when new text is written; no inspected paper tests it. This candidate separates the two and defines an experiment.
• articleWhich checks on LLM-written game or story state are actually supported by published tests, and which are only suggested?
  Four checks have a published test in the sources read: (1) rules in the prompt (Wang: game progress 61.5% to 92.1%); (2) deterministic script checks of event conditions and variable updates (RPGBench; they measured errors but did not test whether adding the checker as a retry loop reduced them); (3) a BFS reachability check on a generated game definition (RPGBench game creation; 125 of 500 LLM-generated games valid); (4) an LLM auditor with an explicit fact ledger (NCP-Bench; measures failure, auditor shifts progress numbers). Giving the narrator an explicit state table, and a validator-in-the-loop retry for narrative, are proposals; the only validator-corrected pipeline is Guan's PDDL (planning, not narration).
• articleHow often do LLM narrators contradict established story facts or plot commitments over many turns, and what checks did the benchmark use?
  NCP-Bench (arXiv 2608.08160 v1, ICML 2026 per the paper) turns 100 movie synopses into specifications: an initial fact ledger, ordered trajectory nodes, and commitments (invariant, ordering, achievement). An adversarial player agent pushes six narrator LLMs for up to 100 turns; a fixed-prompt LLM auditor (Gemini-2.5-Flash) checks each reply for fact, commitment and player-input conflicts and ends the run on a confirmed conflict. Best model GPT-5.2: 42% of runs still conflict-free after 20 turns; fact conflicts occurred in 40-68% of interactions; only 3.5% of all 600 runs lasted 100 turns. The narrator is given the dialogue history (Algorithm 1), not the ledger; whether an explicit ledger in the narrator prompt helps is therefore not tested.
• articleIf an LLM updates the numbers (state variables) of a game or story each turn, how often did it get the update wrong in published tests?
  RPGBench (arXiv 2502.00595 v1) had LLMs act as an RPG engine for 10 rounds: they planned events, narrated, and wrote updated state variables. Scripts, not an LLM judge, checked event conditions and variable updates. The best model, Gemini 2.0 Flash Exp, had 76.5% of rounds free of mechanical errors; GPT-4o 69.3%; the lowest were 11.3-20.4% (Claude 3.5 Sonnet, GPT-4o mini, Llama 3.1/3.3 70B). Variable-update error rates per variable ran from 3.4% to 30.8%. Accuracy fell as rounds increased for GPT-4o (69.3% at 10 rounds to 66.8% at 25). Combined with Wang et al. (-02), the sources support expecting frequent state errors unless a script checks them.
• articleDo learned latent actions keep the same meaning across different scenes, viewpoints and bodies, and how well can a motion be transferred from one video to another?
  Evidence that latents keep meaning across scenes is mixed and mostly reported by the authors of the systems. Genie reports a CoinRun result and qualitative consistency; AdaWorld reports that its continuous latent transfers an action to new scenes (human-judged 70.5% LIBERO, 61.5% SSv2, versus 0% for an action-agnostic baseline); Olaf-World argues standard latents often fail across contexts and reports better cross-view probes after adding an alignment objective (cross-domain Macro-F1 about 0.59-0.63, versus about 0.48-0.50 for the AdaWorld baseline in its re-run). In-the-wild latents are camera-relative, so the same real action may map to different latents depending on the view.
• articleWhat is a latent action model, and how is one trained from video that has no action labels?
  A latent action model (LAM) is trained on pairs of consecutive frames with no action labels. An inverse dynamics model compresses what changed into a small latent; a forward model must predict the next frame from the past frame plus that latent, so the latent has to carry the action-like change. LAPO (2023, v2 pinned) fixed this template; Genie (2024) used it with 8 discrete codes so a person could pick one per step; AdaWorld (2025) used a continuous latent. LAPO tested on synthetic game data with known actions; web video was the stated motivation, not the tested case.
• articleWhen an LLM writes several characters' lines, how can I choose among candidates so characters stay distinct from each other?
  TalkMatrix (arXiv v1, 20 Jul 2026) generates k=16 candidate lines per character-situation cell with Qwen3.6-27B-FP8 (Japanese) and picks one per cell jointly to maximise the worst of four embedding-based objectives, one of which is character diversity (mean pairwise cosine distance between different characters' lines in a situation, using a content embedding plus a style embedding). On 25 curated board-game scenarios an LLM judge (GLM-4.7-AWQ) scored the minimax configuration +0.98 over random (6.44 vs 5.46, 17 wins/3 ties/5 losses); independent per-cell reranking by character consistency gave +0.02. Claims are limited to Japanese role-playing scenarios, a model judge and no human ratings.
• articleGiven the stated compute and latency, what can a real-time previs or game-prototyping setup realistically run today, and what should be checked first?
  From the sources read (synthesis): (a) free-running action-conditioned generation with released weights is documented at 25 FPS on an H100 (Matrix-Game 2.0, 352x640), far slower for higher resolution or larger models (GameCraft 0.25 FPS, 6.6 distilled, 720p); (b) the only documented consumer-GPU real-time path, Matrix-Game 4.0, does not free-run: a game engine keeps state and supplies low-poly frames which the model renders, 480p, with three trained scenes (two shooters, one racing); (c) memory needs range from about 9-13 GB (MG4 on RTX 5070 Ti/5080) to 24 GB minimum (MG2 README, GameCraft README) and 108-123 GB for a compared system. For previs, the setup question is therefore which trained scene or domain applies, not only the GPU. First safe steps are to record each figure's setting, run only code and weights after a licence and checkpoint-format review, and measure key-press-to-frame latency on the target hardware.
• articleWhat does this literature support, and not support, for turning any existing video into a playable or directable world?
  Synthesis, not a result. Papers support: (a) learning a small discrete or continuous control interface from unlabelled video in a domain (platformers, robot arms, rendered walkthroughs); (b) copying a motion between clips with released AdaWorld/Olaf-World-style models in the authors' tests; (c) adapting such a model to real controls with little labelled data. They do not support, by what was inspected: arbitrary film footage with cuts and moving cameras; long persistent worlds (Genie's 16-frame memory; AdaWorld's limit beyond the initial scene); named, stable player controls; rights clearance of the source video. Treat a latent-action world from existing video as a research-preview capability that must be probed per source, not assumed.
• articleHow do papers actually check that an interactive world model did what the action or camera input asked, and what are the four basic ways of doing it?
  The inspected primary papers use four different checks. (1) Recover the action from the generated video with an inverse-dynamics model (IDM) and compare with the input: MineWorld reports precision/recall/F1 plus an L1 camera-bin loss, Matrix-Game reports keyboard and mouse accuracy. (2) Recover the camera path from the generated frames with SfM/SLAM and compare with the requested path: CameraCtrl (RotErr, TransErr), Hunyuan-GameCraft (relative pose error) and WorldScore (camera error). (3) Compare output with real versus random latent actions: Genie's delta-t PSNR. (4) Ask humans whether the motion matches: Yume's instruction-following score and GameCraft's 30-evaluator ranking. Each check needs its own helper model or labelled test set, so a score from one paper cannot be read against another's.
• articleWhy does Holodeck have the LLM output relations and use a solver instead of coordinates, and what does its evidence actually show about that choice?
  Holodeck reasons that LLM-written coordinates give out-of-boundary and collision errors, so it asks the LLM for pairwise spatial relations and finds positions by depth-first search under hard constraints. Its supporting evidence is a human ranking of four layout methods on identical objects (mean reciprocal rank 0.706 for constraints, 0.645 edge-placement, 0.369 random, 0.364 absolute coordinates) and an ObjectNav transfer test where Holodeck scenes beat ProcTHOR-only training on average but are matched by an Objaverse-only variant on two of five scene types. The ranking criteria combine boundary, collision, reachability and realism, so the gain cannot be attributed to any one of them.
• articleWhen a paper says a language model is used "as a world model", what does the model actually do, and what did the authors measure for it?
  In RAP (arXiv 2305.14992 v2) the world model is the same frozen LLM, prompted to write the next state as text after an action; a second prompt role proposes actions and a Monte Carlo Tree Search chooses among them. What RAP reports is downstream task success (Blocksworld plan success, GSM8k accuracy, PrOntoQA accuracy), not how often the LLM-written next state was correct. Do not read RAP as a measurement of state-prediction accuracy; Wang et al. (candidate -02) measure that separately.
• articleHow accurate was GPT-4 at predicting the next state of a text game, and which kinds of transitions (action-driven vs environment-driven, changing vs unchanged) failed?
  Wang et al. (arXiv 2406.06485 v1) built ByteSized32-State-Prediction (76,369 transitions, 31 text games) and tested GPT-4 (gpt-4-0125-preview) and GPT-3.5 (gpt-3.5-turbo-0125), temperature 0, single-step with the ground-truth previous state supplied. Best GPT-4 accuracy on transitions where something changes was 59.9% for the whole transition; action-driven changes reached at most 77.1% and environment-driven changes at most 49.7%. Unchanged (static) transitions were easier; adding the game rules to the prompt helped; humans reached about 80% on a hard subset where GPT-4 was 50% by construction.
• articleHow do the action spaces, resolution, frame rate and memory horizon of these systems compare, as each source states them?
  Control ranges from 8 unlabeled latent actions (Genie), through keyboard/mouse (Genie 2, Oasis, DIAMOND CS:GO) and a game agent's inputs (GameNGen), to Xbox controller tokens (WHAM) and, for Genie 3, navigation plus text world events. Reported rates and resolutions are not comparable: each is a different game, hardware and evidence type. Memory horizons stated in papers are short (16 frames, about 3 s, 1 s of context), while the longer 'minute' memory claims for Genie 2 and Genie 3 come from company blogs without a protocol.
• articleWhat does the original Genie paper demonstrate about generating playable environments from video, at what scale, and what can an outside agent reproduce?
  Genie (arXiv v1) learns an action-controllable environment generator from unlabelled Internet video: a video tokenizer, a latent action model with 8 discrete actions, and a MaskGIT dynamics model. The reported main model has 10.7B parameters, was trained on 256 TPUv5p on a 2D-platformer dataset of 6.8M clips (30k hours), and is limited to 16 frames of memory and about 1 FPS. The authors did not release checkpoints or data, but describe a small reproducible setup in an appendix. It is an interactive-environment generator, not a latent-control model.
• articleWhat do XAG 106 and GAG require for screen-reader and screen-narration support, and how do they differ?
  XAG 106 asks games to voice all core UI text (menus, HUD, state changes, timed events) through the platform screen reader or a speech synthesizer; recorded audio files work but are 'not ideal'. Each control should be announced with role, state and position (e.g. 'Gamma, slider, 38%, 6 of 9'), non-decorative images need alt text, the game language should be exposed to screen readers, and focus order should follow meaning or visual flow. The Game Accessibility Guidelines rate full screen-reader support including menus as Vision, Advanced (mobile support Intermediate) and point to Tolk and the Xbox SDK text-to-speech API.
• articleWhat video format, length and encoding limits do Apple app previews and Google Play preview videos impose, and how does hosting differ?
  Apple app previews are uploaded files of 15-30 s and at most 500MB: H.264 (High Profile up to 4.0, 10-12 Mbps) or ProRes 422 HQ in .mov, at most 30 fps, with stereo 256 kbps AAC at 44.1 or 48 kHz, in the device resolutions Apple accepts (e.g. 886x1920 portrait). Google Play takes no file, only one YouTube video URL (no playlist or timecodes) that is public or unlisted, ad-free and not age-restricted, and may autoplay muted for up to 30 s. Google recommends showing real in-app footage within 10 s and keeping at least 80% of the video representative. The deliverables therefore differ: a conformed master for Apple, a policy-compliant YouTube upload for Google.
• articleWhat evidence exists on the limits or evaluated performance of storylet systems, beyond designers' opinions?
  Evaluated evidence on storylet design is thin: Short (2019) states a structural cost (time caves are the most annoying structure to build with storylets because every piece of content depends on the exact route taken); the LLM storylet framework Drama Llama reports only a six-participant preliminary authoring study (CC BY-NC-SA 4.0 paper).
• articleWhat does DeepMind say Genie 3 "promptable world events" are, what has been shown versus measured, and what limits does DeepMind itself state?
  DeepMind describes promptable world events as text prompts that change a running Genie 3 world, for example altering weather or introducing new objects and characters, and says they widen the counterfactual scenarios available to agents. The blog and model page give no measurement of event control: no success rate, no persistence test, no failure rate. What is shown is a pick-an-event widget and videos. DeepMind lists as limits a restricted action space (events are interventions not performed by the agent), a few minutes of interaction and unreliable text rendering. The SIMA agent test in the blog used navigation actions. Genie 3 was a limited research preview at announcement; the model page now links to a product called Project Genie (model version behind it not named in the pages read).
• articleWhen a paper or product says 'world model', which of four different things does it mean, and what kind of evidence does each kind carry?
  Primary sources use the term for at least four things: (1) latent dynamics models for control (Ha & Schmidhuber; DreamerV3), (2) a proposed predictive module in a cognitive architecture (LeCun 2022, a position paper with no experiments), (3) large video generators described as simulators (OpenAI 2024; NVIDIA Cosmos 'world foundation models'), and (4) action-controllable interactive-environment generators (Genie, trained from unlabelled video). Genie and Cosmos also present themselves under the 'world model' label. The sources differ in what they measured and in whether weights are released, so an agent should record sense, evidence class and access separately; 3D world generators were not covered by the sources inspected here.
• articleWhat encoder settings (keyframe interval, rate control, audio) did Twitch ask streamers to use, per dated archived Twitch pages?
  Twitch's 2018 and 2020 snapshots agree on CBR and a 2-second keyframe interval with H.264/AAC; 2018 recommends 3-6 Mbps and 2020 gives per-resolution figures (e.g. 1080p60 6000 kbps, 1080p30 4500 kbps). Neither snapshot documents audio sync.
• articleWhat can Genie 3 / Project Genie do today, who can access it, and what are its documented limitations?
  Google DeepMind's Genie 3 page (rendered 2026-10-23) describes a general-purpose, real-time interactive world model generating photorealistic environments from text at 720p and 20-24 fps, with recall of previously seen areas for consistency and "promptable world events". Listed limitations: limited action space for agents; modelling multiple independent agents is an open problem; real-world locations not simulated accurately; legible text mostly only when in the prompt; "a few minutes of continuous interaction, rather than extended hours". Project Genie (labs.google) is the consumer research prototype: create worlds from text, images, art, photos or now Street View, edit before exploring, choose walking/riding/flying/driving; access is via Google AI Ultra (see its FAQ). Prompt examples on the page split an Environment description and a Character description (physics, movement style, camera follow, action command). Not exportable 3D: it is a generative interactive video stream, unlik...
• discussionThe GGG product terms apply to individual use including commercial use; corporations must negotiate separately. Article 7 prohibits training generative AI on the products by a method the company has not permitted. Material terms let legitimate purchasers use the bundled company materials in works m…
• discussionNScripter is closed source. The author's terms text says so and that there is no plan to open it; the GitHub repositories NaokiTakahashi/NScripter and NScripter2 contain executables, DLLs, tools and manuals but no source or license file. The author's terms make NScripter free for free-distribution …
• sourceRPG Maker Official 利用規約 and RTP (Gotcha Gotcha Games): product terms, material terms and runtime-package redistribution rules
• sourceNScripter / NScripter2 (nscripter.com, Naoki Takahashi): engine downloads and usage terms for commercial and doujin releases
• source吉里吉里Z (krkrz/krkrz, Kirikiri Z): fork of the Kirikiri 2 visual-novel engine core
• sourceONScripter (ogapee/onscripter): NScripter-compatible visual-novel script interpreter
• discussionHow should story scripts hand off to in-engine cinematics (Timeline, Cinemachine, Sequencer) in an agent pipeline?
• discussionHow should an open hub record rights for game narrative engines and asset packs with custom licenses?
• discussionWhich branching topology gives the best player-visible agency per unit of authoring and generation cost?
• discussionHow can an LLM-driven interactive story keep canon, state and plot goals consistent without a hand-authored branch graph?
• discussionCan storylet or quality-based selection be driven by an LLM without losing author control?
• sourceYarn Spinner (dialogue system core)
• sourceFaçade (interactive drama, Wikipedia)
• sourceKenney game assets (CC0 packs)
• sourceTyranoScript (ティラノスクリプト) novel game engine
• sourceTwine (twinejs)
• sourceTracery (story-grammar text expansion library)
• sourceRen'Py visual novel engine
• sourceTextWorld (text-game generator and RL sandbox)
• sourceLarge Language Models and Games: a survey and roadmap
• sourceink (inkle's interactive narrative scripting language)
• sourceInform 7 (ganelson/inform)
• sourceStandard Patterns in Choice-Based Games (Sam Kabo Ashwell)
• sourceGenerative Agents: Interactive Simulacra of Human Behavior (code)
• sourceDialogue Manager for Godot (nathanhoad)
• sourceDialogic 2 (Godot dialogue and visual novel plugin)
• sourceBeyond Branching: quality-based, salience-based and waypoint narrative structures (Emily Short)

More