How does WorldRoamBench test whether an interactive world model remembers a place after revisit, why does it avoid frame-pair matching, and what did it find about memory decay?
Problem: an action-conditioned model may not return to the same viewpoint, so comparing the 'return' frame with the first frame confuses memory with action-following error. Conditions: WASD and IJKL interaction of 10 to 60 seconds in Nature, Urban and Indoor scenes, first and third person; v5 abstract states 1000+ test cases and 10+ open and closed models. Method: four dimensions (per-frame action, sliding-window visual drift, physics gated by camera-motion and tracking checks, memory). Scene memory splits the rollout at the executed observation-revisit transition, reconstructs a point cloud of each segment, aligns the two, and normalises by reconstruction sampling density, since two passes over the same area are never point-wise identical. Subject memory uses tracking plus a VLM. Its authors argue trajectory metrics (RPE, ATE) are unfair across models and can hide per-frame failures. Findings (as reported): models regenerate a different scene on a symmetric return path; memory rank...
Problem: an action-conditioned model may not return to the same viewpoint, so comparing the 'return' frame with the first frame confuses memory with action-following error. Conditions: WASD and IJKL interaction of 10 to 60 seconds in Nature, Urban and Indoor scenes, first and third person; v5 abstract states 1000+ test cases and 10+ open and closed models. Method: four dimensions (per-frame action, sliding-window visual drift, physics gated by camera-motion and tracking checks, memory). Scene memory splits the rollout at the executed observation-revisit transition, reconstructs a point cloud of each segment, aligns the two, and normalises by reconstruction sampling density, since two passes over the same area are never point-wise identical. Subject memory uses tracking plus a VLM. Its authors argue trajectory metrics (RPE, ATE) are unfair across models and can hide per-frame failures. Findings (as reported): models regenerate a different scene on a symmetric return path; memory rankings are unstable across horizons (11 of 12 models change rank in Memory from short to long, against 5 in Overall), so early memory scores do not predict long-horizon memory. Limitations: VLM judges are used for subject memory and physics (the paper includes a human verification section I did not read); point-cloud reconstruction quality bounds the score. Version and date: arXiv 2606.31672 v5, 2026-09-18 (title also shown as WorldOdysseyBench in some listings). This extends the throughput-only use of the benchmark in C3-B225-W40-03 and does not rank models.
Curator qualifications: All cited documentation excerpts located on the live pages by curator GET; no video sources. Correction folded 2026-10-24 from C3-B809-03 (curator-reviewed, C3R-0613/0616): Problem. C3-B756-07 adds an aside that the WorldRoamBench paper is "also shown as WorldOdysseyBench in some listings". Method. The lead re-fetched the cited arXiv abstract and HTML pages for 2606.31672 on 2026-10-24 at about 00:20Z: the title is "WorldRoamBench: An Open-World Benchmark for Long-Horizon Stability of Interactive World Models" and the string "WorldOdyssey" does not appear on either page. Treat the alternative name as unsupported; the rest of C3-B756-07 was graded supported by the C3-B762 audit worker, who flagged this aside.
Limitations: Read only the abstract page and the HTML text (math garbled); I did not read the human-verification appendix or the per-model tables. Case counts differ between versions (v5 says 1000+; a search-result summary of an earlier version said 600+). The ranking observation is the authors' own. Corrected by C3-B809-03 (2026-10-24); where the correction disagrees, the correction governs.
Evidence status: statements located in the cited sources by a correlated desk review (Claude); not executed; not independently reviewed.
Structured fields
{
"citations": [
{
"url": "https://arxiv.org/abs/2606.31672",
"note": "Abstract; Introduction; Appendix H scene memory; Memory results; Introduction"
},
{
"url": "https://arxiv.org/html/2606.31672",
"note": "Abstract; Introduction; Appendix H scene memory; Memory results; Introduction"
}
]
}
Evidence claims
[sourced_fact] Problem: an action-conditioned model may not return to the same viewpoint, so comparing the 'return' frame with the first frame confuses memory with action-following error. Conditions: WASD and IJKL interaction of 10 to 60 seconds in Nature, Urban and Indoor scenes, first and third person; v5 abstract states 1000+ test cases and 10+ open and closed models. Method: four dimensions (per-frame action, sliding-window visual drift, physics gated by camera-motion and tracking checks, memory). Scene memory splits the rollout at the executed observation-revisit transition, reconstructs a point cloud of each segment, aligns the two, and normalises by reconstruction sampling density, since two passes over the same area are never point-wise identical. Subject memory uses tracking plus a VLM. Its authors argue trajectory metrics (RPE, ATE) are unfair across models and can hide per-frame failures. Findings (as reported): models regenerate a different scene on a symmetric return path; memory rankings are unstable across horizons (11 of 12 models change rank in Memory from short to long, against 5 in Overall), so early memory scores do not predict long-horizon memory. Limitations: VLM judges are used for subject memory and physics (the paper includes a human verification section I did not read); point-cloud reconstruction quality bounds the score. Version and date: arXiv 2606.31672 v5, 2026-09-18 (title also shown as WorldOdysseyBench in some listings). This extends the throughput-only use of the benchmark in C3-B225-W40-03 and does not rank models. · locator: Abstract; Introduction; Appendix H scene memory; Memory results; Introductionsource
Rights
component
license
hosting
redistribution
modification
training
Evidence and notes
documentation
unknown
permitted
permitted
unknown
unknown
Applies to this original summary only, not to the cited sources.