Exactly how does the Matrix-Game GameWorld Score measure keyboard and mouse following, what do Oasis, MineWorld and Matrix-Game 1.0/2.0 score on it, and what does it need to run?

GameWorld Score (Matrix-Game 1.0 paper and the GameWorldScore directory at commit c7a88bf) scores eight dimensions; the action ones come from an IDM trained on Minecraft (the paper says 90.6% keyboard accuracy, R2 0.97 for mouse) applied to generated clips. The code asks for 76 actions on 32 initial images, i.e. 2432 videos. Paper results (author-run): keyboard/mouse accuracy 0.77/0.56 Oasis, 0.86/0.64 MineWorld, 0.95/0.95 Matrix-Game 1.0. Matrix-Game 2.0 reuses the score on Minecraft (0.91/0.95) but says the action metric cannot be applied to wild scenes, so its comparison against YUME omits action accuracy. The benchmark downloads several checkpoints in pickle-capable formats and the repository carries conflicting licence signals. Curator qualifications: v1: IDM 'achieves 90.6% accuracy on keyboard prediction and an R2 score of 0.97 for mouse'; Table 2 Oasis 0.65/0.48/0.94/0.98/0.77/0.56/0.56/0.86, MineWorld 0.69/0.47/0.95/0.98/0.86/0.64/0.51/0.92, Ours 0.72/0.49/0.97/0.98/0.95/0.95/0.76/0.93. 2.0 Table 1: Oasis 0.27/0.27/0.82/0.99/0.73/0.56/0.18/0.84; Ours 0.61/0.50/0.94/0.98/0.91/0.95/0.64/0.80. 2.0 text: 'Since the action controllability assessment in GameWorld Score Benchmark is designed specifically for Minecraft evaluation, it cannot be directly applied to wild scenes.' Addition: 2.0 states its own protocol, 'All experiments utilize a 597-frame composite action sequence, with evaluation performed on 32 Minecraft scenes and 16 diverse' wild scenes, and says Oasis 'exhibits significant quality degradation after several dozen frames'; the long-sequence protocol plausibly explains the lower Oasis values (curator inference), so the candidate's 'the 2.0 text read does not say how' should be softened. Rights/verification notes are shown as limitations (L00 0019/ADR-0011): they restrict reuse of the cited material or record what was not verified; they are not cleared by this summary. Limitations: [component_reuse_restriction] eligible_dated_with_rights_hold Evidence status: statements located in the cited sources by a correlated desk review (Claude); not executed; not independently reviewed.

{
  "citations": [
    {
      "url": "https://arxiv.org/abs/2506.18701v1",
      "note": "Matrix-Game v1 arXiv 2506.18701v1 PDF (equals pin) -layout: line 815 (IDM accuracy), Table 2 lines 850-862; Matrix-Game 2.0 arXiv HTML v2 (equals pin): Sec. 5.1 settings, Table 1, Sec. 5.2 Table 2 text; licence files as in C3R-0125"
    },
    {
      "url": "https://github.com/SkyworkAI/Matrix-Game/tree/c7a88bfc039cd97695082484986e95b962390676/Matrix-Game-1/GameWorldScore",
      "note": "Matrix-Game v1 arXiv 2506.18701v1 PDF (equals pin) -layout: line 815 (IDM accuracy), Table 2 lines 850-862; Matrix-Game 2.0 arXiv HTML v2 (equals pin): Sec. 5.1 settings, Table 1, Sec. 5.2 Table 2 text; licence files as in C3R-0125"
    },
    {
      "url": "https://arxiv.org/abs/2508.13009v2",
      "note": "Matrix-Game v1 arXiv 2506.18701v1 PDF (equals pin) -layout: line 815 (IDM accuracy), Table 2 lines 850-862; Matrix-Game 2.0 arXiv HTML v2 (equals pin): Sec. 5.1 settings, Table 1, Sec. 5.2 Table 2 text; licence files as in C3R-0125"
    }
  ]
}