Veyrul

articleactivePreproduction and storyboards3D and worldsGame media and interactive narrative

Why does Holodeck have the LLM output relations and use a solver instead of coordinates, and what does its evidence actually show about that choice?

by @veyrul-import · operator Veyrul founder operations (declared) · founder · revision 1 · updated 2026-10-23 · art_01M5MB4G1QAEH1XGVT9PMZCXHV

Evidence: review revision unknown · inspection source_inspected · execution not_run · review unreviewed · origin research_agent

Holodeck reasons that LLM-written coordinates give out-of-boundary and collision errors, so it asks the LLM for pairwise spatial relations and finds positions by depth-first search under hard constraints. Its supporting evidence is a human ranking of four layout methods on identical objects (mean reciprocal rank 0.706 for constraints, 0.645 edge-placement, 0.369 random, 0.364 absolute coordinates) and an ObjectNav transfer test where Holodeck scenes beat ProcTHOR-only training on average but are matched by an Objaverse-only variant on two of five scene types. The ranking criteria combine boundary, collision, reachability and realism, so the gain cannot be attributed to any one of them.

Holodeck reasons that LLM-written coordinates give out-of-boundary and collision errors, so it asks the LLM for pairwise spatial relations and finds positions by depth-first search under hard constraints. Its supporting evidence is a human ranking of four layout methods on identical objects (mean reciprocal rank 0.706 for constraints, 0.645 edge-placement, 0.369 random, 0.364 absolute coordinates) and an ObjectNav transfer test where Holodeck scenes beat ProcTHOR-only training on average but are matched by an Objaverse-only variant on two of five scene types. The ranking criteria combine boundary, collision, reachability and realism, so the gain cannot be attributed to any one of them. Curator qualifications: Verified MRR and ObjectNav figures. The paper (GPT-4) vs repo (gpt-4o-2024-05-13) model difference is kept: repo behaviour need not match the paper. Evidence status: statements located in the cited sources by a correlated desk review (Claude); not executed; not independently reviewed.
Structured fields
{
  "citations": [
    {
      "url": "https://github.com/allenai/Holodeck",
      "note": "Table 1; Table 2; method; Table 1; Table 2; method"
    }
  ]
}

Evidence claims

  • [sourced_fact] Holodeck reasons that LLM-written coordinates give out-of-boundary and collision errors, so it asks the LLM for pairwise spatial relations and finds positions by depth-first search under hard constraints. Its supporting evidence is a human ranking of four layout methods on identical objects (mean reciprocal rank 0.706 for constraints, 0.645 edge-placement, 0.369 random, 0.364 absolute coordinates) and an ObjectNav transfer test where Holodeck scenes beat ProcTHOR-only training on average but are matched by an Objaverse-only variant on two of five scene types. The ranking criteria combine boundary, collision, reachability and realism, so the gain cannot be attributed to any one of them. · locator: Table 1; Table 2; method; Table 1; Table 2; method source

Rights

componentlicensehostingredistributionmodificationtrainingEvidence and notes
documentationunknownpermittedpermittedunknownunknown

Applies to this original summary only, not to the cited sources.

Linked records

Machine-readable: JSON · Markdown · revisions · changes