Which control inputs (ego trajectory, steering/speed, agent boxes, map, weather text, camera layout, robot actions, camera pose) do the driving and robot world models accept, and how is each encoded?
The driving systems expose a small set of structured controls: ego motion (speed + curvature in GAIA-1/2; angle/speed, trajectory, command or goal point in Vista; yaw + velocity in DriveDreamer; 64-point trajectory in Cosmos), other agents (3D boxes in GAIA-2 and DriveDreamer; LLM-written trajectories in DriveDreamer-2), scene attributes (categorical metadata, CLIP or T5 text), and the camera rig (intrinsics/extrinsics in GAIA-2; per-view captions and view embeddings in Cosmos). Robot systems use gripper-frame action vectors (Cosmos on Bridge, 7-D) or whole-body low-level trajectories (1XWM). Camera-pose control with Plucker rays exists in Cosmos but was trained on static real-estate-style scenes. Most control interfaces were trained on proprietary data, and several are demonstrated only in figures.
The driving systems expose a small set of structured controls: ego motion (speed + curvature in GAIA-1/2; angle/speed, trajectory, command or goal point in Vista; yaw + velocity in DriveDreamer; 64-point trajectory in Cosmos), other agents (3D boxes in GAIA-2 and DriveDreamer; LLM-written trajectories in DriveDreamer-2), scene attributes (categorical metadata, CLIP or T5 text), and the camera rig (intrinsics/extrinsics in GAIA-2; per-view captions and view embeddings in Cosmos). Robot systems use gripper-frame action vectors (Cosmos on Bridge, 7-D) or whole-body low-level trajectories (1XWM). Camera-pose control with Plucker rays exists in Cosmos but was trained on static real-estate-style scenes. Most control interfaces were trained on proprietary data, and several are demonstrated only in figures.
Curator qualifications: Driving controls: ego motion, other-agent boxes or trajectories, text; structured and low-dimensional.
Evidence status: statements located in the cited sources by a correlated desk review (Claude); not executed; not independently reviewed.
Structured fields
{
"citations": [
{
"url": "https://arxiv.org/abs/2309.17080v1",
"note": "candidate evidence excerpts"
},
{
"url": "https://arxiv.org/abs/2503.20523v1",
"note": "candidate evidence excerpts"
},
{
"url": "https://arxiv.org/abs/2405.17398v5",
"note": "candidate evidence excerpts"
},
{
"url": "https://arxiv.org/abs/2309.09777v2",
"note": "candidate evidence excerpts"
},
{
"url": "https://arxiv.org/abs/2403.06845v2",
"note": "candidate evidence excerpts"
},
{
"url": "https://arxiv.org/abs/2501.03575v3",
"note": "candidate evidence excerpts"
},
{
"url": "https://arxiv.org/abs/2511.00062v2",
"note": "candidate evidence excerpts"
},
{
"url": "https://www.1x.tech/discover/redwood-ai-world-model",
"note": "candidate evidence excerpts"
},
{
"url": "https://wayve.ai/thinking/gaia-3/",
"note": "candidate evidence excerpts"
},
{
"url": "https://wayve.ai/thinking/gaia-4/",
"note": "candidate evidence excerpts"
}
]
}Evidence claims
- [sourced_fact] The driving systems expose a small set of structured controls: ego motion (speed + curvature in GAIA-1/2; angle/speed, trajectory, command or goal point in Vista; yaw + velocity in DriveDreamer; 64-point trajectory in Cosmos), other agents (3D boxes in GAIA-2 and DriveDreamer; LLM-written trajectories in DriveDreamer-2), scene attributes (categorical metadata, CLIP or T5 text), and the camera rig (intrinsics/extrinsics in GAIA-2; per-view captions and view embeddings in Cosmos). Robot systems use gripper-frame action vectors (Cosmos on Bridge, 7-D) or whole-body low-level trajectories (1XWM). Camera-pose control with Plucker rays exists in Cosmos but was trained on static real-estate-style scenes. Most control interfaces were trained on proprietary data, and several are demonstrated only in figures. · locator: candidate evidence excerpts source
Rights
| component | license | hosting | redistribution | modification | training | Evidence and notes |
|---|---|---|---|---|---|---|
| documentation | unknown | permitted | permitted | unknown | unknown | Applies to this original summary only, not to the cited sources. |
Linked records
- cites → https://arxiv.org/abs/2309.09777v2
- cites → https://arxiv.org/abs/2309.17080v1
- cites → https://arxiv.org/abs/2403.06845v2
- cites → https://arxiv.org/abs/2405.17398v5
- cites → https://arxiv.org/abs/2501.03575v3
- cites → https://arxiv.org/abs/2503.20523v1
- cites → https://arxiv.org/abs/2511.00062v2
- cites → https://wayve.ai/thinking/gaia-3
- cites → https://wayve.ai/thinking/gaia-4
- cites → https://www.1x.tech/discover/redwood-ai-world-model