How was Meta's Navigation World Model used to plan or rank navigation trajectories, with what inputs, outputs, limits and licence?
NWM is a 1B-parameter Conditional Diffusion Transformer predicting the next egocentric image from past frames (context 4) and 3-DoF navigation actions, trained on SCAND, TartanDrive, RECON, HuRoN plus unlabeled Ego4D. Planning: Cross-Entropy Method over 8-step trajectories minimising LPIPS between the final predicted image and a goal image; ranking: 16 or 32 NoMaD samples scored the same way. On RECON NWM planning reaches ATE 1.13 / RPE 0.35 vs NoMaD 1.93 / 0.52; ranking gives 1.83 and 1.78 ATE. Constraints (forward-first etc.) are enforced by zeroing actions. Authors report mode collapse and loss of context out of distribution, trouble with pedestrian motion, 3-DoF only; real-time operation is a proposal (time-skip, distillation), not demonstrated.
Structured fields
{
"citations": [
{
"url": "https://arxiv.org/abs/2412.03572v2",
"note": "nwm README.md at 3f6cd8e7, line 211-212; nwm LICENSE.md at 3f6cd8e7 (gh api -X GET contents listing shows LICENSE.md at repo root)"
},
{
"url": "https://github.com/facebookresearch/nwm/blob/3f6cd8e70d6f2d1e2b9684acff510710135f0f41/README.md",
"note": "nwm README.md at 3f6cd8e7, line 211-212; nwm LICENSE.md at 3f6cd8e7 (gh api -X GET contents listing shows LICENSE.md at repo root)"
}
]
}Evidence claims
- [sourced_fact] NWM is a 1B-parameter Conditional Diffusion Transformer predicting the next egocentric image from past frames (context 4) and 3-DoF navigation actions, trained on SCAND, TartanDrive, RECON, HuRoN plus unlabeled Ego4D. Planning: Cross-Entropy Method over 8-step trajectories minimising LPIPS between the final predicted image and a goal image; ranking: 16 or 32 NoMaD samples scored the same way. On RECON NWM planning reaches ATE 1.13 / RPE 0.35 vs NoMaD 1.93 / 0.52; ranking gives 1.83 and 1.78 ATE. Constraints (forward-first etc.) are enforced by zeroing actions. Authors report mode collapse and loss of context out of distribution, trouble with pedestrian motion, 3-DoF only; real-time operation is a proposal (time-skip, distillation), not demonstrated. · locator: nwm README.md at 3f6cd8e7, line 211-212; nwm LICENSE.md at 3f6cd8e7 (gh api -X GET contents listing shows LICENSE.md at repo root) source
Rights
| component | license | hosting | redistribution | modification | training | Evidence and notes |
|---|---|---|---|---|---|---|
| documentation | unknown | permitted | permitted | unknown | unknown | Applies to this original summary only, not to the cited sources. |