What do the learned latent actions actually correspond to, and how can I find out what each one does?
What a latent corresponds to is an empirical question per dataset. LAPO found, on synthetic Procgen data with known actions, clusters that match the true actions in many games and fragments in games with partial observability. Genie authors report consistent latent meanings (for example down, up, left on robot video) by inspection of generated frames. On diverse in-the-wild video, a later study reports actions that are camera-relative and localised in space, not tied to one body. None of these gives a naming for latent codes in advance; you find out by trying each code.
What a latent corresponds to is an empirical question per dataset. LAPO found, on synthetic Procgen data with known actions, clusters that match the true actions in many games and fragments in games with partial observability. Genie authors report consistent latent meanings (for example down, up, left on robot video) by inspection of generated frames. On diverse in-the-wild video, a later study reports actions that are camera-relative and localised in space, not tied to one body. None of these gives a naming for latent codes in advance; you find out by trying each code.
Curator qualifications: Verified both statements; the Genie 'consistent' vs LAPO 'not necessarily state-invariant' tension is correctly framed as different data and claims.
Evidence status: statements located in the cited sources by a correlated desk review (Claude); not executed; not independently reviewed.
Structured fields
{
"citations": [
{
"url": "https://arxiv.org/abs/2312.10812v2",
"note": "method; conclusion; Sec. on decoders"
},
{
"url": "https://arxiv.org/abs/2402.15391v1",
"note": "method; conclusion; Sec. on decoders"
},
{
"url": "https://arxiv.org/abs/2601.05230v2",
"note": "method; conclusion; Sec. on decoders"
}
]
}Evidence claims
- [sourced_fact] What a latent corresponds to is an empirical question per dataset. LAPO found, on synthetic Procgen data with known actions, clusters that match the true actions in many games and fragments in games with partial observability. Genie authors report consistent latent meanings (for example down, up, left on robot video) by inspection of generated frames. On diverse in-the-wild video, a later study reports actions that are camera-relative and localised in space, not tied to one body. None of these gives a naming for latent codes in advance; you find out by trying each code. · locator: method; conclusion; Sec. on decoders source
Rights
| component | license | hosting | redistribution | modification | training | Evidence and notes |
|---|---|---|---|---|---|---|
| documentation | unknown | permitted | permitted | unknown | unknown | Applies to this original summary only, not to the cited sources. |