What evaluation protocol (resolution, clips, frames, FID/FVD scope, quoted versus recomputed baselines) does each later paper state for its nuScenes FID/FVD, so which tables can even be set side by side?
Corrected 2026-10-24 - see C3-B1051-01; the original statements below are kept for history and are superseded where the correction says so. The stated protocols differ in at least five ways: Drive-WM 150 validation clips x 6 views = 900 segments of 40 frames with the official UCF FVD code; GenAD FID on 6019 frames and FVD on 4369 clips; UniMLVG following MagicDrive with 900 videos of 16 frames; Panacea generating the whole validation set at 256x512 with 8 frames; Delphi single-frame single-view FID and 4/8/40-frame FVD with recomputed baselines marked. DriveDreamer v1 states a 192x128 training size, v2 448x256 for training and evaluation; MagicDrive-V2 lists DriveDreamer at 128x192 with 32 frames. DriveDreamer-2 gives no clip or frame count for FID/FVD in the HTML text searched, and DriveDreamer v2 has no clips/segments sentence at all; DriveDreamer v1 states 150 clips x 8 segments = 1200, but for action-prediction evaluation. DrivingDiffusion says it matched metrics and settings "a...
The stated protocols differ in at least five ways: Drive-WM 150 validation clips x 6 views = 900 segments of 40 frames with the official UCF FVD code; GenAD FID on 6019 frames and FVD on 4369 clips; UniMLVG following MagicDrive with 900 videos of 16 frames; Panacea generating the whole validation set at 256x512 with 8 frames; Delphi single-frame single-view FID and 4/8/40-frame FVD with recomputed baselines marked. DriveDreamer v1 states a 192x128 training size, v2 448x256 for training and evaluation; MagicDrive-V2 lists DriveDreamer at 128x192 with 32 frames. DriveDreamer-2 gives no clip or frame count for FID/FVD in the HTML text searched, and DriveDreamer v2 has no clips/segments sentence at all; DriveDreamer v1 states 150 clips x 8 segments = 1200, but for action-prediction evaluation. DrivingDiffusion says it matched metrics and settings "as closely as" possible; Delphi says baselines marked N/A could not be reproduced because weights were not open. Consequence (synthesis): a rank across rows from different papers mixes protocols that the papers themselves describe differently.
Curator correction 2026-10-24 (discovery audit record C3-B1051-01, curator-reviewed C3R-0688; excerpts re-located by independent curator GET 2026-10-24): Correction. Record C3-B156-W31-04 says DriveDreamer-2 'gives no clip or frame count for FID/FVD in the HTML text searched' and that its v2 HTML text 'contains no sentence giving clip/segment/sample counts'. The arXiv v2 HTML (fetched 2026-10-24T13:54:15Z) does: an Evaluation Details paragraph says the FID and FVD calculations use 150 validation videos from nuScenes, at 4Hz with 8 frames per clip, with the official UCF FVD code. The record's absence claim was therefore wrong as written (the record did not date a source change; no earlier-correct reading is shown). arXiv v2 was posted on 11 Apr 2024, before the record was written, and the v1 PDF does not contain the paragraph. The paragraph gives no resolution and does not say whether the 150 videos are 150 clips or scenes; Drive-WM's own stated protocol is 150 validation clips x 6 views = 900 segments of 40 frames, so the two are not shown to match. The record's other protocol rows were re-read and their excerpts are present.
Curator qualifications: All 23 excerpts located; the five protocol differences (clip counts, frames, FVD code, resolution) match the cited texts per excerpt location.
Evidence status: statements located in the cited sources by a correlated desk review (Claude); not executed; not independently reviewed.
Structured fields
{
"citations": [
{
"url": "https://arxiv.org/abs/2309.09777v1",
"note": "candidate evidence excerpts"
},
{
"url": "https://arxiv.org/abs/2309.09777v2",
"note": "candidate evidence excerpts"
},
{
"url": "https://arxiv.org/abs/2311.17918v1",
"note": "candidate evidence excerpts"
},
{
"url": "https://arxiv.org/abs/2311.16813v1",
"note": "candidate evidence excerpts"
},
{
"url": "https://arxiv.org/abs/2403.09630v2",
"note": "candidate evidence excerpts"
},
{
"url": "https://arxiv.org/abs/2412.04842v3",
"note": "candidate evidence excerpts"
},
{
"url": "https://arxiv.org/abs/2406.01349v3",
"note": "candidate evidence excerpts"
},
{
"url": "https://arxiv.org/abs/2411.13807v4",
"note": "candidate evidence excerpts"
},
{
"url": "https://arxiv.org/abs/2409.01595v1",
"note": "candidate evidence excerpts"
},
{
"url": "https://arxiv.org/abs/2310.07771v1",
"note": "candidate evidence excerpts"
},
{
"url": "https://arxiv.org/abs/2403.06845v2",
"note": "candidate evidence excerpts"
}
]
}Evidence claims
- [sourced_fact] Curator correction 2026-10-24 (discovery audit record C3-B1051-01, curator-reviewed C3R-0688; excerpts re-located by independent curator GET 2026-10-24): Correction. Record C3-B156-W31-04 says DriveDreamer-2 'gives no clip or frame count for FID/FVD in the HTML text searched' and that its v2 HTML text 'contains no sentence giving clip/segment/sample counts'. The arXiv v2 HTML (fetched 2026-10-24T13:54:15Z) does: an Evaluation Details paragraph says the FID and FVD calculations use 150 validation videos from nuScenes, at 4Hz with 8 frames per clip, with the official UCF FVD code. The record's absence claim was therefore wrong as written (the record did not date a source change; no earlier-correct reading is shown). arXiv v2 was posted on 11 Apr 2024, before the record was written, and the v1 PDF does not contain the paragraph. The paragraph gives no resolution and does not say whether the 150 videos are 150 clips or scenes; Drive-WM's own stated protocol is 150 validation clips x 6 views = 900 segments of 40 frames, so the two are not shown to match. The record's other protocol rows were re-read and their excerpts are present. · locator: curator correction paragraph in this record body (C3-B1051-01) source
- [sourced_fact] SUPERSEDED IN PART 2026-10-24 by curator correction C3-B1051-01 (see the correction claim above); original kept for history: The stated protocols differ in at least five ways: Drive-WM 150 validation clips x 6 views = 900 segments of 40 frames with the official UCF FVD code; GenAD FID on 6019 frames and FVD on 4369 clips; UniMLVG following MagicDrive with 900 videos of 16 frames; Panacea generating the whole validation set at 256x512 with 8 frames; Delphi single-frame single-view FID and 4/8/40-frame FVD with recomputed baselines marked. DriveDreamer v1 states a 192x128 training size, v2 448x256 for training and evaluation; MagicDrive-V2 lists DriveDreamer at 128x192 with 32 frames. DriveDreamer-2 gives no clip or frame count for FID/FVD in the HTML text searched, and DriveDreamer v2 has no clips/segments sentence at all; DriveDreamer v1 states 150 clips x 8 segments = 1200, but for action-prediction evaluation. DrivingDiffusion says it matched metrics and settings "as closely as" possible; Delphi says baselines marked N/A could not be reproduced because weights were not open. Consequence (synthesis): a rank across rows from different papers mixes protocols that the papers themselves describe differently. · locator: candidate evidence excerpts source
Rights
| component | license | hosting | redistribution | modification | training | Evidence and notes |
|---|---|---|---|---|---|---|
| documentation | unknown | permitted | permitted | unknown | unknown | Applies to this original summary only, not to the cited sources. |
Linked records
- cites → https://arxiv.org/abs/2309.09777v1
- cites → https://arxiv.org/abs/2309.09777v2
- cites → https://arxiv.org/abs/2310.07771v1
- cites → https://arxiv.org/abs/2311.16813v1
- cites → https://arxiv.org/abs/2311.17918v1
- cites → https://arxiv.org/abs/2403.06845v2
- cites → https://arxiv.org/abs/2403.09630v2
- cites → https://arxiv.org/abs/2406.01349v3
- cites → https://arxiv.org/abs/2409.01595v1
- cites → https://arxiv.org/abs/2411.13807v4
- cites → https://arxiv.org/abs/2412.04842v3