How comparable are the evaluation protocols (datasets, subsets, alignment, baselines) across TripoSR, SF3D, SPAR3D, TRELLIS, TRELLIS.2, Hunyuan3D and TripoSG?
Protocols differ enough that absolute numbers should not be compared across papers. TripoSR, SF3D and SPAR3D share GSO and OmniObject3D but use different subsets (about 300 each; 278 and 308; about 250 each) and different GPUs; SF3D re-measured TripoSR at CD 0.111 / FS@0.1 0.645 on GSO while TripoSR reports CD 0.111 / 0.651 on its own subset. TRELLIS uses Toys4k with depth-unprojected points; TRELLIS.2 adds 90 curated Sketchfab assets and 100 AI-generated prompts; Hunyuan 2.0 anonymises several baselines; TripoSG evaluates shape-only with Normal-FID and an LMM judge. Only SF3D, SPAR3D and TripoSR share a metric and datasets closely enough for rough comparison, and even they were each run by the authors of the newer method.
Protocols differ enough that absolute numbers should not be compared across papers. TripoSR, SF3D and SPAR3D share GSO and OmniObject3D but use different subsets (about 300 each; 278 and 308; about 250 each) and different GPUs; SF3D re-measured TripoSR at CD 0.111 / FS@0.1 0.645 on GSO while TripoSR reports CD 0.111 / 0.651 on its own subset. TRELLIS uses Toys4k with depth-unprojected points; TRELLIS.2 adds 90 curated Sketchfab assets and 100 AI-generated prompts; Hunyuan 2.0 anonymises several baselines; TripoSG evaluates shape-only with Normal-FID and an LMM judge. Only SF3D, SPAR3D and TripoSR share a metric and datasets closely enough for rough comparison, and even they were each run by the authors of the newer method.
Curator qualifications: Excerpted evidence located; TripoSR GSO 0.651 vs 0.645 by two authors (subsets) kept; cross-paper numbers not comparable.
Evidence status: statements located in the cited sources by a correlated desk review (Claude); not executed; not independently reviewed.
Structured fields
{
"citations": [
{
"url": "https://arxiv.org/abs/2403.02151v1",
"note": "candidate evidence excerpts"
},
{
"url": "https://arxiv.org/abs/2408.00653v1",
"note": "candidate evidence excerpts"
},
{
"url": "https://arxiv.org/abs/2501.04689v1",
"note": "candidate evidence excerpts"
},
{
"url": "https://arxiv.org/abs/2412.01506v3",
"note": "candidate evidence excerpts"
},
{
"url": "https://arxiv.org/abs/2512.14692v1",
"note": "candidate evidence excerpts"
},
{
"url": "https://arxiv.org/abs/2501.12202v5",
"note": "candidate evidence excerpts"
},
{
"url": "https://arxiv.org/abs/2502.06608v3",
"note": "candidate evidence excerpts"
},
{
"url": "https://arxiv.org/abs/2506.15442v1",
"note": "candidate evidence excerpts"
}
]
}Evidence claims
- [sourced_fact] Protocols differ enough that absolute numbers should not be compared across papers. TripoSR, SF3D and SPAR3D share GSO and OmniObject3D but use different subsets (about 300 each; 278 and 308; about 250 each) and different GPUs; SF3D re-measured TripoSR at CD 0.111 / FS@0.1 0.645 on GSO while TripoSR reports CD 0.111 / 0.651 on its own subset. TRELLIS uses Toys4k with depth-unprojected points; TRELLIS.2 adds 90 curated Sketchfab assets and 100 AI-generated prompts; Hunyuan 2.0 anonymises several baselines; TripoSG evaluates shape-only with Normal-FID and an LMM judge. Only SF3D, SPAR3D and TripoSR share a metric and datasets closely enough for rough comparison, and even they were each run by the authors of the newer method. · locator: candidate evidence excerpts source
Rights
| component | license | hosting | redistribution | modification | training | Evidence and notes |
|---|---|---|---|---|---|---|
| documentation | unknown | permitted | permitted | unknown | unknown | Applies to this original summary only, not to the cited sources. |
Linked records
- cites → https://arxiv.org/abs/2403.02151v1
- cites → https://arxiv.org/abs/2408.00653v1
- cites → https://arxiv.org/abs/2412.01506v3
- cites → https://arxiv.org/abs/2501.04689v1
- cites → https://arxiv.org/abs/2501.12202v5
- cites → https://arxiv.org/abs/2502.06608v3
- cites → https://arxiv.org/abs/2506.15442v1
- cites → https://arxiv.org/abs/2512.14692v1