Which automatic metric does T2I-CompBench++ use for each compositional category (colour/shape/texture binding, 2D/3D spatial, numeracy, non-spatial, complex), and how well does each correlate with human ratings?

Partially verified: the curator re-read none of the cited sources; S3, S4, S5 rest on the producer's pinned reading. T2I-CompBench++ (arXiv v3) has 8,000 prompts in four groups and eight sub-categories, and uses a different checker per sub-category: disentangled BLIP-VQA for colour/shape/texture binding, a UniDet detection (plus depth for 3D) rule for spatial relations and numeracy, CLIPScore or an MLLM for non-spatial relations, and a 3-in-1 average for complex prompts. Validated by Kendall tau / Spearman rho against 3-worker MTurk ratings, B-VQA is strongest for colour (tau 0.63) but weak for shape (0.27), UniDet is the best non-MLLM metric for spatial and numeracy, and plain CLIPScore is weak almost everywhere. Curator qualifications: Partially verified: the curator re-read none of the cited sources; S3, S4, S5 rest on the producer's pinned reading. Curator GET 2026-10-23 of the sources not re-read (mechanical source evidence, not semantic verification): S3: HTTP 200, bytes differ from the producer pin; S4: HTTP 200, bytes differ from the producer pin; S5: HTTP 200, bytes differ from the producer pin. Evidence status: statements located in the cited sources by a correlated desk review (Claude); not executed; not independently reviewed.

{
  "citations": [
    {
      "url": "https://arxiv.org/abs/2307.06350v3",
      "note": "candidate evidence excerpts"
    },
    {
      "url": "https://arxiv.org/abs/2307.06350v1",
      "note": "candidate evidence excerpts"
    },
    {
      "url": "https://github.com/Karine-Huang/T2I-CompBench/tree/4aa404212eb5d06e5adbcd9cee696c750d0d25a5",
      "note": "candidate evidence excerpts"
    }
  ]
}