How could an agent check one generated image against its prompt, using what these benchmarks do, and how far can it trust each check?
Proposal (synthesis, untested): split the prompt into elements; check counts, positions and colours with detector/crop-classifier rules, quoted text with an independent OCR and exact match, and open relations with per-element yes/no VLM questions that carry parent dependencies; repeat the VLM judge and keep a human-labelled sample to measure agreement. Each piece is documented in the benchmarks (GenEval, T2I-CompBench++, DPG-Bench, MARIO-Eval, AnyText) but the combined agent procedure has not been reported or tested in these sources.
Proposal (synthesis, untested): split the prompt into elements; check counts, positions and colours with detector/crop-classifier rules, quoted text with an independent OCR and exact match, and open relations with per-element yes/no VLM questions that carry parent dependencies; repeat the VLM judge and keep a human-labelled sample to measure agreement. Each piece is documented in the benchmarks (GenEval, T2I-CompBench++, DPG-Bench, MARIO-Eval, AnyText) but the combined agent procedure has not been reported or tested in these sources.
Curator qualifications: Labelled untested proposal for per-image prompt checking built from verified benchmark mechanics.
Evidence status: labelled proposal derived from the cited sources; untested; correlated desk review (Claude); not independently reviewed.
Structured fields
{
"citations": [
{
"url": "https://arxiv.org/abs/2310.11513v1",
"note": "candidate evidence excerpts"
},
{
"url": "https://arxiv.org/abs/2307.06350v3",
"note": "candidate evidence excerpts"
},
{
"url": "https://github.com/TencentQQGYLab/ELLA/tree/3c228f1dc6c4d3cad0a47493816151a419f14db3",
"note": "candidate evidence excerpts"
},
{
"url": "https://github.com/microsoft/unilm/tree/31c5b904ca1bf2afb4c234a6675c683a4e5fc7cd/textdiffuser",
"note": "candidate evidence excerpts"
},
{
"url": "https://arxiv.org/abs/2311.03054v5",
"note": "candidate evidence excerpts"
}
]
}Evidence claims
- [proposal] Proposal (synthesis, untested): split the prompt into elements; check counts, positions and colours with detector/crop-classifier rules, quoted text with an independent OCR and exact match, and open relations with per-element yes/no VLM questions that carry parent dependencies; repeat the VLM judge and keep a human-labelled sample to measure agreement. Each piece is documented in the benchmarks (GenEval, T2I-CompBench++, DPG-Bench, MARIO-Eval, AnyText) but the combined agent procedure has not been reported or tested in these sources. · locator: candidate evidence excerpts source
Rights
| component | license | hosting | redistribution | modification | training | Evidence and notes |
|---|---|---|---|---|---|---|
| documentation | unknown | permitted | permitted | unknown | unknown | Applies to this original summary only, not to the cited sources. |
Linked records
- cites → https://arxiv.org/abs/2307.06350v3
- cites → https://arxiv.org/abs/2310.11513v1
- cites → https://arxiv.org/abs/2311.03054v5
- cites → https://github.com/TencentQQGYLab/ELLA/tree/3c228f1dc6c4d3cad0a47493816151a419f14db3
- cites → https://github.com/microsoft/unilm/tree/31c5b904ca1bf2afb4c234a6675c683a4e5fc7cd/textdiffuser