I see Physics-IQ Verified scores and a "net improvement" column quoted for video models; what do "Verified" and "net improvement" mean, and which view of the leaderboard am I looking at?
Verified (arXiv v1, 2026-06-17) is a re-scoring of the 2025 Physics-IQ benchmark: cleaned prompts and ground truth, and a per-sample score (mean of four clipped ratios) averaged over 198 first-take samples. "Net improvement" is not in the paper: it is a view on the Anates-run leaderboard page that shows each row minus the average of the rows in that track. The page opens on image-to-video, single-generation rows, net-improvement view, so a quoted rank without track, sampling and view is ambiguous.
Verified (arXiv v1, 2026-06-17) is a re-scoring of the 2025 Physics-IQ benchmark: cleaned prompts and ground truth, and a per-sample score (mean of four clipped ratios) averaged over 198 first-take samples. "Net improvement" is not in the paper: it is a view on the Anates-run leaderboard page that shows each row minus the average of the rows in that track. The page opens on image-to-video, single-generation rows, net-improvement view, so a quoted rank without track, sampling and view is ambiguous.
Curator qualifications: Verified the SOTA rule and the DeepMind disclaimer. 'Net improvement' as a page view (not a paper metric) is code-derived from the page records.
Evidence status: statements located in the cited sources by a correlated desk review (Claude); not executed; not independently reviewed.
Structured fields
{
"citations": [
{
"url": "https://arxiv.org/abs/2606.18943v1",
"note": "Verified table rows 1-2, 4, 5, 12-14; rule L105; disclaimer L562"
},
{
"url": "https://physics-iq-verified.anates.ai/",
"note": "Verified table rows 1-2, 4, 5, 12-14; rule L105; disclaimer L562"
},
{
"url": "https://physics-iq-verified.anates.ai/_next/static/chunks/76-2a2d962ffb0c0e39.js",
"note": "Verified table rows 1-2, 4, 5, 12-14; rule L105; disclaimer L562"
},
{
"url": "https://github.com/google-deepmind/physics-IQ-benchmark/blob/d3f3de16a9c66b4d3bfd6bdb5518570e8b72ad48/README.md",
"note": "Verified table rows 1-2, 4, 5, 12-14; rule L105; disclaimer L562"
}
]
}Evidence claims
- [sourced_fact] Verified (arXiv v1, 2026-06-17) is a re-scoring of the 2025 Physics-IQ benchmark: cleaned prompts and ground truth, and a per-sample score (mean of four clipped ratios) averaged over 198 first-take samples. "Net improvement" is not in the paper: it is a view on the Anates-run leaderboard page that shows each row minus the average of the rows in that track. The page opens on image-to-video, single-generation rows, net-improvement view, so a quoted rank without track, sampling and view is ambiguous. · locator: Verified table rows 1-2, 4, 5, 12-14; rule L105; disclaimer L562 source
Rights
| component | license | hosting | redistribution | modification | training | Evidence and notes |
|---|---|---|---|---|---|---|
| documentation | unknown | permitted | permitted | unknown | unknown | Applies to this original summary only, not to the cited sources. |