How does ViStoryBench define and score character consistency for story visualisation (metrics, dataset, headline numbers, validity checks, stated limits), and which components are under which licence?

ViStoryBench (arXiv 2505.24862v5, 'Accepted by CVPR 2026' per the arXiv API comment) provides 80 stories with 344 characters, 509 reference images, 1,317 shots across 10 style genres (39 photorealistic), in English and Chinese, and 12 automated metrics: cross/self style similarity (CSD), cross/self character identification similarity (CIDS: Grounding DINO crops, ArcFace/AdaFace/FaceNet for realistic characters and CLIP otherwise, bipartite matching), VLM-judged prompt alignment (scene, shot, character interaction, individual action), on-stage character count, aesthetics, Inception score and a copy-paste detector. Table 2 gives the copy-paste baseline (CIDS cross 0.929) as a reference and e.g. OmniGen2 0.548, UNO 0.485 and StoryDiffusion 0.397/0.436 on cross-CIDS. Self-CIDS correlates with a user study at Kendall 0.4978, Spearman 0.6759, Pearson 0.7956 (excluding outliers). Authors state no background-reference evaluation, VLM hallucination risk, uncontrolled language effects and a fair-use claim for third-party images. Components are licensed separately: paper text CC BY-NC-ND 4.0, evaluation code Apache-2.0, dataset MIT (README/card), evaluator weights third-party. Curator correction 2026-10-23 (discovery audit record C3-B681-01; sources re-fetched 2026-10-23, excerpts re-located by curator GET): Problem. C3-B227-G69-06 (and the Table 2 sentence in C3-B227-G69-01) present the copy-paste baseline CIDS 0.929 cross / 0.984 self as a fixed reference. Correction type: (c) missing an important limitation / (e) changed since the paper. Conditions. Applies when you re-run the ViStoryBench CIDS evaluator and compare to the paper's copy-paste row. Method. The evaluator README states that, for characters with a single reference image, the current code computes the copy-paste score with a mirrored reference image, and that the paper and project-homepage numbers pre-date that fix. Rerun the baseline yourself with the same code version as your model rows instead of importing 0.929/0.984. Limitations. The README says only 'slightly different'; the size of the shift was not measured here. Version and date. README at pinned commit b44ec91 (identical bytes at main, fetched 2026-10-23). Curator qualifications: ViStoryBench: 80 stories, 344 characters, 1,317 shots, 12 automated metrics incl. character identification similarity; accepted at CVPR 2026 per arXiv comment. Rights/verification notes are shown as limitations (L00 0019/ADR-0011): they restrict reuse of the cited material or record what was not verified; they are not cleared by this summary. Limitations: [component_reuse_restriction] Dataset images noted as fair use (rights hold). Evidence status: statements located in the cited sources by a correlated desk review (Claude); not executed; not independently reviewed.

{
  "citations": [
    {
      "url": "https://arxiv.org/html/2505.24862v5",
      "note": "candidate evidence excerpts"
    },
    {
      "url": "https://github.com/ViStoryBench/vistorybench/tree/b44ec9108668cc2bcc8c5280886b235e9fb8bea9",
      "note": "candidate evidence excerpts"
    },
    {
      "url": "https://huggingface.co/datasets/ViStoryBench/ViStoryBench/tree/92f845531b67e97a67ae04b256ec5d8c020e8341",
      "note": "candidate evidence excerpts"
    },
    {
      "url": "https://export.arxiv.org/api/query?id_list=2505.24862,2405.01434,2402.03286,2311.10093,2312.04461,2401.07519&max_results=10",
      "note": "candidate evidence excerpts"
    },
    {
      "url": "https://raw.githubusercontent.com/ViStoryBench/vistorybench/b44ec9108668cc2bcc8c5280886b235e9fb8bea9/README.md",
      "note": "correction source (record C3-B681-01): README note after the evaluation CLI options; same note"
    }
  ]
}