How well do image detectors trained on one generator family detect images from generators they never saw, and does a better feature space fix it?

Ojha et al. (v2) show that a ProGAN-trained real-vs-fake classifier keeps near-perfect accuracy on GANs but falls toward chance on unseen diffusion and autoregressive generators, labelling the fakes as real; they report that nearest-neighbour or linear probing on frozen CLIP ViT features reaches about 82-84 percent average accuracy on those unseen families against 53-58 percent for trained networks. A later benchmark (AIDE paper) re-ran UnivFD and eight other detectors and found UnivFD at 78.43 mean accuracy on the 16-generator AIGCDetect set with 50.75 on DALLE2. On the harder Chameleon set UnivFD caught 74.97 percent of fakes but passed only 41.09 percent of real images. The two papers measure different test sets and neither is a field estimate. Curator correction 2026-10-23 (discovery audit record C3-B678-01; sources re-fetched 2026-10-23, excerpts re-located by curator GET): Problem. C3-B116-G42-02 says the ProGAN-trained real-vs-fake classifier 'keeps near-perfect accuracy on GANs' and gives UnivFD's Chameleon result as 74.97 percent of fakes caught and 41.09 percent of real images passed. Conditions. Two conditions were dropped. (1) Ojha et al. (arXiv 2302.10174v2) call the generalisation 'excellent' only within the ProGAN-to-StyleGAN family; the same paper's Table 1 shows the plain ProGAN-trained classifier averaging 80.77 (CycleGAN) and 79.25 (GauGAN), with fake-image accuracy of 62.91 and 59.1, and the text gives the best trained-network baseline as about 85 percent average on GANs, versus about 93 and 95 percent for the NN and linear-probe variants. So 'near-perfect' holds for the seen generator and close relatives, not for all GANs. (2) In the AIDE paper (arXiv 2406.19435v3) Table 5 the 74.97/41.09 pair is UnivFD trained on SD v1.4 images, given as 'fake image / real image' accuracy. In the ProGAN-trained row the same UnivFD column reads 3.18/97.83, i.e. almost every fake passed as real. Method. Re-read both tables with the column headers (the arXiv HTML flattens them; column order CycleGAN, GauGAN, LDM, Guided, DALL-E for Table 1; CNNSpot, FreDect, Fusing, GramNet, LNP, UnivFD, DIRE, PatchCraft, NPR, AIDE for Table 5) and always state the training set when quoting a detector score. Failure modes. Quoting 74.97/41.09 without the training set implies a general UnivFD result; it is one training configuration and it trades a 41 percent real-image pass rate for the catch rate. Limitations. Flattened-table reading; the ordering above was inferred from the table headers and from the arithmetic mean check on the AIGCDetect row (the 16 UnivFD values average 78.43). Version/date. arXiv 2302.10174v2 and 2406.19435v3, both the latest versions on 2026-10-23. Curator qualifications: Verified the presence of the per-family tables. The re-run discrepancies (AIDE, Deepfake-Eval) are correctly left unreconciled. Evidence status: statements located in the cited sources by a correlated desk review (Claude); not executed; not independently reviewed.

{
  "citations": [
    {
      "url": "https://arxiv.org/abs/2302.10174v2",
      "note": "results tables; Table 6 (JPEG/blur robustness); abstract; Table 3"
    },
    {
      "url": "https://arxiv.org/abs/2406.19435v3",
      "note": "results tables; Table 6 (JPEG/blur robustness); abstract; Table 3"
    },
    {
      "url": "https://arxiv.org/abs/2503.02857v5",
      "note": "results tables; Table 6 (JPEG/blur robustness); abstract; Table 3"
    },
    {
      "url": "https://arxiv.org/html/2302.10174v2",
      "note": "correction source (record C3-B678-01): Sec. 1 / Table 1 (flattened); Sec. 1 results paragraph; Table 5 caption; Table 5, SD v1.4 training row (flattened); Table 5, ProGAN training row (flattened)"
    },
    {
      "url": "https://arxiv.org/html/2406.19435v3",
      "note": "correction source (record C3-B678-01): Sec. 1 / Table 1 (flattened); Sec. 1 results paragraph; Table 5 caption; Table 5, SD v1.4 training row (flattened); Table 5, ProGAN training row (flattened)"
    }
  ]
}