How accurate were licence labels on GitHub, Hugging Face and Papers with Code according to the Data Provenance Initiative audit, and how was that measured?

The 2023 Data Provenance audit re-traced licences for 1,858 datasets in 44 finetuning collections and reports that hosting-site labels are often missing (70%+) or wrong (50%+ error): GitHub, Hugging Face and Papers with Code matched the annotated licence 43%, 35% and 54% of the time and were too permissive 29%, 27% and 16% of the time. The authors call their protocol a best-effort check, not legal advice. An agent should not treat a hosting-site licence label as verified. Curator qualifications: Paper read 2026-10-22 from producer copy; percentages are the authors' reported results, not re-derived. Attribution of errors to code-vs-data licence confusion not located in curator search (producer reading). Aggregator Unspecified 69-72% vs 30.7% figure from table, producer reading. Limitations: Paper read 2026-10-22 from producer copy; percentages are the authors' reported results, not re-derived. Evidence status: statements located in the cited sources by a correlated desk review (Claude); not executed; not independently reviewed.

{
  "citations": [
    {
      "url": "https://arxiv.org/pdf/2310.16787",
      "note": "arxiv.org/abs/2310.16787 (curator fetch); producer PDF sha256 6766e913dad8c29c = pin: abstract, Section 2, Section 3 after Table 2"
    },
    {
      "url": "https://github.com/Data-Provenance-Initiative/Data-Provenance-Collection/blob/ce6662deccf011574427b6f112df479d261960b8/README.md",
      "note": "arxiv.org/abs/2310.16787 (curator fetch); producer PDF sha256 6766e913dad8c29c = pin: abstract, Section 2, Section 3 after Table 2"
    },
    {
      "url": "https://github.com/Data-Provenance-Initiative/Data-Provenance-Collection/blob/ce6662deccf011574427b6f112df479d261960b8/data_summaries-speech/AMI.json",
      "note": "arxiv.org/abs/2310.16787 (curator fetch); producer PDF sha256 6766e913dad8c29c = pin: abstract, Section 2, Section 3 after Table 2"
    }
  ]
}