Data Provenance Initiative: Data Provenance Collection
Source: https://github.com/data-provenance-initiative/data-provenance-collection
Structured fields
{
"archived": false,
"revision": {
"ref": "main",
"pinned": false
},
"checked_at": "2026-10-17T00:00:00.000Z",
"usefulness": "Audit of 44 data collections (1,800+ text finetuning datasets) documenting sources, licenses and creators, with scripts to filter by self-reported license constraints. Method and data for license-aware dataset selection; text-focused, but the audit approach transfers to media datasets.",
"source_type": "dataset",
"availability": "available",
"source_owner": "Data Provenance Initiative",
"next_check_at": "2026-10-31T00:00:00.000Z",
"known_limitations": [
"Text datasets only; licenses are self-reported."
],
"upstream_updated_at": "2025-03-26T02:46:12.000Z"
}Evidence claims
- [upstream_claim] README: volunteer effort for transparency in training datasets; audit of 44 collections spanning 1800+ finetuning text-to-text datasets documenting web and machine sources, licenses, creators and metadata; scripts filter by license constraints. · locator: README intro source
- [sourced_fact] Repo metadata: Apache-2.0, not archived, last push 2025-03-26. · locator: GitHub API fields source
- [upstream_claim] Research review verdict at export: reviewed (as of 2026-10-17). Recorded by the research pipeline, not a hub review outcome; evidence.review stays unreviewed until an independent reviewer records one.
Rights
| component | license | hosting | redistribution | modification | training | Evidence and notes |
|---|---|---|---|---|---|---|
| code | Apache-2.0 | permitted | permitted | permitted | unknown | Apache-2.0 for the scripts. |
| dataset | unknown | unknown | unknown | unknown | unknown | License terms of the compiled metadata and the legal-notice section were not read. |