Does the cache stay portable across machines and repos: what do symlinks, shared Xet blobs and HF_HUB_DISABLE_SYMLINKS change, and is hf_transfer still used?
Problem. A cache copied to a NAS, Windows host or second repo behaves oddly, or disk use is higher than expected. Conditions. Recent huggingface_hub with hf_xet. Method. The cache is symlink-based (snapshots/ points to blobs/); where symlinks are unsupported it stores files directly in snapshots/. HF_HUB_DISABLE_SYMLINKS=1 forces that mode, at the cost of duplicated big files, and the docs name a shared NAS used across operating systems as the use case. By default, Xet files are also stored once at <CACHE_DIR>/blobs/<prefix>/<xet_hash> and symlinked into repos so identical files across repos are not re-fetched; HF_HUB_DISABLE_SHARED_BLOBS=1 opts out, and it is off when Xet is disabled or symlinks are unsupported. Fast transfer is hf_xet (installed with huggingface_hub since 0.32.0); HF_HUB_ENABLE_HF_TRANSFER is deprecated, and HF_XET_HIGH_PERFORMANCE=1 uses more network and CPU. Failure modes. Older tools may delete a shared file used by other repos (it is re-downloaded); old cache ...
Structured fields
{
"citations": [
{
"url": "https://huggingface.co/docs/huggingface_hub/main/en/package_reference/environment_variables.md",
"note": "HF_HUB_DISABLE_SYMLINKS; Shared blobs across repos; Shared blobs across repos; hf_xet; HF_XET_HIGH_PERFORMANCE"
},
{
"url": "https://huggingface.co/docs/huggingface_hub/main/en/guides/manage-cache.md",
"note": "HF_HUB_DISABLE_SYMLINKS; Shared blobs across repos; Shared blobs across repos; hf_xet; HF_XET_HIGH_PERFORMANCE"
},
{
"url": "https://huggingface.co/docs/huggingface_hub/main/en/guides/download.md",
"note": "HF_HUB_DISABLE_SYMLINKS; Shared blobs across repos; Shared blobs across repos; hf_xet; HF_XET_HIGH_PERFORMANCE"
}
]
}Evidence claims
- [sourced_fact] Problem. A cache copied to a NAS, Windows host or second repo behaves oddly, or disk use is higher than expected. Conditions. Recent huggingface_hub with hf_xet. Method. The cache is symlink-based (snapshots/ points to blobs/); where symlinks are unsupported it stores files directly in snapshots/. HF_HUB_DISABLE_SYMLINKS=1 forces that mode, at the cost of duplicated big files, and the docs name a shared NAS used across operating systems as the use case. By default, Xet files are also stored once at <CACHE_DIR>/blobs/<prefix>/<xet_hash> and symlinked into repos so identical files across repos are not re-fetched; HF_HUB_DISABLE_SHARED_BLOBS=1 opts out, and it is off when Xet is disabled or symlinks are unsupported. Fast transfer is hf_xet (installed with huggingface_hub since 0.32.0); HF_HUB_ENABLE_HF_TRANSFER is deprecated, and HF_XET_HIGH_PERFORMANCE=1 uses more network and CPU. Failure modes. Older tools may delete a shared file used by other repos (it is re-downloaded); old cache scanners may flag the top-level blobs folder. Limitations. Extends C3-B609-05; does not cover xet speed measurements. Version/date: huggingface_hub docs main branch fetched 2026-10-24 (stamp per bibliography); behaviour may differ in older releases. · locator: HF_HUB_DISABLE_SYMLINKS; Shared blobs across repos; Shared blobs across repos; hf_xet; HF_XET_HIGH_PERFORMANCE source
Rights
| component | license | hosting | redistribution | modification | training | Evidence and notes |
|---|---|---|---|---|---|---|
| documentation | unknown | permitted | permitted | unknown | unknown | Applies to this original summary only, not to the cited sources. |