Is a face-recognition similarity threshold equally reliable for every kind of face (NIST IR 8280 demographic effects)?

Problem: an identity check that is tuned on one group may over-flag or under-flag others. Conditions: any automated pass/fail on face similarity. Method (NIST's finding): algorithms submitted to NIST's evaluation showed demographic differentials of various magnitudes; false-positive differentials were much larger than false-negative ones; across demographics false-positive rates often varied by factors of 10 to beyond 100, and results depended on the algorithm and on the photo set. Mapping to AI review (inference): set a similarity threshold per character or per cast from known-good pairs rather than a global number; treat a pass as weak evidence and a fail as a prompt for human review; log the model name and version used. Failure modes: a gate that never fails because the threshold is too loose; a gate that fails characters of one appearance more often. Limitations: the report tests 1:1 verification and 1:N on photographs with mugshot, visa and application images, not stylised or generated faces; algorithms evaluated were 2018-2019 submissions, so current models may differ. Version/date: NIST IR 8280, December 2019; fetched 2026-10-23. Curator qualifications: All cited documentation excerpts located on the live pages by curator GET; no video sources. Limitations: The report's algorithms and datasets are dated (2018-2019) and not about generated faces; the mapping to AI QC is inference. Only the executive summary was read closely. Evidence status: statements located in the cited sources by a correlated desk review (Claude); not executed; not independently reviewed.

{
  "citations": [
    {
      "url": "https://nvlpubs.nist.gov/nistpubs/ir/2019/NIST.IR.8280.pdf",
      "note": "Executive Summary; Executive Summary"
    }
  ]
}