How does the TUNa unreliable-narrator dataset instruct annotators to treat a narrator's statements against other characters' views and the text's own facts, and what labels and agreement does that produce?

TUNa (ACL 2025) is a narrator-level annotation: ten literature-trained annotators label each short first-person snippet for intra-narrational (verbal tics), inter-narrational (same narrator changing over time, or another character contradicting) and inter-textual (naif, madman, picaro, clown) unreliability, with kappa 0.71-0.75 and a short written justification per label. Its instruction to judge from the narrative only, to assume reliability by default, and to leave unresolved which character is right means it records a conflict between narrator belief and another character's view without deciding the story-world fact. Labels are per snippet, not per statement, the corpus is short and English, and the full dataset is available by email request (not inspected). Curator correction 2026-10-24 (discovery audit record C3-B1123-01, curator-reviewed C3R-0711; excerpts re-located by independent curator GET 2026-10-24): Correction. Record C3-B201-UO-SP01-05 says TUNa's 'instruction to judge from the narrative only, to assume reliability by default, and to leave unresolved which character is right' means the dataset records a narrator-versus-character conflict without deciding the story-world fact. The first two are annotator instructions (arXiv 2506.10231v1 Appendix B.1: classify using defining characteristics within the narrative only; do not create hypotheticals; assume the narrator is reliable unless unreliable traits are present). The third is not an annotator instruction: Appendix B.1 gives annotators no rule about resolving or not resolving which character is right. The sentence 'The reader does not know which character understands the situation best' is the authors' comment on one constructed example in Section 3.2, which defines the other-character-contradiction label as another character contradicting the narrator, typically in direct dialogue. The record's other figures checked out against arXiv v1 (10 annotators, 817 narratives, kappa 0.75/0.71/0.73, mean 21.2 tokens per explanation; repository README unchanged at e9b7d91 on 2026-10-24; arXiv still lists only v1). The overstatement was present when the record was written (2026-10-22); it is not a change in the source. Use: TUNa shows how narrator-versus-character conflict is labelled and how the authors describe the reader's uncertainty in an example; it does not show that annotators were told to leave the conflict unresolved or told anything about world facts. Curator qualifications: TUNa narrator-level unreliability labels (intra-, inter-narrational, inter-textual) with kappa 0.71-0.75; judge from text only, assume reliability by default. 10/10 located. Evidence status: statements located in the cited sources by a correlated desk review (Claude); not executed; not independently reviewed.

{
  "citations": [
    {
      "url": "https://arxiv.org/html/2506.10231v1",
      "note": "candidate evidence excerpts"
    },
    {
      "url": "https://raw.githubusercontent.com/adbrei/unreliable-narrators/e9b7d91f2e33e057ff35a63a3e9b43a63daaa586/README.md",
      "note": "candidate evidence excerpts"
    }
  ]
}