How well can people actually tell which character said a line, and what does that mean for checking my own dialogue?
Two human studies on human-authored dialogue are in the sources. In Speaker Verification, ten non-expert annotators judging whether two sets of utterances came from the same speaker scored 56.50% accuracy on the base level when shown only utterances (67.48% when shown conversations); in PersonaEval, 20 educated volunteers scored 90.8% when choosing among named candidate roles with profiles and context, on cases chosen because a strong model got them wrong. The gap suggests context cues (who addresses whom, plot) carry most of the human signal. No human study of attributing LLM-generated lines to characters was found.
Two human studies on human-authored dialogue are in the sources. In Speaker Verification, ten non-expert annotators judging whether two sets of utterances came from the same speaker scored 56.50% accuracy on the base level when shown only utterances (67.48% when shown conversations); in PersonaEval, 20 educated volunteers scored 90.8% when choosing among named candidate roles with profiles and context, on cases chosen because a strong model got them wrong. The gap suggests context cues (who addresses whom, plot) carry most of the human signal. No human study of attributing LLM-generated lines to characters was found.
Curator qualifications: Human baselines on human-written dialogue: 56.50% (bare utterances) / 67.48% (with conversation) same-speaker accuracy; 90.8% when choosing among profiled roles with context (hard sample).
Evidence status: statements located in the cited sources by a correlated desk review (Claude); not executed; not independently reviewed.
Structured fields
{
"citations": [
{
"url": "https://arxiv.org/html/2405.10150v2",
"note": "candidate evidence excerpts"
},
{
"url": "https://arxiv.org/html/2508.10014v1",
"note": "candidate evidence excerpts"
}
]
}Evidence claims
- [sourced_fact] Two human studies on human-authored dialogue are in the sources. In Speaker Verification, ten non-expert annotators judging whether two sets of utterances came from the same speaker scored 56.50% accuracy on the base level when shown only utterances (67.48% when shown conversations); in PersonaEval, 20 educated volunteers scored 90.8% when choosing among named candidate roles with profiles and context, on cases chosen because a strong model got them wrong. The gap suggests context cues (who addresses whom, plot) carry most of the human signal. No human study of attributing LLM-generated lines to characters was found. · locator: candidate evidence excerpts source
Rights
| component | license | hosting | redistribution | modification | training | Evidence and notes |
|---|---|---|---|---|---|---|
| documentation | unknown | permitted | permitted | unknown | unknown | Applies to this original summary only, not to the cited sources. |