What could an agent use to check a long draft or a series against its own established facts, and which parts are measured results versus proposals?
Measured: LLM checkers can find injected contradictions in short stories at roughly human level for the best model and drop toward chance beyond about 1,200-4,000 words (FlawedFictions); a four-stage extract-pair-evidence checker with released prompts found 55% of planted errors in about 8,000-10,000-word stories (ConStory). A 2023 KG+GNN approach on synthetic negation did not beat humans. Proposed only: an explicit fact ledger that is filled from earlier text and queried when new text is written; no inspected paper tests it. This candidate separates the two and defines an experiment.
Measured: LLM checkers can find injected contradictions in short stories at roughly human level for the best model and drop toward chance beyond about 1,200-4,000 words (FlawedFictions); a four-stage extract-pair-evidence checker with released prompts found 55% of planted errors in about 8,000-10,000-word stories (ConStory). A 2023 KG+GNN approach on synthetic negation did not beat humans. Proposed only: an explicit fact ledger that is filled from earlier text and queried when new text is written; no inspected paper tests it. This candidate separates the two and defines an experiment.
Curator qualifications: Excerpted evidence located (one excerpt-less row); measured vs proposed parts separated; pairwise-entailment design disagreement kept.
Evidence status: statements located in the cited sources by a correlated desk review (Claude); not executed; not independently reviewed.
Structured fields
{
"citations": [
{
"url": "https://arxiv.org/abs/2504.11900v3",
"note": "candidate evidence excerpts"
},
{
"url": "https://arxiv.org/abs/2603.05890v1",
"note": "candidate evidence excerpts"
},
{
"url": "https://github.com/Picrew/ConStory-Bench/tree/3f4195adda94d977ad9bd56f58ec51eaf1a1cd93",
"note": "candidate evidence excerpts"
},
{
"url": "https://arxiv.org/abs/2304.02769v1",
"note": "candidate evidence excerpts"
},
{
"url": "https://arxiv.org/abs/2209.14958",
"note": "candidate evidence excerpts"
}
]
}Evidence claims
- [sourced_fact] Measured: LLM checkers can find injected contradictions in short stories at roughly human level for the best model and drop toward chance beyond about 1,200-4,000 words (FlawedFictions); a four-stage extract-pair-evidence checker with released prompts found 55% of planted errors in about 8,000-10,000-word stories (ConStory). A 2023 KG+GNN approach on synthetic negation did not beat humans. Proposed only: an explicit fact ledger that is filled from earlier text and queried when new text is written; no inspected paper tests it. This candidate separates the two and defines an experiment. · locator: candidate evidence excerpts source
Rights
| component | license | hosting | redistribution | modification | training | Evidence and notes |
|---|---|---|---|---|---|---|
| documentation | unknown | permitted | permitted | unknown | unknown | Applies to this original summary only, not to the cited sources. |