How does Re3 generate a 2,000-word story automatically, and what did its human evaluation and ablations show?
Partially verified: the curator re-read none of the cited sources; S2 rests on the producer's pinned reading. Re3 (Yang et al., EMNLP 2022) has four modules: Plan (setting, characters, three-point outline via GPT3-Instruct-175B), Draft (recursively rebuilt prompt from relevant plan items, summaries and the last passage, GPT3-175B), Rewrite (rerank continuations with trained coherence and relevance classifiers plus heuristics) and Edit (character attribute contradiction detection and correction). Against GPT-3 rolling-window baselines, MTurk raters judged 14.3 and 20.0 points more Re3 stories coherent and premise-relevant; ablations show Plan and Rewrite matter and Edit does not; inter-annotator agreement was low.
Structured fields
{
"citations": [
{
"url": "https://arxiv.org/abs/2210.06774v3",
"note": "candidate evidence excerpts"
}
]
}Evidence claims
- [sourced_fact] Re3 (Yang et al., EMNLP 2022) has four modules: Plan (setting, characters, three-point outline via GPT3-Instruct-175B), Draft (recursively rebuilt prompt from relevant plan items, summaries and the last passage, GPT3-175B), Rewrite (rerank continuations with trained coherence and relevance classifiers plus heuristics) and Edit (character attribute contradiction detection and correction). Against GPT-3 rolling-window baselines, MTurk raters judged 14.3 and 20.0 points more Re3 stories coherent and premise-relevant; ablations show Plan and Rewrite matter and Edit does not; inter-annotator agreement was low. · locator: candidate evidence excerpts source
Rights
| component | license | hosting | redistribution | modification | training | Evidence and notes |
|---|---|---|---|---|---|---|
| documentation | unknown | permitted | permitted | unknown | unknown | Applies to this original summary only, not to the cited sources. |