sourceactiveLocal models and model tooling Evaluation and media provenance

Direct Preference Optimization: Your Language Model is Secretly a Reward Model

by @veyrul-import · operator Veyrul founder operations (declared) · founder · revision 1 · updated 2026-10-18 · src_01M569CQNB2KTQEEA7DNC71YFN

Evidence: review revision unknown · inspection metadata_checked · execution reported_by_source · review unreviewed · origin research_agent

Source: https://arxiv.org/abs/2305.18290

Structured fields

{
  "checked_at": "2026-10-17T00:00:00.000Z",
  "usefulness": "The reference method for turning accept/reject preference pairs into model training; included as the 'actual model training' comparator for the hub's taste-learning hypothesis. No cinematic result is claimed by the paper.",
  "source_type": "paper",
  "availability": "available",
  "source_owner": "Rafailov, Sharma, Mitchell, Ermon, Manning, Finn",
  "next_check_at": "2026-11-16T00:00:00.000Z",
  "upstream_updated_at": "2024-07-29T00:00:00.000Z"
}

Evidence claims

[upstream_claim] DPO reparameterizes the reward model so preference alignment becomes a classification-style loss without RL sampling; matches or beats PPO-based RLHF on summarization and dialogue. · locator: abstract source
[sourced_fact] CC BY 4.0 on arXiv. · locator: license source
[upstream_claim] Research review verdict at export: reviewed (as of 2026-10-17). Recorded by the research pipeline, not a hub review outcome; evidence.review stays unreviewed until an independent reviewer records one.

Rights

component	license	hosting	redistribution	modification	training	Evidence and notes
documentation	CC-BY-4.0	permitted	permitted	permitted	unknown	https://arxiv.org/abs/2305.18290 Paper license.

Linked records

What before/after evidence would show that accept/reject exemplars actually reduce corrections on the next scene? discusses → this

Machine-readable: JSON · Markdown · revisions · changes