{"schema":"catalog-search/v2","items":[{"kind":"record","record":{"id":"src_01M6V3Z1ABJQPBYGCF0FRVRMHB","type":"source","title":"lm-evaluation-harness: unified framework for testing generative language models, README overview","summary":"The README says the project provides a unified framework to test generative language models on a large number of different evaluation tasks.","snippet":"README says the project provides a unified framework to <b>test</b> generative <b>language</b> <b>models</b> on a large number of different <b>evaluation</b> tasks. What it supplies: The README says","status":"active","categories":["T20"],"evidence":{"inspection":"source_inspected","execution":"not_run","review":"unreviewed"},"url":"https://veyrul.com/sources/src_01M6V3Z1ABJQPBYGCF0FRVRMHB","api_url":"https://veyrul.com/v1/records/src_01M6V3Z1ABJQPBYGCF0FRVRMHB","updated_at":"2026-11-07T13:42:58.599Z","score":0.9892}},{"kind":"record","record":{"id":"src_01M569CRG5KWSXHGTG80FBSTE5","type":"source","title":"LoRA: Low-Rank Adaptation of Large Language Models","summary":null,"snippet":"The foundational paper for LoRA, the adapter method behind most","status":"active","categories":["T09"],"evidence":{"inspection":"metadata_checked","execution":"not_run","review":"unreviewed"},"url":"https://veyrul.com/sources/src_01M569CRG5KWSXHGTG80FBSTE5","api_url":"https://veyrul.com/v1/records/src_01M569CRG5KWSXHGTG80FBSTE5","updated_at":"2026-10-18T01:18:06.328Z","score":0.0174}},{"kind":"record","record":{"id":"src_01M6W7JNQ4P7SJT7P69SW752JM","type":"source","title":"DSPy: framework for programming rather than prompting language models, README overview","summary":"The README presents DSPy as a way to build LM systems from Python modules and let optimizers tune prompts and weights.","snippet":"README presents DSPy as a way to build <b>LM</b> systems from Python modules and let optimizers tune prompts and weights. What it supplies: The README presents DSPy","status":"active","categories":["T09"],"evidence":{"inspection":"source_inspected","execution":"not_run","review":"unreviewed"},"url":"https://veyrul.com/sources/src_01M6W7JNQ4P7SJT7P69SW752JM","api_url":"https://veyrul.com/v1/records/src_01M6W7JNQ4P7SJT7P69SW752JM","updated_at":"2026-11-08T00:05:22.242Z","score":0.0187}},{"kind":"record","record":{"id":"exp_01M64GTAY57GSANKA2AY3W981B","type":"experiment","title":"Seedance 2.5 style test: painterly period battle with powder smoke (20 s, 21:9)","summary":"Wide painterly battle: massed ranks, cannon smoke, commanders at a tent, a cavalry charge, a river crossing and a closing wide of the commander.","snippet":"Wide painterly battle: massed ranks, cannon <b>smoke</b>, commanders at a tent, a cavalry charge, a river crossing and a closing wide of the commander. A dignified period battle","status":"completed","categories":["T06","T02"],"evidence":{"inspection":"source_inspected","execution":"reported_by_source","review":"unreviewed"},"url":"https://veyrul.com/experiments/exp_01M64GTAY57GSANKA2AY3W981B","api_url":"https://veyrul.com/v1/records/exp_01M64GTAY57GSANKA2AY3W981B","updated_at":"2026-10-29T22:15:56.927Z","score":0.0598}},{"kind":"record","record":{"id":"ast_01M64VQTTPW79B3AKCXJMKCX0J","type":"asset","title":"Seedance 2.5 style test: painterly period battle with powder smoke (20 s, 21:9) — original video","summary":"Founder-cleared historical Higgsfield output. Original video available; audio has not been listened to.","snippet":"historical video from the founder Higgsfield account. Public use authorized on 2026-10-29. <b>Sampled</b>-frame observations: https://veyrul.com/experiments/exp_01M64GTAY57GSANKA2AY3W981B. Audio has not been listened to. This","status":"active","categories":["T06","T02"],"evidence":{"inspection":"metadata_checked","execution":"reported_by_source","review":"unreviewed"},"url":"https://veyrul.com/assets/ast_01M64VQTTPW79B3AKCXJMKCX0J","api_url":"https://veyrul.com/v1/records/ast_01M64VQTTPW79B3AKCXJMKCX0J","updated_at":"2026-10-29T22:17:28.580Z","score":0.0223}},{"kind":"record","record":{"id":"exp_01M64VFCD9DM37B53NZTQNGD96","type":"experiment","title":"Seedance 2.5 sequence run with one linked image: 20 s dusty slope, a rolling bundle, and two close faces with red marks","summary":"A dusty sepia slope with a large bundle of rope and bone and two running figures: the bundle rolls and a figure falls, dust fills the frame, a close of a pale woman with wild hair and red marks on her cheeks, a close of a bearded man with red marks, and two standing figures in the dust.","snippet":"dust. Render a rope snapping on a dusty loading slope, two people in hide <b>harness</b> as a charcoal and bone look","status":"completed","categories":["T06","T02"],"evidence":{"inspection":"source_inspected","execution":"reported_by_source","review":"unreviewed"},"url":"https://veyrul.com/experiments/exp_01M64VFCD9DM37B53NZTQNGD96","api_url":"https://veyrul.com/v1/records/exp_01M64VFCD9DM37B53NZTQNGD96","updated_at":"2026-10-29T22:48:32.395Z","score":0.015}},{"kind":"record","record":{"id":"exp_01M64V6P74BBZ33W2HDN7TFPPV","type":"experiment","title":"Seedance 2.5 reference-anchored test: 20 s gilded airship flagship at sunrise with a deck battle","summary":"A cloud-sea sunrise with an airship, a crowded deck with cannons and barrels, a cannon firing, smoke, a woman with a weapon on a gilded vessel and a gun tower firing in sparks.","snippet":"sunrise with an airship, a crowded deck with cannons and barrels, a cannon firing, <b>smoke</b>, a woman with a weapon on a gilded vessel and a gun tower firing","status":"completed","categories":["T06","T02"],"evidence":{"inspection":"source_inspected","execution":"reported_by_source","review":"unreviewed"},"url":"https://veyrul.com/experiments/exp_01M64V6P74BBZ33W2HDN7TFPPV","api_url":"https://veyrul.com/v1/records/exp_01M64V6P74BBZ33W2HDN7TFPPV","updated_at":"2026-10-29T22:24:39.671Z","score":0.0283}},{"kind":"record","record":{"id":"src_01M66Q662445VWT7JK03WB0S11","type":"source","title":"Krea2 Harness (ComfyUI registry node ComfyUI-Krea2-Harness)","summary":"ComfyUI node registry custom node pack ComfyUI-Krea2-Harness at revision 3a6470361978: latest version 0.1.261015; publisher ane5s; declared licence {\"file\": \"LICENSE\"}; repository https://github.com/ANe5s/ComfyUI-Krea2-Harness. Metadata-only index entry; not tested guidance.","snippet":"ComfyUI node registry custom node pack ComfyUI-Krea2-<b>Harness</b> at revision 3a6470361978: latest version 0.1.261015; publisher ane5s; declared licence {\"file\": \"LICENSE\"}; repository https://github.com/ANe5s/ComfyUI-Krea2-Harness. Metadata-only index entry","status":"active","categories":["T08"],"evidence":{"inspection":"metadata_checked","execution":"not_run","review":"unreviewed"},"url":"https://veyrul.com/sources/src_01M66Q662445VWT7JK03WB0S11","api_url":"https://veyrul.com/v1/records/src_01M66Q662445VWT7JK03WB0S11","updated_at":"2026-10-30T15:34:52.689Z","score":0.0147}},{"kind":"record","record":{"id":"exp_01M64V5ZSF7E0PZMPKZ4DM18QK","type":"experiment","title":"Seedance 2.5 reference-anchored test: 20 s golden-hour ranch, a woman and a pale horse at a gate","summary":"Golden prairie: a barn and mountains, a woman in a hat walking with chimney smoke, a horse over a gate, a hand on its cheek, the horse bolting through dust and a wide ranch view.","snippet":"Golden prairie: a barn and mountains, a woman in a hat walking with chimney <b>smoke</b>, a horse over a gate, a hand on its cheek, the horse bolting through","status":"completed","categories":["T06","T02"],"evidence":{"inspection":"source_inspected","execution":"reported_by_source","review":"unreviewed"},"url":"https://veyrul.com/experiments/exp_01M64V5ZSF7E0PZMPKZ4DM18QK","api_url":"https://veyrul.com/v1/records/exp_01M64V5ZSF7E0PZMPKZ4DM18QK","updated_at":"2026-10-29T22:23:29.875Z","score":0.0275}},{"kind":"record","record":{"id":"exp_01M64VF5GF2CRDYGG4R5RJ8R5X","type":"experiment","title":"Seedance 2.5 sequence run with one linked image: 20 s brass pipe corridor, a steam blast and two people crawling","summary":"A brass-pipe corridor in steam: two figures in long coats walking, one thrown back by a steam jet and pinned on the floor, a close of a dark-haired woman in steam, a close of a man in a brass respirator, and two figures crawling and huddled on a grating.","snippet":"blowing in a brass pump hall, two people crawling along a grating as a <b>smoke</b> matte painting look","status":"completed","categories":["T06","T02"],"evidence":{"inspection":"source_inspected","execution":"reported_by_source","review":"unreviewed"},"url":"https://veyrul.com/experiments/exp_01M64VF5GF2CRDYGG4R5RJ8R5X","api_url":"https://veyrul.com/v1/records/exp_01M64VF5GF2CRDYGG4R5RJ8R5X","updated_at":"2026-10-29T22:48:22.437Z","score":0.0069}},{"kind":"record","record":{"id":"exp_01M64V5YDGYQPGDEJQWF67W04S","type":"experiment","title":"Seedance 2.5 reference-anchored test: 20 s night aerial over a burning plain to lit streets","summary":"Charcoal terrain with ember fires and smoke, craters, small searchlight beams, a lit road grid and lit streets in the last frames.","snippet":"Charcoal terrain with ember fires and <b>smoke</b>, craters, small searchlight beams, a lit road grid and lit streets in the last frames. A night aerial over a vast wounded","status":"completed","categories":["T06","T02"],"evidence":{"inspection":"source_inspected","execution":"reported_by_source","review":"unreviewed"},"url":"https://veyrul.com/experiments/exp_01M64V5YDGYQPGDEJQWF67W04S","api_url":"https://veyrul.com/v1/records/exp_01M64V5YDGYQPGDEJQWF67W04S","updated_at":"2026-10-29T22:23:40.663Z","score":0.0133}},{"kind":"record","record":{"id":"exp_01M64VF2RBEC0Z6JGH9KJWBK3E","type":"experiment","title":"Seedance 2.5 sequence run with one linked image: 20 s misty flooded hall, a red lamp passed between two crouched figures","summary":"A misty pillared hall with a flooded floor: a blurred figure lunging beside a red lamp, two figures in a splash and then crouched with the lamp, a close of a long-haired woman with bared teeth, a close of a pale lined-faced man, and a closing two-shot with the lamp passed.","snippet":"adults in a flooded ruined hall who pass a small hand-lamp as a <b>smoke</b> matte painting look","status":"completed","categories":["T06","T02"],"evidence":{"inspection":"source_inspected","execution":"reported_by_source","review":"unreviewed"},"url":"https://veyrul.com/experiments/exp_01M64VF2RBEC0Z6JGH9KJWBK3E","api_url":"https://veyrul.com/v1/records/exp_01M64VF2RBEC0Z6JGH9KJWBK3E","updated_at":"2026-10-29T22:48:22.271Z","score":0.0168}},{"kind":"record","record":{"id":"src_01M6JQBD24ACRNMG25E0TKZR19","type":"source","title":"harness-hub/harnesses-analysis","summary":"Hugging Face Space harness-hub/harnesses-analysis at revision c8c2d25c0ee7: SDK static; declared licence not declared; access gating not reported. Metadata-only index entry; not tested guidance.","snippet":"Hugging Face Space <b>harness</b>-hub/harnesses-analysis at revision c8c2d25c0ee7: SDK static; declared licence not declared; access gating not reported. Metadata-only index entry; not <b>tested</b> guidance. declared tags (declared","status":"active","categories":["T09"],"evidence":{"inspection":"metadata_checked","execution":"not_run","review":"unreviewed"},"url":"https://veyrul.com/sources/src_01M6JQBD24ACRNMG25E0TKZR19","api_url":"https://veyrul.com/v1/records/src_01M6JQBD24ACRNMG25E0TKZR19","updated_at":"2026-11-04T07:28:36.896Z","score":0.0132}},{"kind":"record","record":{"id":"exp_01M64V3408CYTDTJXPFSMKK72T","type":"experiment","title":"Seedance 2.5 style test: 1940s noir office with a veiled visitor and a photograph (20 s)","summary":"Black-and-white noir: a detective at a desk under a fan, a veiled woman in a doorway, a photograph held in close-up, venetian-blind shadows across her face, a file drawer and cigarette smoke.","snippet":"close-up, venetian-blind shadows across her face, a file drawer and cigarette <b>smoke</b>. A two-hander noir scene built around a photograph","status":"completed","categories":["T06","T02"],"evidence":{"inspection":"source_inspected","execution":"reported_by_source","review":"unreviewed"},"url":"https://veyrul.com/experiments/exp_01M64V3408CYTDTJXPFSMKK72T","api_url":"https://veyrul.com/v1/records/exp_01M64V3408CYTDTJXPFSMKK72T","updated_at":"2026-10-29T22:18:15.514Z","score":0.0131}},{"kind":"record","record":{"id":"exp_01M64VBJS3J9RQJZQ6XA7KAHN6","type":"experiment","title":"Seedance 2.5 text-only style test: 20 s park match with a girl in a white headscarf kit in a flat two-tone look","summary":"A lit park pitch at dusk: families on rugs, a man in grey, a girl in white sliding into a play, a kick, a cheering crowd and a hug.","snippet":"A lit park pitch at dusk: families on rugs, a","status":"completed","categories":["T06","T02"],"evidence":{"inspection":"source_inspected","execution":"reported_by_source","review":"unreviewed"},"url":"https://veyrul.com/experiments/exp_01M64VBJS3J9RQJZQ6XA7KAHN6","api_url":"https://veyrul.com/v1/records/exp_01M64VBJS3J9RQJZQ6XA7KAHN6","updated_at":"2026-10-29T22:35:54.501Z","score":0.0062}},{"kind":"record","record":{"id":"dsc_01M569J94B42C5Z2TVZBFHPYVH","type":"discussion","title":"Can evaluation keep continuity, composition, lighting intent, camera feasibility, aesthetic preference and narrative purpose as separate dimensions without collapsing into a leaderboard?","summary":null,"snippet":"VBench's 16 dimensions are visual; TTCW's 14 <b>tests</b> are textual; ShotBench's 8 are perceptual. None includes 'does this shot serve the scene's purpose'. What rubric","status":"open","categories":["T20"],"evidence":{"inspection":"discovered","execution":"not_run","review":"unreviewed"},"url":"https://veyrul.com/discussions/dsc_01M569J94B42C5Z2TVZBFHPYVH","api_url":"https://veyrul.com/v1/records/dsc_01M569J94B42C5Z2TVZBFHPYVH","updated_at":"2026-10-18T01:21:07.202Z","score":0.0038}},{"kind":"record","record":{"id":"src_01M6V885CD1WJ8ZQ3D1NMW05Y6","type":"source","title":"MLX LM: text generation and fine-tuning on Apple silicon, README overview","summary":"The README presents MLX LM as a Python package for running and fine-tuning LLMs on Apple silicon with MLX.","snippet":"README presents MLX <b>LM</b> as a Python package for <b>running</b> and fine-tuning LLMs on Apple silicon with MLX. What it supplies: The README presents MLX <b>LM</b>","status":"active","categories":["T09"],"evidence":{"inspection":"source_inspected","execution":"not_run","review":"unreviewed"},"url":"https://veyrul.com/sources/src_01M6V885CD1WJ8ZQ3D1NMW05Y6","api_url":"https://veyrul.com/v1/records/src_01M6V885CD1WJ8ZQ3D1NMW05Y6","updated_at":"2026-11-07T14:57:51.976Z","score":0.0129}},{"kind":"record","record":{"id":"src_01M66FBXWAV115GYX33Q3JK3VG","type":"source","title":"pico-lm/pico-decoder-large","summary":"Hugging Face model pico-lm/pico-decoder-large at revision dc1a407c6f5e: text-generation; declared licence apache-2.0; not gated. Metadata-only index entry; not tested guidance.","snippet":"<b>tested</b> guidance. declared lineage (declared by the uploader, not verified): trained or <b>evaluated</b> on datasets pico-<b>lm</b>/pretokenized-dolma\ncreated at: 2025-02-13T07:12:43.000Z\ngated","status":"active","categories":["T09"],"evidence":{"inspection":"metadata_checked","execution":"not_run","review":"unreviewed"},"url":"https://veyrul.com/sources/src_01M66FBXWAV115GYX33Q3JK3VG","api_url":"https://veyrul.com/v1/records/src_01M66FBXWAV115GYX33Q3JK3VG","updated_at":"2026-10-30T13:18:12.309Z","score":0.0104}},{"kind":"record","record":{"id":"exp_01M64VBMTKC9QER3SKB06JKVY0","type":"experiment","title":"Seedance 2.5 text-only look test: 20 s man before a vast crowd and a haloed white face, photoreal dark fantasy","summary":"A pale floor with a lit stair and a distant crowd: a man reaching toward camera, a white statue-like face with a ring halo, two raised fingers, a smoke column and the crowd close.","snippet":"camera, a white statue-like face with a ring halo, two raised fingers, a <b>smoke</b> column and the crowd close. Render a man addressing a vast crowd","status":"completed","categories":["T06","T02"],"evidence":{"inspection":"source_inspected","execution":"reported_by_source","review":"unreviewed"},"url":"https://veyrul.com/experiments/exp_01M64VBMTKC9QER3SKB06JKVY0","api_url":"https://veyrul.com/v1/records/exp_01M64VBMTKC9QER3SKB06JKVY0","updated_at":"2026-10-29T22:41:05.542Z","score":0.013}},{"kind":"record","record":{"id":"exp_01M64V4442PWJ88NXVANCY8CG9","type":"experiment","title":"Seedance 2.5 style test: phalanx on a shore before a burning walled city (20 s)","summary":"Bronze-and-sea epic: a burning fortress on a headland, a front-rank helmeted soldier, ranks of shields along the shoreline, a walled city with a siege engine and a gate with a fireball.","snippet":"Bronze-and-sea epic: a burning fortress on a headland","status":"completed","categories":["T06","T02"],"evidence":{"inspection":"source_inspected","execution":"reported_by_source","review":"unreviewed"},"url":"https://veyrul.com/experiments/exp_01M64V4442PWJ88NXVANCY8CG9","api_url":"https://veyrul.com/v1/records/exp_01M64V4442PWJ88NXVANCY8CG9","updated_at":"2026-10-29T22:19:41.736Z","score":0.0058}},{"kind":"guide_preview","preview":{"schema":"guide-preview/v1","id":"art_01M6TX698DX3HMKM77XQP8WJA6","type":"article","revision":1,"title":"Evaluate a language model with lm-evaluation-harness: validate, smoke test, run and keep samples","coverage":"For people who want to score a language model on a named benchmark from the command line and be able to audit the result. You end with a scored run and saved per-example inputs and outputs. Based on the project's interface documentation and not run.","example":"Hypothetical, not run: a 20-example smoke test and then a full hellaswag evaluation of gpt2, with samples saved for inspection.","tips":["Never report a score from a run that used the example limit; the documentation marks it as for testing only.","Open a few saved samples and read the prompt and model output yourself; a wrongly formatted prompt makes the score meaningless.","Only enable the unsafe-code or remote-code switches for task sources you trust."],"categories":["T20","T09"],"tags":["lm-eval","lm-evaluation-harness","model-evaluation","benchmarks","hellaswag","local-llm","reproducibility"],"evidence":{"inspection":"source_inspected","execution":"not_run","review":"unreviewed","review_revision":null},"media":[]}}],"truncated":true,"has_more":true,"next_cursor":"eyJyZWZlcmVuY2UiOiJNakEiLCJwcmV2aWV3IjpudWxsLCJxIjoiRXZhbHVhdGUgYSBsYW5ndWFnZSBtb2RlbCB3aXRoIGxtLWV2YWx1YXRpb24taGFybmVzczogdmFsaWRhdGUsIHNtb2tlIHRlc3QsIHJ1biBhbmQga2VlcCBzYW1wbGVzIiwidHlwZXMiOltdLCJjYXRlZ29yaWVzIjpbXX0","ordering":"bounded_reference_and_preview_lanes"}