EQ-Bench Creative Writing v3
2026-10-12 · Docs and copy
- What it measures
- Creative writing on prompts: character and insight, interest and originality, prose quality, plot and character coherence, following the brief, world and atmosphere, plus avoiding cliches, show-off vocab, and metaphor overload.
- How
- EQ-Bench Creative Writing v3. Thirty-two writing prompts × 3 iterations (96 items), temp 0.7, min_p 0.1. Claude 4 Sonnet grades a rubric absolute score; Claude Sonnet 4.6 then runs pairwise matchups against neighbors (Glicko, with + margins) for relative Elo. Pairwise inputs are truncated to 4000 characters and judged both ways for position bias.
- What the score means
- The leaderboard metric is normalized Elo (higher is better). Relative ratings drift when models are added, so scores are anchored to DeepSeek-R1 = 1500 and ministral-3b = 200. Composite treats it as an arena board: twice the expected win rate vs the leader (leader = 100). Rubric, Slop, Repetition, and Length stay source-side auxiliaries and do not enter this site's composite.
- Caveats
- An LLM judge has self-preference and taste bias; the authors say this is not a roleplay eval and does not match your taste. English only. Source 🆕 marks newly added models, not this site's preliminary half-weight flag.
- Updates
- Source https://eqbench.com/creative_writing.html ; methodology https://eqbench.com/about.html#creative-writing-v3 ; harness https://github.com/EQ-bench/creative-writing-bench . This page fetched all 140 rows on 2026-10-12 from the CSV embedded in creative_writing.js (v=1.0.91), ordered by Elo descending.
Full board
140
| # | Model | Score |
|---|
| 1 | gpt-6-astra | 2173.3 |
| 2 | claude-fable-5-1 | 2162.0 |
| 3 | claude-opus-5 | 2132.6 |
| 4 | gpt-6-sol | 2124.7 |
| 5 | kimi-k3 | 2082.3 |
| 6 | GLM-5.3 | 2075.0 |
| 7 | claude-opus-5-5 | 2050.1 |
| 8 | grok-4.7 | 2006.7 |
| 9 | ?ox-alpha | 1971.7 |
| 10 | gpt-5.6-sol | 1971.6 |
| 11 | claude-fable-5 | 1943.1 |
| 12 | ?aion-3.5 | 1929.3 |
| 13 | muse-spark-1.1 | 1927.1 |
| 14 | ?claude-opus-4-7 | 1914.3 |
| 15 | ?Hemmingway-1 | 1906.4 |
| 16 | muse-spark-1.3 | 1905.9 |
| 17 | gpt-5.6-terra | 1854.9 |
| 18 | gpt-5.5 | 1843.9 |
| 19 | ?Qwen3.8-2.4T-A95B | 1842.5 |
| 20 | muse-spark-1.2 | 1840.4 |