Two Moves Are Better Than Ten

What a tiny Stockfish hint reveals about LLM chess

With no candidate help, Luna struggled against a 1320-rated Stockfish opponent. Given ten strong moves to consider, it still struggled. Given only two, the combined system held its own against Stockfish at 2820.

That does not mean an unaided language model became a grandmaster. It means the system around the model changed what the model was capable of expressing. In our first exploratory ladder, that change was worth more than 1,600 points of local, protocol-specific Elo.

The shape of the result was stranger than the size. Five candidates helped. Three helped much more. Two were strongest. Ten barely moved the needle.

The experiment began as a practical question—how much can a better harness help an LLM play chess?—and ended by opening a deeper one:

Is the model missing good ideas, or does it fail when it has too many ideas to compare?

A chess move is several problems disguised as one

To make a strong move from a board, an agent must do at least five things:

1. reconstruct the current state;
2. generate plausible actions;
3. compare those actions;
4. calculate relevant replies;
5. evaluate the resulting positions.

A normal game score collapses all five into a win, loss, or draw. If a model loses, we do not know whether it never considered the right move, considered it and rejected it, hallucinated the board, stopped calculating too early, or misjudged the final position.

Our candidate harness intervenes between generation and selection. Stockfish produces a small set of strong legal moves. Their order is shuffled. The model sees UCI and SAN notation, but no rank, score, or hint about which move is best. It can choose one of the suggestions or ignore them and play any legal move.

That turns one opaque question—“Can the model play chess?”—into a more precise one: “What happens when good proposals are available but the model must still decide?”

Why we played complete games

Chess is played from move one to the end. A position benchmark can tell us whether a model finds a tactic in a frozen diagram. A complete game tests whether the agent can maintain state, live with its earlier inaccuracies, preserve a plan, manage a growing history, and convert whatever position it creates.

Every game in this ladder began from the standard initial position. The LLM played Black and made every Black move. Stockfish played White in rated LimitStrength mode. Games continued for as many as 200 plies.

This gives the result ecological validity, but it also creates a limitation. Once two conditions choose different moves, they enter different positions. Full games tell us whether a harness works end to end; they do not, by themselves, tell us why.

The protocol

Dimension	Setting
Model	openai/gpt-5.6-luna
Model color	Black
Opponent	Stockfish 18, UCI rated mode
Candidate analysis	0.1 seconds per move
Candidate conditions	0, 10, 5, 3, or 2 moves
Candidate presentation	Randomized UCI + SAN
Candidate scores/ranks	Hidden
Reasoning effort	High
Maximum game length	200 plies
Anchor block	10 clean games
Provider errors	Excluded and reported separately

The ladder increased Stockfish strength when the model scored strongly and repeated an anchor when the score was near the informative middle. The rating calculation used game score—wins plus half of draws—and a 35-point correction for playing Black.

These choices define a local rating protocol. The results are not FIDE Elo, chess.com Elo, Lichess Elo, or the upstream llm_chess Dragon leaderboard.

The result

Condition	Tested anchors	Combined clean W-D-L	Joint local Elo (profile 95%)
Regular	1320	0-5-5 (N=10)	1067 [705, 1351]
Top 10	1320	2-2-6 (N=10)	1127 [805, 1407]
Top 5	1320-1920	28-14-18 (N=60)	1830 [1713, 1950]
Top 3	1320-2520	43-22-15 (N=80)	2431 [2317, 2548]
Top 2	1320-2820	41-11-8 (N=60)	2832 [2680, 2992]

Figure — Luna candidate-count rating and token volume

The token panel is descriptive rather than budget-matched: shorter games and different terminal reasons affect its denominator. It nevertheless captures an important systems observation from the completed pilot. Top-2 did not buy its strength by simply emitting the most tokens; it was the lowest-token condition on the all-game average as well as the strongest fitted condition.

The joint estimates use every anchor for a condition; brackets are profile 95% intervals. The exact numbers deserve less attention than the curve. If candidate help were simply an engine-strength dial, ten strong candidates should not be almost useless while two are transformative. The nominal source engine and wall-clock budget were held fixed, but the realized sets were not guaranteed to be nested. Candidate width was therefore not the only realized difference.

The strongest system did not receive the engine's evaluation. It received two unranked proposals and still had to interpret the position and commit. This is evidence that Luna can use a narrow oracle proposal scaffold in this protocol; it is not yet evidence that unaided Luna contains an isolated latent critic or particular amount of chess judgment.

It also suggests that a long list of plausible moves is not equivalent to useful help.

Four explanations are still alive

1. Selection overload

Comparing several close, plausible moves may be harder than judging whether one of two moves survives tactical and strategic scrutiny.

2. Attention dilution

The larger candidate block may consume attention that would otherwise represent the board, calculate replies, or preserve the plan.

3. Search interference

Each plausible move opens a line of analysis. Ten good moves may induce shallow, inconsistent exploration, while two permit deeper comparison.

4. Sampling or protocol artifact

The blocks are small, adaptive, Black-only, and drawn from a single model and candidate seed. The apparent curve may be exaggerated by correlated openings, game trajectories, or ordinary variance.

The current evidence shows that the system changed. It does not yet prove which mechanism caused the change.

There is another important pilot limitation: each candidate-count condition ran a separate time-limited MultiPV search. The sets came from the same engine and budget, but top two is not guaranteed to be an exact prefix of top ten. The confirmatory experiment will generate one deterministic max-k set, cache it, and derive every smaller set as a nested prefix. That change is essential before we attribute the curve to bandwidth alone.

What this result does not prove

• Luna does not independently generate 2800-strength moves.
• Top two is not established as the universally optimal candidate count.
• The candidate-count curve has not yet been replicated across colors and model families.
• Full games do not separate proposal quality from comparison difficulty.
• Ten-game blocks do not justify narrow confidence in a point estimate.
• The local ratings are not directly comparable to the public Dragon leaderboard.

The accurate headline is deliberately longer:

Luna with a top-two Stockfish policy oracle scored near 2803 in this local, Black-only full-game protocol.

All-anchor analysis update: A later joint Davidson W-D-L fit over all 220 frozen Luna games estimates regular near 1067 (profile 95% interval 705–1351) and top-2 near 2832 (2680–2992). The original 1164 and 2803 values above are single-anchor point-score transforms retained to document the exploratory chronology. Post three explains why the estimators differ.

The harness is part of the player.

The replication is already uncomfortable

We began the same five conditions with DeepSeek V4 Flash. The runs immediately exposed another systems lesson. The model is inexpensive per token, but high reasoning inside a multi-turn chess agent can generate hundreds of thousands of completion-side tokens per game, including provider-reported reasoning usage. The cohort accumulated well over one hundred worker-hours while several first-anchor blocks were still incomplete.

“Flash” pricing did not imply flash agentic latency. Tokens, tool turns, wall time, and parallelism are part of the architecture, not bookkeeping footnotes.

DeepSeek's final curve will be reported only after its anchors settle. Provisional perfect scores will not be converted into fake finite Elo estimates.

The experiment that matters next

The next study will hold the position and candidate quality fixed while varying what the model must compare.

On the same positions, in repeated randomized orders, the model will receive:

• no candidates;
• all legal moves;
• its own generated candidates;
• two, three, five, or ten engine moves;
• the best move plus obvious losers;
• the best move plus plausible tactical traps;
• strong candidates with the engine's best move omitted;
• successor boards instead of move notation;
• the opponent's best reply after each candidate.

We will measure best-move inclusion, selected rank, centipawn and WDL regret, catastrophic blunders, order sensitivity, confidence, latency, tokens, and cost.

If candidate count still matters after quality, ambiguity, and order are controlled, the “too many good ideas” hypothesis becomes real. If successor boards or opponent replies erase the effect, the bottleneck is more likely grounded search. If the model's own candidate recall is poor but its oracle-set selection is strong, policy generation becomes the target.

From leaderboard to laboratory

The initial question was whether we could beat a chess leaderboard by improving the harness. The more valuable question is now whether chess can reveal the division of labor inside an intelligent system.

The model may be a weak proposer and a strong critic. It may have a useful value function without a reliable search process. It may calculate well only when the state transition is externalized. It may reason effectively until the game history overwhelms its board representation.

Each possibility implies a different architecture.

That is the program from here: use full games to discover consequential system effects, use paired positions to explain them, and then use the explanation to build a better reasoner.

---

Reproducibility note

The raw game JSON, PGNs, run metadata, clean-game rules, block summaries, token counts, provider errors, budget corrections, and append-only status chronology are retained in the repository. The Luna cohort is content-addressed and frozen, and the joint rating release includes profile uncertainty plus color-correction sensitivity. The exploratory raw logs remain outside the publication bundle; released tables and figures name the frozen content set they consume.