The Candidate-Set Paradox

When more strong options make a reasoning system weaker

More information should help.

If a chess engine offers ten strong legal moves, that list contains at least as many useful ideas as a list of two. A capable reasoner can ignore the extra options. At worst, the additional moves should be harmless.

Our Luna pilot produced the opposite shape.

Condition	Games	Combined W-D-L	Joint local Elo	Profile 95%
No candidates	10	0–5–5	1067	705–1351
Top 10	10	2–2–6	1127	805–1407
Top 5	60	28–14–18	1830	1713–1950
Top 3	80	43–22–15	2431	2317–2548
Top 2	60	41–11–8	2832	2680–2992

Descriptively, top-2 finished 1,705 local Elo above top-10 and 1,765 above unaided play. Top-3 finished 601 points above top-5. The progression is not monotonic in information volume.

That is the candidate-set paradox: a larger set of strong proposals can produce a weaker complete system.

It is also, at this stage, a paradox with several possible explanations and no licensed causal headline.

Why the obvious explanation is seductive

The natural story is choice overload. Luna may possess useful comparative chess judgment but only across a narrow consideration set. Two moves permit focused calculation. Ten moves invite shallow commentary on every option, leaving too little depth to refute any one line.

That story fits the observed curve and a familiar experience with language models: generating many plausible alternatives is easy; disciplined elimination is hard.

But a curve that looks like overload is not proof of overload. Changing k changes several properties simultaneously:

• probability that an excellent move appears;
• average quality of the weakest candidate;
• number of comparisons required;
• prompt length and visual density;
• number of reply trees invited;
• value ambiguity among plausible choices;
• realized Stockfish search under a time limit; and
• downstream positions reached in full games.

“Number of moves” is a bundle, not yet an isolated treatment.

The pilot effect is a systems effect

The top-2 system combines two components:

1. Stockfish proposes a tiny high-quality policy support.
2. Luna interprets the position, compares the proposals, and commits.

The 2832 estimate belongs to that pair. It does not imply that Luna would have generated the same moves unaided. Nor does it imply that blindly choosing the first engine move would produce the same games: ranks and scores were hidden, order was randomized, and the model could reject every suggestion.

Candidate assistance intervenes specifically between proposal generation and selection. Its success therefore suggests a decomposition:

board state → candidate policy → comparative search → value judgment → move

Unaided play asks one model to perform the entire chain. The harness externalizes part of the policy. The remaining curve tells us something about the capacity of the comparison and value stages—but only after candidate construction is held fixed.

Five live explanations

1. Comparison overload

The model may have a limited comparative workspace. With two moves, it can calculate a reply to each and make a direct contrast. With ten, it may substitute verbal plausibility for search.

Predictions:

• regret increases with k even when the best move and candidate values are controlled;
• order instability grows with list size;
• analysis mentions more candidates but reaches less depth per candidate; and
• extra reasoning tokens do not fully repair the effect.

2. Attention dilution

The candidate block may crowd out the board or game history. The failure would then be representational rather than comparative: the model loses access to the state while reading the options.

Predictions:

• length-matched irrelevant or duplicate candidates also hurt;
• compact identifiers outperform verbose UCI-plus-SAN rendering;
• repeating the board after the candidate list repairs performance; and
• successor boards help mainly because they refresh state, not because they add search.

3. Search interference

Every plausible move opens a separate reply tree. A language model may begin several lines, fail to finish them, and merge conclusions across branches. Two options constrain search breadth enough to permit deeper refutation.

Predictions:

• showing the opponent's best reply narrows or eliminates the top-10 penalty;
• short principal variations outperform move-only lists at larger k;
• the effect is strongest in tactical positions; and
• explicit branch-by-branch scratch space reduces cross-line contamination.

4. Candidate realization

The exploratory harness ran a separate 0.1-second MultiPV search for every condition. Top-2 is therefore not guaranteed to be the literal first two moves of top-10. Concurrent CPU load can also change a wall-clock search.

Predictions:

• the curve shrinks when every condition is derived from one cached fixed-node max-10 analysis;
• realized candidate values explain much of the between-condition difference; or
• a rerun with a different candidate seed changes the ordering substantially.

5. Trajectory and sampling noise

The blocks are small, Black-only, adaptively allocated, and begin from one position. Once conditions choose different moves, they face different games.

Predictions:

• the ordering weakens on identical paired positions;
• color reversal or opening randomization changes the curve; or
• replication across model snapshots produces materially different optima.

All five explanations are compatible with the initial full-game result.

Coverage and burden pull in opposite directions

A useful conceptual model separates two functions.

Coverage(k) is the probability that the candidate set contains a move the model could use to achieve low regret. Coverage usually rises with k.

SelectionCost(k) is the probability that comparison, attention, or search fails because the set is difficult to process. Selection cost may also rise with k.

System quality depends on both:

useful assistance(k) ≈ coverage(k) − selection cost(k)

The optimum is not necessarily the smallest list. If top-1 exposes the engine's answer directly, the model contributes almost no selection. If top-2 omits a critical alternative under a weaker proposer, coverage may be inadequate. If top-20 overwhelms the critic, breadth becomes counterproductive.

The “right k” is therefore a property of the proposer, critic, position, and interface—not a universal constant.

Value gap matters more than count alone

Consider two top-5 sets.

In the first, one move is clearly winning and four are obvious errors. The task is detection.

In the second, five moves are positionally plausible, three are nearly equal, and one contains a delayed tactical defect. The task is comparative search.

Both prompts contain five moves, but they measure different difficulty. A clean candidate experiment must register:

• best-versus-second value gap;
• number of candidates within a narrow WDL band;
• presence of superficial tactical temptations;
• delayed poison and required refutation depth;
• whether the best move is omitted; and
• source rank of the selected move.

Without these variables, a k curve mixes list width with the composition of the list.

The confirmatory experiment

E020/E021 is registered before paid paired-position observations. It freezes 600 positions balanced across tactical, ambiguous, and quiet strata; opening, middlegame, and endgame phases; and White and Black to move.

For each position:

1. Stockfish 18 performs one fixed-node max-10 analysis.
2. Top-2, top-3, top-5, and top-10 are exact ranked prefixes.
3. Model-facing lists hide ranks, scores, and principal variations.
4. Presentation order is reproducible; candidate-bearing cells have two total orders on the 200 ambiguous positions in locked v1.
5. Every condition receives a fresh model context.

The core conditions are:

• naked position;
• model-generated shortlist;
• oracle top-2;
• oracle top-5;
• oracle top-10; and
• top-5 with the best move omitted.

Primary outcomes are WDL regret and catastrophic-blunder rate. Secondary outcomes include best-move selection, source rank, legality, order instability, latency, tokens, and self-generated best-move recall.

Illegal or malformed moves remain model failures. Provider errors are excluded from chess-quality endpoints, reported separately, and resampled only to restore the registered paired cell.

The position builder reconstructs 27,284 retained pilot position occurrences, which collapse to 25,621 unique normalized keys, and excludes them from an deterministic project-generated positions whose source-game lineage is disjoint from the pilot. The frozen 600-position artifact is a composition-audited feasibility substrate, not the final candidate pool; heavy fixed-node labeling of a separately frozen overcomplete pool remains pending until the live evaluation suites release the host.

The count of 600 is a fixed collection target, not proof of power for every interesting effect. Under the frozen conservative two-test precision envelope, the 80%-power minimum detectable WDL difference ranges from about 0.0126 when the paired SD is 0.10 to 0.0503 when it is 0.40. Detecting 0.02 at that planning power requires paired SD at or below about 0.159, or a prospectively versioned extension; the unknown variance cannot be replaced after the fact by a favorable observed effect.

Tests that distinguish the mechanisms

The basic k curve is only the beginning. Several crossed interventions make the hypotheses disagree.

Intervention	Overload	Attention dilution	Search interference
Compact candidate notation	Small effect	Large repair	Small effect
Repeat board after list	Small effect	Large repair	Possible repair
Show opponent best reply	Moderate repair	Small effect	Large repair
Length-matched duplicates	Count-dependent	Length-dependent harm	Little new branching
Independent scratch branch per move	Large repair	Mixed	Large repair
Shuffle identical set	Higher instability at large k	Mixed	Higher branch instability

The goal is not to attach a fashionable label to a surprising chart. It is to make the explanations produce different predictions.

The efficiency clue

The Luna trajectory data contains another descriptive signal. Median total tokens per game were 326,245 unaided, 95,570 with top-10, 127,662 with top-5, 101,295 with top-3, and 59,563 with top-2.

These values pool different opponent anchors and game lengths, so they do not show that smaller lists inherently use fewer tokens. They do show that strength, trajectory, and inference cost moved together in the realized systems. The strongest scaffold was not the one producing the longest traces.

That possibility matters beyond chess: better constraints can increase both accuracy and efficiency by preventing unproductive search.

What would change our mind?

The candidate-overload interpretation fails if nested paired sets eliminate the top-10 disadvantage. It weakens if candidate value realization explains the curve, if the ordering does not replicate across colors, or if larger lists perform equally after prompt length is controlled.

The broader proposal-versus-selection decomposition fails if Luna cannot choose strongly from oracle sets on held-out positions, or if its self-generated shortlists already contain the engine-best move at the same rate.

Top-2 is not sacred. Another model may peak at five. Tactical positions may prefer one. Quiet positions may benefit from breadth. A better compact policy model may make ten useful. Those differences would make candidate curves more interesting, not less.

From paradox to architecture

If the effect survives, the design lesson is not “always show two moves.” It is that proposal bandwidth should adapt to critic capacity and position uncertainty.

A chess-native reasoner could:

1. generate a compact policy support;
2. estimate uncertainty and candidate value gaps;
3. widen the set only when coverage is doubtful;
4. allocate separate search to each surviving branch;
5. compare calibrated successor values; and
6. request deeper engine or symbolic help when confidence remains low.

That architecture treats candidate count as a control variable. The system spends breadth where breadth adds information and depth where comparison is the bottleneck.

The pilot gave us a striking curve. The laboratory's job is to turn that curve into a mechanism.

---

Reproducibility note

The table and descriptive contrasts are generated from the frozen Luna cohort by research/analysis/candidate_curve_summary.py, joining the joint Davidson rating release with the trajectory summary. The source explicitly marks every contrast non-causal. The confirmatory protocol is frozen in research/experiments/E020_E021_PREREGISTRATION.md; deterministic candidate and event-ledger implementations are tested but have not yet produced paid confirmatory observations.