Research roadmap · Chess, Scaffolded

01 Luna frozen / DeepSeek terminal data, interim cohort

Selected research gate

Adaptive ladder

Is the complete agent measured under one finished protocol—or only observed midstream?

Decision rule. Admit protocol-specific cohort estimates only after terminal collection, error accounting, rating, and freeze agree; a ladder score is not unaided model strength.

Inspect the protocol leaderboard

Completed evidence

Luna contributes 220 clean games and is terminal and frozen. DeepSeek contributes 100 clean games; its rows are terminal, but DeepSeek remains provisional at the public cohort level: it is terminal data with an interim, unfrozen public cohort. Publication authorization is not granted. No terminal DeepSeek rating is public.

Open scientific gate

No gate in this dependency set is open. Terminal-data gates are closed; public cohort freeze and publication authorization remain separate.

Depends on

Terminal evaluation host; content-addressed cohort generation; terminal protocol rating; and request-level billing or a fail-closed unavailable record.

Falsifier

An unreconciled terminal inventory, optional-stopping exposure, or unclassified provider failure falsifies the terminal-data claim.

Stop condition

Stop at protocol-specific observation while any ladder gate is open. Even with terminal DeepSeek rows, do not publish a frozen DeepSeek rating or a joint ranking until public cohort freeze and publication authorization are granted.

02 Inventory reconciled

Selected research gate

Terminal freeze

Can every downstream number be traced to one finished, immutable generation?

Decision rule. Freeze the source generation before any paired pool, label, or result reads a moving cohort.

Why the result must wait

03 Blocked by eligibility

Selected research gate

Paired position pool

Can an intervention be compared on exactly the same chess decision?

Decision rule. Admit only positions with complete provenance, source-game isolation, balanced controls, and a frozen candidate/action universe.

Read the paired-position case

Completed evidence

A110A's locked immediate-prompt audit found 11,002 of 11,002 direct all-legal selections inside the displayed candidate set; a separate post-hoc same-position sensitivity found 23,332 of 23,353 inside. The 100% locked-primary membership and 99.91% post-hoc sensitivity are parser-membership observations, not selector quality, move value, causal benefit, or Elo.

Open scientific gate

Open: S08 — Joint E000/E001 leakage qualification and label eligibility.

Depends on

Verified terminal freeze; source-game isolation; eligible positions; and an independent label plan.

Falsifier

A leakage audit failure, source-game overlap, incomplete candidate provenance, or unbalanced control assignment falsifies pool eligibility.

Stop condition

Stop pool admission on any provenance or balance failure. Exclude the position rather than repairing eligibility after outcomes are visible.

04 Blocked by label eligibility

Selected research gate

Two-pass labels

Which actions were available, and how good were they, without turning disagreement into truth?

Decision rule. Use independent passes for candidate availability and engine quality; disagreement is an exclusion, not a forced label.

See why the engine waits

05 Local preflight only

Selected research gate

Randomized model experiment

Does changing access to candidate or state information change the decision?

Decision rule. Run only the registered factorial contrast after the client, route, budget, and label gates pass; a clean compiler is not a model effect.

Explore the A105 instrument

Completed evidence

A105's local preflight rendered 128 cells with 0 renderer failures. Its analysis preflight covers 256 synthetic cells, with 0 provider calls and 0 observed model outcomes. The installed AutoGen Responses adapter rewrote required submit_move tool choice to auto before HTTP serialization.

Open scientific gate

Open: S08 — Joint E000/E001 leakage qualification and label eligibility; S12 — Paid budget consent and provider-qualified A105 route.

Depends on

Frozen labels; validated client and transport contract; randomized schedule; and paid-budget authorization.

Falsifier

Any difference outside the two registered spans is an automatic no-go. A contrast whose registered uncertainty includes zero is not a positive mechanism result.

Stop condition

No paid A105 model run is authorized until S12 passes and the route is rechecked. R3's installed Responses path remains the observed NO-GO; R3A is isolated and not integrated into the live harness. No provider-received bytes, provider compatibility, provider behavior, model effect, chess-quality effect, power result, final-position result, paid execution, or causal benefit has been observed.

06 Specified / no outcomes

Selected research gate

Independent evaluator

Does the effect survive a judge that did not create the treatment?

Decision rule. Treat blinding, dependence-aware uncertainty, missingness bounds, and registered sensitivity as admission criteria for every mechanism headline.

Inspect the analysis philosophy

07 Results gated

Selected research gate

Evidence-bounded publication

Can a reader distinguish completed evidence from a registered plan?

Decision rule. Publish only what its artifact and claim state support. The strongest admissible public form is living methods and observation working paper.

Open the evidence atlas

Completed evidence

4 frozen observations, 1 frozen no-signal result, and 2 preregistered tests are public. The release inventory has 549 of 577 artifacts present.

Open scientific gate

5 of 12 submission gates remain open: S08 — Joint E000/E001 leakage qualification and label eligibility; S09 — Paired mechanism outcomes; S10 — Draft evidence bundle; S11 — Publication evidence bundle; S12 — Paid budget consent and provider-qualified A105 route.

Depends on

Reconciled artifacts; claim-state review; public source identities; and a reproducibility audit.

Falsifier

A missing artifact, untraceable number, or failed reproduction downgrades or rejects the claim.

Stop condition

While any fatal submission gate or publication blocker remains, publish bounded methods and observations only; keep terminal and causal headlines locked.

Every result has
a gate before it.

Adaptive ladder

Terminal freeze

Paired position pool

Two-pass labels

Randomized model experiment

Independent evaluator

Evidence-bounded publication

Build downstream.
Claim upstream.

Parallelize instruments

Serialize evidence

Publish refusals

Separate states

Follow the evidence,
not just the score.

Adaptive ladder

Build downstream.Claim upstream.

Parallelize instruments

Serialize evidence

Publish refusals

Separate states

Follow the evidence,not just the score.

Build downstream.
Claim upstream.

Follow the evidence,
not just the score.