Skip to content

merge(stats): plan the m6a paired statistics milestone - #51

Merged
LittleCoinCoin merged 2 commits into
mainfrom
claude/m6a-plan-105cfa
Nov 18, 2026
Merged

LittleCoinCoin merged 2 commits into
mainfrom
claude/m6a-plan-105cfa

Conversation

@LittleCoinCoin

Copy link
Copy Markdown
Member

Adds the campaign __roadmap__/m6a-paired-stats/, the plan Eliott approved on 2026-11-18 (round 4) for the M6a leaf (candler-v1/…/m5b/stats/m6a_paired_stats.md). It also ticks M5b's gates and records amendment A9.

Commits.

  • rdm(isolation): ticks M5b's six own gates, each with its evidence (CI run, guard names, the live smoke job). M5b stays in progress until its last gate, stats/, is done. Ticks the stats level's precondition and the M6a leaf's two preconditions, and marks both in progress.
  • rdm(stats): the campaign (14 leaves over four levels) and A9.

A9, gate changes Eliott agreed in round 2. Runs are not paired, so McNemar is dropped and "paired" becomes "blocked": blocks come from the other declared factors. Fisher becomes its mid-p, CMH gets an exact conditional fallback, the bootstrap becomes a stratified permutation test with a BCa interval, and Holm is the only multiplicity method.

Tree. Depth is order.

  • L0: exact_tests, cmh, resampling, treatment_setting (T2), treatment_verdict (T5), arm_counts (E1), execution_order (B3).
  • L1 resolve/: level_resolve (T3, T4), blocking.
  • L2 resolve/wired/: level_wiring, analyze_output.
  • L3 resolve/wired/proven/: golden_blocked, known_truth, design_record.
  • Five verifiers, dispatched by the implementers. No gate needs Docker or a model call. DIGEST and GOLDEN must not move.

Budget. The cap is $105, with a stop and report to Eliott at $84. Planning measured $11.67 against its $10 cap.

Validated: dirtree-rdm validate gives OK on __roadmap__/m6a-paired-stats/ and __roadmap__/candler-v1/.

🤖 Generated with Claude Code

LittleCoinCoin and others added 2 commits November 17, 2026 20:09
M5b's six own gates are met by PR #38 (merge 1e6dc91); its last gate
is stats/, which M6a starts now. The candler-M6 reminder is read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Author the campaign __roadmap__/m6a-paired-stats/ from the approved
round-4 plan: 14 leaves over four levels (L0 tests, settings, counts and
order; L1 level resolve and blocking; L2 job wiring and analyze output;
L3 golden, known-truth and design record), each with targeted test
commands, owned-file regexes and, for five of them, a verifier brief.

Record amendment A9 on the stats leaf: runs are not paired, so McNemar
and fixed-sequence are dropped, "paired" becomes "blocked", Fisher
becomes its mid-p, CMH gains an exact fallback, the bootstrap becomes a
stratified permutation test with BCa, and Holm is the only multiplicity
method.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@LittleCoinCoin
LittleCoinCoin merged commit 9560cd3 into main Nov 18, 2026
7 checks passed
@LittleCoinCoin
LittleCoinCoin deleted the claude/m6a-plan-105cfa branch November 18, 2026 02:25
LittleCoinCoin added a commit that referenced this pull request Nov 18, 2026
`main` has been red since 9560cd3 (#51, roadmap-only): 20 candler-core
guards fail on CI with "`.claude/skills/htn/SKILL.md` is not in commit
<main's HEAD>". The code at #50, the last green run, is the same.

## Cause

`Swatinem/rust-cache` saves `target/` after deleting every file outside
the build profiles. It keeps the directories (cleanup rule: "Files other
than `CACHEDIR.TAG` are removed").

So the cache restores the shared fixture `target/tmp/job_skill_repo` as
empty folders, `.git` included. `skill_repo()` only checked that `.git`
exists, so it reused the skeleton. git did not recognise the skeleton as
a repository and searched upward until it reached the checkout's own
repository.

## Fix (test support only)

- The fixture is reused only when `.git/HEAD` is a file.
- A skeleton is pruned by removing empty directories only, deepest
first, so a repository that another nextest process has just placed is
never deleted.
- git runs with `GIT_CEILING_DIRECTORIES` set to the scratch directory,
so a missing repository fails at once instead of reading the checkout's.

## Evidence

With a planted skeleton in an in-repo `target/`:
- the old fixture fails with CI's exact error (2 of 3 sampled guards);
- the fixed one passes 16 of 16 sampled guards;
- the full candler-core suite passes 329 of 329;
- fmt and clippy (`-D warnings`) are clean.

This also unblocks #52, whose 21 failures share
this cause.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant