Repository navigation
merge(stats): plan the m6a paired statistics milestone - #51
Merged
Merged
Conversation
Author the campaign __roadmap__/m6a-paired-stats/ from the approved round-4 plan: 14 leaves over four levels (L0 tests, settings, counts and order; L1 level resolve and blocking; L2 job wiring and analyze output; L3 golden, known-truth and design record), each with targeted test commands, owned-file regexes and, for five of them, a verifier brief. Record amendment A9 on the stats leaf: runs are not paired, so McNemar and fixed-sequence are dropped, "paired" becomes "blocked", Fisher becomes its mid-p, CMH gains an exact fallback, the bootstrap becomes a stratified permutation test with BCa, and Holm is the only multiplicity method. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
LittleCoinCoin
added a commit
that referenced
this pull request
Nov 18, 2026
`main` has been red since 9560cd3 (#51, roadmap-only): 20 candler-core guards fail on CI with "`.claude/skills/htn/SKILL.md` is not in commit <main's HEAD>". The code at #50, the last green run, is the same. ## Cause `Swatinem/rust-cache` saves `target/` after deleting every file outside the build profiles. It keeps the directories (cleanup rule: "Files other than `CACHEDIR.TAG` are removed"). So the cache restores the shared fixture `target/tmp/job_skill_repo` as empty folders, `.git` included. `skill_repo()` only checked that `.git` exists, so it reused the skeleton. git did not recognise the skeleton as a repository and searched upward until it reached the checkout's own repository. ## Fix (test support only) - The fixture is reused only when `.git/HEAD` is a file. - A skeleton is pruned by removing empty directories only, deepest first, so a repository that another nextest process has just placed is never deleted. - git runs with `GIT_CEILING_DIRECTORIES` set to the scratch directory, so a missing repository fails at once instead of reading the checkout's. ## Evidence With a planted skeleton in an in-repo `target/`: - the old fixture fails with CI's exact error (2 of 3 sampled guards); - the fixed one passes 16 of 16 sampled guards; - the full candler-core suite passes 329 of 329; - fmt and clippy (`-D warnings`) are clean. This also unblocks #52, whose 21 failures share this cause. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the campaign
__roadmap__/m6a-paired-stats/, the plan Eliott approved on 2026-11-18 (round 4) for the M6a leaf (candler-v1/…/m5b/stats/m6a_paired_stats.md). It also ticks M5b's gates and records amendment A9.Commits.
rdm(isolation): ticks M5b's six own gates, each with its evidence (CI run, guard names, the live smoke job). M5b stays in progress until its last gate,stats/, is done. Ticks the stats level's precondition and the M6a leaf's two preconditions, and marks both in progress.rdm(stats): the campaign (14 leaves over four levels) and A9.A9, gate changes Eliott agreed in round 2. Runs are not paired, so McNemar is dropped and "paired" becomes "blocked": blocks come from the other declared factors. Fisher becomes its mid-p, CMH gets an exact conditional fallback, the bootstrap becomes a stratified permutation test with a BCa interval, and Holm is the only multiplicity method.
Tree. Depth is order.
exact_tests,cmh,resampling,treatment_setting(T2),treatment_verdict(T5),arm_counts(E1),execution_order(B3).resolve/:level_resolve(T3, T4),blocking.resolve/wired/:level_wiring,analyze_output.resolve/wired/proven/:golden_blocked,known_truth,design_record.Budget. The cap is $105, with a stop and report to Eliott at $84. Planning measured $11.67 against its $10 cap.
Validated:
dirtree-rdm validategives OK on__roadmap__/m6a-paired-stats/and__roadmap__/candler-v1/.🤖 Generated with Claude Code