Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions RESEARCH.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ Last reviewed: 2026-08-04.

**Implementations.**
- PR #15 (Reframe self-review as rubric-based and CI-grounded): shipped 2026-05-25. Observed effect: pending — needs N cycles of data. First applied to PR #26 (one cycle, all rubric items PASS); too early to tell whether the rubric catches real defects that free-form review would miss.
- PR #28 (Generalize the Phase 4 external-signal step to support no-CI repos): shipped 2026-05-25. Observed effect: pending — needs N cycles of data. Replaces the hardcoded "wait for CI green" anchor with `{{EXTERNAL_SIGNAL_LABEL}}` / `{{EXTERNAL_SIGNAL_CMD}}`, so a repo without CI still anchors the rubric on *some* external signal instead of falling back to unanchored judgment — what this finding requires is that an anchor exists, not that it is specifically CI. `create-dev-loop` itself is the case in miniature: it gained CI in PR #65 (2026-07-27), but that workflow only runs `scripts/check_docs.py`, so the anchor for a *behavioral* change here is still the manual `/create-dev-loop` validation checklist rather than the green check.
- PR #28 (Generalize the Phase 4 external-signal step to support no-CI repos): shipped 2026-05-25. Observed effect: pending — needs N cycles of data. Replaces the hardcoded "wait for CI green" anchor with `{{EXTERNAL_SIGNAL_LABEL}}` / `{{EXTERNAL_SIGNAL_CMD}}`, so a repo without CI still anchors the rubric on *some* external signal instead of falling back to unanchored judgment — what this finding requires is that an anchor exists, not that it is specifically CI. `create-dev-loop` itself is the case in miniature: it gained CI in PR #65 (2026-07-27), but that workflow only runs `scripts/check_docs.py` and that script's own unit tests, so the anchor for a *behavioral* change here is still the manual `/create-dev-loop` validation checklist rather than the green check.
- PR #34 (Structure Phase 9 self-audit prompts and label gap issues): shipped 2026-05-25. Observed effect: pending — needs N cycles of data. Extends the rubric-over-free-form pattern from Phase 4 (self-review) to Phase 9 (self-audit), with a 6-item structured checklist and a 5-label taxonomy for filed gap issues.
- PR #54 (Apply consolidated template-rule backlog, #44/#46/#47): shipped 2026-06-14. Observed effect: pending — needs N cycles of data. Sharpens the external-anchor definition (#44: UNVERIFIED handling when the anchor can't run; #46: green CI is not verification when CI's scope excludes the changed files) and grounds the Tests-fix rubric item empirically (#47: stash-and-run FAIL→PASS, not scored from reasoning alone) — directly targeting the self-preference failure mode this finding's evidence documents for judgment-based rubric scoring.
- PR #57 (Fix documentation drift across the three sources of truth): shipped 2026-07-25. Observed effect: pending — needs N cycles of data. Restores `{{EXTERNAL_SIGNAL_LABEL}}` in the Phase 1 carried-over-PR bullet, where PR #54 had reintroduced a hardcoded "CI must be green first" after PR #28 generalized the anchor. An anchor named `CI` in a repo that has no CI is an anchor the loop cannot check, which degrades the self-review to exactly the unanchored critique this finding shows is neutral-to-harmful.
Expand Down Expand Up @@ -204,7 +204,7 @@ Last reviewed: 2026-08-04.
- Consider periodic regression checks: dry-run the generated skill against a known-good fixture repo and compare output to a recorded baseline. Any repo you maintain works as the fixture — the value is in the recorded baseline and the diff against it, not in which repo produced it.

**Implementations.**
- PR #33 (Template-version header for drift detection): shipped 2026-05-25. Observed effect: pending — needs N cycles of data. Adds `<!-- template-version: SHA -->` and `<!-- generated-at: ISO-timestamp -->` HTML comments to the top of every newly generated skill, plus `{{TEMPLATE_VERSION}}` and `{{GENERATED_AT}}` rows to the Step 4 substitution table. Implements the second bullet of this finding's implications; the periodic regression-check (third bullet) is a separate future change. Acting on a detected drift is also deferred — that step belongs in this repo's own generated dev-loop skill (its Phase 9 self-audit), not in the template here.
- PR #33 (Template-version header for drift detection): shipped 2026-05-25. Observed effect: pending — needs N cycles of data. Adds `<!-- template-version: SHA -->` and `<!-- generated-at: ISO-timestamp -->` HTML comments to the top of every newly generated skill (they have since moved directly below the H1 heading, the fix for #36), plus `{{TEMPLATE_VERSION}}` and `{{GENERATED_AT}}` rows to the Step 4 substitution table. Implements the second bullet of this finding's implications; the periodic regression-check (third bullet) is a separate future change. Acting on a detected drift is also deferred — that step belongs in this repo's own generated dev-loop skill (its Phase 9 self-audit), not in the template here.
- PR #35 (Update mode — re-derive existing skills from current template): shipped 2026-05-25. Observed effect: pending — needs N cycles of data. Adds an `update` mode to `/create-dev-loop` so the retrofit work that CLAUDE.md's promotion checklist currently mandates can be driven by the skill itself rather than by hand. Complements PR #33's drift detection — #33 detects drift; PR #35 is the remediation channel.

---
Expand Down
Loading