[anth-volk]

Fixes #4202

Summary

• make quality applies ruff's fixes (ruff check --fix ., then ruff format .), then runs make quality-check.

• make quality-check runs every check without changing files: uv lock --check, ruff check ., ruff format --check ., lint-imports --config pyproject.toml, mypy, and scripts/run_quality_guards.py. Everything except the lock check runs through uv run --frozen --extra dev.

  • The guards run with python -S, without site-packages. Pull-request CI then still shows that they need only the standard library, which is how push.yml runs them.
  • They run with RULES_BASE_REF set to git merge-base HEAD origin/<base>, where the base is CI's GITHUB_BASE_REF, or master locally. The rules guard now takes RULES_BASE_REF, a revision used as given, before origin/$GITHUB_BASE_REF.
    • Locally, a branch is compared with where it left master, instead of being skipped, or wrongly compared with master's newer tip.
    • In CI, the merge is compared with the commit it is built on.
  • The quality guards include deptry today. The suppressions guard (Check that every noqa and type: ignore comment names a code and gives a reason #4183) and the asserting-tests guard (Check that every test function asserts something #4188) join automatically when they merge, through GUARDS.

• pr.yml: the quality-guards job, now named "Quality checks", runs make quality, then git diff --exit-code HEAD (HEAD, so staged changes count too). A pull request whose code ruff would fix fails there, and the log shows the fix to commit.

  • The lint and typecheck-v2 jobs fold into it and leave CI complete's needs. Its other steps are unchanged: the registry tool's unittest run, and the shellcheck install the ci-scripts guard needs.

• The rules registry follows make targets. New scripts/rules/makefile.py reads the Makefile's recipes and follows each target's prerequisites and the make or $(MAKE) calls in its recipe.

  • Rule invocations: a rule's [[rule.ci]] invocation now counts when it is in a command of a target the job runs. The 7 rules on pr.yml's lint job and the mypy rule move to quality-guards with their invocations unchanged (ruff check ., ruff format --check, mypy). Each still fails the registry check if its command leaves the target.
  • Import Linter guard: it checks the exact lint-imports command in those recipes too. Make's - prefix, which ignores a command's failure, doesn't match.
  • What can hide a failure: a later recipe for a target replaces an earlier one, as in make, and target-specific variables aren't read as prerequisites. A Makefile gets no credit for its targets' commands, and the registry's message says why, if it:
    • sets .IGNORE or .ONESHELL (with one colon or two), MAKEFLAGS, GNUMAKEFLAGS, MAKEFILES, SHELL or .SHELLFLAGS;
    • uses syntax the reader doesn't follow: include, a conditional, define, load, $(eval ...), a recipe after ;, or a computed target or variable name;
    • or has a GNUmakefile or makefile beside it, which GNU make reads first.
  • Calls that credit no target: a make call credits no target when:
    • it runs no recipe, ignores failures, treats a target as up to date or only prints something: -n, -q, -t, -i, -o, -v, -h, their long forms and accepted abbreviations;
    • it reads another Makefile or more of one: -C, -f, -E, --eval;
    • a variable that can hide failures is set before it or as an argument;
    • or make is an argument, not the command (echo make quality).
  • The - prefix: a recipe command with it, whose failure make ignores, gives no credit to a rule's invocation or to the locked-installs guard's uv lock --check. That guard now follows make calls through the same reader, and follows none in a Makefile with a problem.
  • locked-installs guard: it now follows $(MAKE) as well as make when it looks for uv lock --check.

• New enforced rule api.quality.make-quality. Its tests in tests/unit/test_make_quality.py check:

  • the two targets;
  • that make quality, run for real with a stand-in uv on PATH, runs the fixes and every check in order when they pass, and fails whenever any one of them fails. This catches what the reader doesn't recognize: .IGNORE::, GNUMAKEFLAGS, computed names, or a recipe defined only outside dry runs;
  • that make -n quality, run with GITHUB_ACTIONS=true, prints the same commands;
  • that git diff --exit-code HEAD comes right after make quality;
  • that neither step is skipped (if:), allowed to fail (continue-on-error), or given its own shell: or working-directory:;
  • that the job isn't allowed to fail and sets no default shell;
  • that no env: block for them sets MAKEFLAGS, GNUMAKEFLAGS or MAKEFILES, and no earlier step writes to GITHUB_ENV or GITHUB_PATH;
  • that the Makefile has nothing that hides a failing command and no syntax the reader doesn't follow, and that quality-check neither fixes, formats nor relocks;
  • that no other pr.yml job runs ruff, mypy, Import Linter or the quality guards, itself or through a make target.

  The proof uses fixtures under rules/fixtures/api.quality.make-quality.

• test_declared_dependencies.py checks that every job running the guards has uv on Python 3.12. It now follows make targets: it failed in CI run 42065704340, where the guards run only inside make quality.

• Docs: AGENTS.md, the GitHub PRs skill ("Commit Hygiene", which the rule cites), the testing skill and README.md say to run make quality before committing.

  • The GitHub PRs skill lists how a local run can differ from CI's: the job also runs the registry tool's own tests, the base branch for comparisons, shellcheck, and uv's version.
  • The rules-registry skill describes invocations in make targets.

Design choices for review

• One job instead of three. CI runs the same command an agent runs, as plan §9.1.5 asks. The checks no longer run in parallel. They take a few minutes together, well under the test shards, so CI complete shouldn't finish later.
• The job keeps the id quality-guards. On master, 23 rules (24 [[rule.ci]] references) and 16 rule proofs reference pr.yml:quality-guards, and Check that every noqa and type: ignore comment names a code and gives a reason #4183 and Check that every test function asserts something #4188 add more. Renaming the job would break their rules once they merge. Only the display name changes.
• One failure at a time. make quality stops at the first failing check, where the three jobs used to fail independently. The merge is blocked either way, but a pull request with several kinds of problem sees them over several runs.
• mypy runs on Python 3.12 now. The old typecheck-v2 job set no UV_PYTHON, so it ran on the Python 3.14 that .python-version selects, with uv 0.13.7. It now runs on 3.12 with uv 0.13.4. [tool.mypy] python_version = "3.11" still decides the language version it checks against.
• The quality-guards job's id stays. Check that every noqa and type: ignore comment names a code and gives a reason #4183, now merged, adds a rule on pr.yml:quality-guards too.
• The registry tool's tests stay out of make quality. They are a whole test suite, which agents in this workstream don't run locally, so only the CI job runs them.
• diff-cover isn't in make quality. It needs every test shard's coverage report, so Require 80% coverage on the lines a pull request changes, with diff-cover #4200 runs it in a pr.yml job of its own.
• push.yml is unchanged. Its versioning and publish-git-tag jobs wait for its lint job, and .github/release_commit.py names "Lint" and "Quality guards" as merge-run checks. Moving them would change what a release waits for, which is a separate decision.
• uv lock --check uses the local uv. CI pins uv 0.13.4. A different local uv could disagree about uv.lock; uv 0.12.23 agrees on this branch.
• Changes to the rule machinery. This changes scripts/rules/, which api.process.rule-machinery-code-owned (doc-only) asks a core developer other than the author to approve.

CI wiring and environment variables

The quality-guards job:

Variable	Provider
UV_PYTHON	The literal "3.12" in the job's env:, as before. The Makefile also sets it if unset.
ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION	Inherited from the workflow's env:, the literal true; not used.
OLD_SIMULATION_GATEWAY_URL	Inherited from the workflow's env:, the repository Actions variable; not used.
GITHUB_BASE_REF	Set by GitHub on pull_request events to the base branch. make quality-check uses master when it is unset or empty.
RULES_BASE_REF	Set by the make quality-check recipe, for that one command, to git merge-base HEAD origin/<base>, and read by scripts/rules/baseline.py. If git can't find the merge base, it is empty and the guard falls back to origin/$GITHUB_BASE_REF. The CI log doesn't show its value.
GITHUB_ACTIONS	Set by GitHub. The ci-scripts guard reads it to fail, rather than skip, when shellcheck is missing, and the rules guard reads it to fail, rather than skip, when it has no base to compare with.

• No secret is read. actions/checkout and astral-sh/setup-uv use the automatic github.token.
• make is on ubuntu-latest. The job installs uv 0.13.4 and shellcheck as before. uv run syncs .venv from uv.lock.
• The removed lint and typecheck-v2 jobs read no variable or secret either.
• The job keeps its full-CI condition for stacked pull requests, and no path filter.

Testing

Red, at b10bd5120 (tests and fixtures only):

• tests/unit/test_make_quality.py and scripts/rules/tests/test_makefile.py: ModuleNotFoundError for scripts.rules.makefile.
• test_import_linter...test_a_make_target_the_job_runs_must_run_every_contract_too: lint-imports --config pyproject.toml || true in a make target accepted.
• test_locked_installs...test_a_lock_check_in_a_recursive_make_target_counts: "no pull request job runs uv lock --check for .".

Green, at 4dc657fdc: the same 12 tests pass, by node ID (3 pytest, 9 unittest). This commit's CI run failed on two things that the first review round found:

• test_every_job_running_the_guards_has_uv_on_python_3_12 didn't look inside make targets.
• tests/unit/test_make_quality.py's import order failed ruff's I001. A ruff cache had hidden that locally, and make quality would have fixed it, so the claim that it changed no file was wrong at that commit.

Review round 1, red at b6bf00214 (tests and fixtures only):

• The 13 test_makefile.py tests failed to import makefile_problems.
• In test_make_quality.py, the 6 new cases (step continue-on-error, step condition, job continue-on-error, .IGNORE, a check in another job's make target, pipx run ruff) failed, and so did the check of the repository's Makefile, which didn't yet run the guards with python -S and a base branch.

Green at 916ef62c3: those tests, the changed test_declared_dependencies test and the rest of this PR's tests pass. That is 16 unittest IDs and 4 pytest IDs (9 cases).

Review round 2, red at eb12b1bdd (tests and fixtures only): 22 subtests in 5 unittest functions and 10 pytest cases failed. These covered:

• syntax the reader doesn't follow, and a GNUmakefile;
• calls that credit no target;
• RULES_BASE_REF;
• make -n quality;
• MAKEFLAGS in env:, a shell: override, a step between make quality and git diff, and a relock.

Green at 46fc1191b: all 21 unittest IDs and 5 pytest functions (17 cases) pass.

Review round 3, red at c84108df1 (tests only): 18 unittest subtests and 7 pytest cases failed. They cover:

• .IGNORE::, GNUMAKEFLAGS, MAKEFILES and computed names;
• more make calls that should credit nothing;
• --prefixed commands for the registry and the locked-installs guard;
• workflow defaults, GITHUB_ENV, GNUMAKEFLAGS, MAKEFILES and working-directory;
• GNUMAKEFLAGS for the rule as a whole.

The run of make quality with a stand-in uv already passed for the repository's Makefile, and it reports .IGNORE::, computed MAKEFLAGS and the recipe defined only outside dry runs. Green at b76e7acdf: all 23 unittest IDs and 7 pytest functions (28 cases) pass.

Then master (with #4183) was merged in at 194bd0006. The generated files were rendered again, keeping this branch's AGENTS.md paragraph. The same tests and make quality pass there.

master (with #4188, then #4196, #4200 and #4205) was merged in again, at 9ad5c4416 and 30a97a067.

• In the merged pr.yml, CI complete needs every job, including Require 80% coverage on the lines a pull request changes, with diff-cover #4200's diff-coverage and Measure the unit tests' coverage on master and flag every Codecov upload #4196's coverage uploads.
• No rule refers to the removed lint or typecheck-v2 jobs.
• make quality runs Make v2 file-layer exceptions expire and cover whole files #4205's updated v2-layers guard.
• The rule's tests confirm that no other job runs a check itself: diff-coverage runs only diff-cover.
• The tests and make quality pass at 30a97a067.

Also run locally at 194bd0006:

• make quality: every step passes, and it changed no file (ruff run with --no-cache too);
• python -m scripts.rules guard --base-ref origin/master and python -m scripts.rules validate: pass;
• the changelog fragment check.

Per this workstream's rules, only the new tests were run locally; CI runs the rest.

🤖 Generated with Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 52.27%. Comparing base (8ab6df6) to head (30a97a0).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff             @@
##           master    #4203       +/-   ##
===========================================
- Coverage   92.11%   52.27%   -39.85%     
===========================================
  Files         297      297               
  Lines       16673    16673               
  Branches     2719     2719               
===========================================
- Hits        15359     8715     -6644     
- Misses        798     7386     +6588     
- Partials      516      572       +56     

Flag	Coverage Δ
integration	52.27% <ø> (ø)
unit	?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.