Skip to content

A command whose exit status its script throws away is advisory (|| true, pipes without pipefail, sequences without set -e, conditions) - #338

Merged
lefticus merged 1 commit into
cpp-best-practices:mainfrom
leftibot:masked-exit
Nov 22, 2026
Merged

lefticus merged 1 commit into
cpp-best-practices:mainfrom
leftibot:masked-exit

Conversation

@leftibot

Copy link
Copy Markdown
Collaborator

Part of #334.

A command whose exit status its script throws away can't fail CI, but until now it counted as a gate. Only continue-on-error and allow_failure made a run advisory. Now each command's place in its script decides it too. A masked run is advisory, with the reason, and checks say so: ruff format runs in CI but cannot fail it (|| true).

What counts as masked

Pattern Reason shown
`cmd
`cmd tee log` without pipefail
cmd; other or a later line, without set -e followed by other commands without set -e
cmd && other, then more lines, under set -e (errexit ignores it) before && in the middle of the script
if cmd, elif, while, until an if condition, …
! cmd negated by !
cmd & run in the background

It is not masked when:

  • the fallback fails again: || exit 1, || exit, || false, || { …; exit 1; };
  • the fallback tests the status (|| [ $? -eq 124 ], which zig's fuzzing uses to accept timeout);
  • the next command reads $? or PIPESTATUS;
  • the command is the script's last.

A fallback group that goes on past its line isn't read, so it isn't judged. Commands inside $(…) aren't judged.

Which shell runs the script

  • GitHub Actions:
    • the default shell is bash -e {0}: errexit, no pipefail;
    • shell: bash (or defaults.run.shell) is -eo pipefail;
    • sh is -e;
    • a bash -l {0}-style wrapper is read for its flags;
    • PowerShell, cmd, and a shell named by an expression aren't judged.
  • GitLab: the runner's bash and sh scripts start with set -eo pipefail. Each script line is read on its own, so only fallbacks, pipes without pipefail and conditions apply. Windows (PowerShell) isn't judged.
  • make recipe lines and package.json scripts: sh -c, with no errexit. make's - prefix, and just's, ignores the line's failure (make ignores its failure (-)).
  • Repository shell scripts: the options come from the shell command that runs the script (bash -e ci/lint.sh), or from its shebang when it runs itself (#!/bin/bash -e). A sourced script (. ci/env.sh) isn't judged.
  • Throughout: set -e, set +e, set -o pipefail and set -euo pipefail change the options from that point on.

How it's built

  • facts/shell.py: each ShellCommand now carries what decides its status:
    • the operator after it;
    • the condition it's in;
    • its || fallback, if that fallback can succeed;
    • whether the next command reads its status;
    • whether it's nested in a substitution.
  • facts/exit_status.py (new): Shell (errexit, pipefail), shell_of and set_options (read -eo pipefail and the like), and masked, which walks a script and gives each command its reason.
  • script_facts takes the shell. Script carries shell and advisory for expansions.
  • The make parser now keeps each recipe line's prefixes, so - can be read; tests/test_expansion.py changes to match.

Corpus snapshots, checked by hand

Four .facts.json snapshots change, and no result does:

  • well_configured_c and _cpp: find … | xargs clang-format … makes find advisory. That's right, and find isn't a tool.
  • well_configured_kubernetes: helm template charts/shop | kubeconform … makes helm template advisory. That's right: if helm fails, kubeconform validates empty input and passes.
  • well_configured_lua: curl … | tar xz makes curl advisory. That's right, and harmless.

The first run also flagged zig's timeout 60 zig build fuzz || [ $? -eq 124 ], a false positive that turned fuzz-testing into WARN. A fallback that tests $? now counts as failing again, with a test.

Not covered

PowerShell ($ErrorActionPreference, $LASTEXITCODE) isn't judged yet; #334's fifth pattern says "possibly later". GitLab lines are read one at a time, so cmd && other in the middle of a GitLab script isn't caught.

Gates

Run with the fixed gate script, which checks every step's own exit status: ruff, ruff format, mypy, lizard (length ≤ 30), tests (2,473 OK, 13 expected failures), coverage (99%; the new and changed lines are all covered), jscpd (0 clones) and strict self-validation all pass. One test, for a shell an expression names, was added after the run; I checked it fails without its code and passes with it.

🤖 Generated with Claude Code

https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt

`cmd || true`, a pipe without pipefail, a command followed by others
without set -e, one before && in the middle of a script under set -e,
a condition (if, elif, while, until, !) and a background command
cannot fail CI. Each is now advisory, with the reason. The shell
decides which apply: GitHub's default bash -e versus shell: bash's
-eo pipefail, GitLab's set -eo pipefail, sh -c for make recipe lines
and package scripts, a script's own shell command or shebang, and set
along the way. make's and just's `-` prefix ignores a line's failure.

A fallback that fails again, or tests $? (zig's `|| [ $? -eq 124 ]`),
keeps the failure, as does a status the next command reads.

Part of cpp-best-practices#334

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
@codecov

codecov Bot commented Nov 22, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.75%. Comparing base (169cc1f) to head (6c7c1d4).

Additional details and impacted files
@@           Coverage Diff            @@
##             main     #338    +/-   ##
========================================
  Coverage   99.75%   99.75%            
========================================
  Files         298      299     +1     
  Lines       17971    18136   +165     
  Branches     2055     2086    +31     
========================================
+ Hits        17927    18092   +165     
  Misses         44       44            

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@lefticus
lefticus merged commit d3dc4c2 into cpp-best-practices:main Nov 22, 2026
26 checks passed
@lefticus
lefticus deleted the masked-exit branch November 22, 2026 02:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants