Skip to content

✨ feat: generate PEP 770 SBOMs for Python wheels - #2

Merged
gaborbernat merged 115 commits into
mainfrom
add-sbom-hook
Nov 8, 2026
Merged

gaborbernat merged 115 commits into
mainfrom
add-sbom-hook

Conversation

@gaborbernat

@gaborbernat gaborbernat commented Nov 7, 2026 •

Copy link
Copy Markdown
Member

Projects that ship wheels have no reusable way to put a software bill of materials where PEP 770 tells installers to keep it, in .dist-info/sboms/. virtualenv carries its own generator, and the other tox-dev projects carry none. python-sbom turns that generator into a package: a hatchling project enables a build hook with one table in pyproject.toml, a project on another backend such as meson-python runs python-sbom embed on the built wheel, and python-sbom archive describes other zip artifacts, such as virtualenv's zipapp. 📦

The CycloneDX 1.6 document records the project, the runtime dependencies it declares, the wheels it bundles, and the build toolchain down to file hashes. Nothing about the build machine goes in, so a rebuild with the same inputs reproduces it byte for byte. python-sbom spdx renders it as SPDX 2.3. python-sbom validate takes CycloneDX input only: it checks the document against the official CycloneDX schema, shipped with the package so validation works offline. For documents python-sbom wrote, it also checks the references and the SPDX rendering, plus an optional licence allow-list.

Architecture

The hatch hook and the command line sit at the edge: they gather the options, the content provider, the build scripts and the build facts, then call the pipeline for their artifact. Each pipeline opens the wheel or archive once and reads it into a tree, then a pure function of the model maps that tree to components. One shared step adds what every artifact of the project carries, and the result is written back. Nothing below the hook imports hatchling, so the command line installs without it.

flowchart LR
    hook[hatch hook] --> pipelines
    cli[CLI: embed / archive / spdx / validate] --> pipelines
    pipelines[wheel and archive pipelines] --> collect[collect: settings, facts, revision, wheel or archive tree]
    collect --> reader[artifact reader: one checked handle per build]
    pipelines --> model[model: pure document building, shared graph rules]
    pipelines --> formats[CycloneDX JSON / SPDX 2.3]
    pipelines --> write[wheel and archive writing]
    provider[ContentProvider] --> pipelines
    cli --> validate[validate: schemas, references, licences]
    validate --> model
    classDef edge fill:#cfe4ff,stroke:#1f6feb,color:#0b1f3a;
    classDef core fill:#d8f3dc,stroke:#2d6a4f,color:#081c15;
    class hook,cli,provider edge;
    class pipelines,collect,reader,model,formats,write,validate core;
Loading

The SBOM describes the bytes the artifact ships. Bundled wheels are matched to the artifact's own copies by digest and read from there, so a source wheel that changed after the build is refused rather than described. Generation, validation and the licence policy share one definition of what an artifact holds and one reference check, so they judge the same components wherever the document places them.

Projects extend the document through one seam: a ContentProvider receives a ProviderContext (project root, settings, identity, and a reader for the artifact's members) and returns Contributions: extra components, dependencies, root properties and tools. virtualenv uses it to describe the pip and setuptools wheels it seeds, in both its wheel and its zipapp, with no private imports. The public API is that seam plus every type it exposes, and a test fails if a public signature reaches a private type.

Behaviour worth knowing

  • Wheels are read and rewritten in place: unchanged members keep their bytes, and only RECORD and the new SBOM are written. Bundled wheels are checked against their RECORD before anything about them is recorded.
  • The source commit comes from git when the project is tracked, from an sdist built by the hook, or from .git_archival.txt; a modified checkout is marked dirty.
  • Validation uses jsonschema-rs, which agreed with python-jsonschema on every document except cases where python-jsonschema contradicts the RFCs. Hostile documents are bounded by size, depth and a component cap of 100,000 (adjustable with --max-components).
  • python-sbom's own wheel carries its own SBOM, and the package check validates it.

Port the CycloneDX SBOM generator from pypa/virtualenv's hatch_build.py
into a reusable hatchling plugin, with the project-specific parts
(supplier, repository, bundled wheels) moved into hook options and a
subclass method for per-wheel properties. The SPDX rendering and the
validator ship as the hatch-pep770 spdx and validate commands.
Generate the SBOM from the wheel's METADATA so the hatchling hook and the
new embed command share one code path, and expose the hook, the Project
settings, the building blocks and the SPDX and validation helpers from
the package root. Raise the floor to Python 3.11.
@gaborbernat gaborbernat changed the title ✨ feat: add PEP 770 SBOM build hook ✨ feat: add PEP 770 SBOM build hook and embed command Nov 7, 2026
@gaborbernat gaborbernat added the enhancement New feature or request label Nov 7, 2026
The distribution, import package, console script and hatch hook are now
python-sbom (python_sbom for imports), and the build hook class is
SbomBuildHook.
@gaborbernat gaborbernat changed the title ✨ feat: add PEP 770 SBOM build hook and embed command ✨ feat: add the python-sbom build hook and embed command Nov 7, 2026
Join the build script path as a string so a drive letter survives on
Windows, and admit simplint 0.7.1 past the seven-day cutoff so the
cooled hook job can install it.
A pure-Python package needs every supported Python on Linux, the oldest
and newest on Windows, the newest on macOS, and the tooling once. ty
checks every platform in its single job instead.
The committed wheel tripped Scorecard's Binary-Artifacts check and could
not be reviewed; a fixture now writes the same small wheel with fixed
timestamps, so its hashes stay stable.
The core (collect, model, formats, wheel, validate, cli) no longer imports
hatchling; the build hook moves to python_sbom.hatch behind a Backend ABC,
with hatchling as the hatch extra. Build facts are gathered once at the
edge and passed in. Also fixes five review findings: unique SPDX ids for
conditional requirements, distinct refs for bundled wheel variants,
marker-aware runtime dependencies, structural validation before deeper
checks, and SPDX rendering of versionless roots.
Split settings and build facts, the document model, wheel archives,
formats and validation into subpackages that import in one direction,
with build_sbom as the only pipeline. A Backend now just returns the
request, project content comes from a ContentProvider passed in, and
the hatch hook writes the SBOM in finalize from the built wheel, so a
later hook can no longer make it stale.

Address the first review's correctness and security findings: bundled
wheels are checked against their RECORD, embedded entries are the same
on every OS and keep the wheel's mode, repeated requirements collapse,
other tools' SBOMs get the schema check alone, and bad input raises an
error the CLI reports instead of a traceback.
A custom hook loads the shipped hook from the source tree, since the
package cannot be a build requirement of itself, and pkg_meta fails
when the built wheel lacks .dist-info/sboms/.
Cover the content provider, the backends and the pipeline, the source
revision chain, embedding after auditwheel, and why build tools are
listed down to their files. Redraw the architecture diagram from the
import graph, render type aliases and the TypedDicts, and keep
hatchling's lifecycle methods out of the API reference.
Trim .gitignore and CODEOWNERS, run simplint between pyproject-fmt and
ruff, point Homepage at the documentation and add the Python modules
classifier.
@gaborbernat gaborbernat changed the title ✨ feat: add the python-sbom build hook and embed command ✨ feat: generate PEP 770 SBOMs for Python wheels Nov 7, 2026
gaborbernat and others added 7 commits November 6, 2026 23:09
The public API is now the content provider surface the hatchling hook
calls, plus the types its signatures reach; embedding, archives, SPDX
and validation run through the command line. A provider receives a
context with the project root, settings, identity and a reader for the
artifact, so one provider serves the wheel and an archive. The new
archive command describes a zipapp with the same bundled-wheel
identities the wheel SBOM uses.

Address the second review: build facts survive egg-info and broken
requirements, git is used only for a tracked project without running a
parent repository's hooks, an sdist carries its commit, bundled wheels
must match every RECORD row, decoy dist-info and duplicate members are
refused, and validation reports hostile documents instead of raising.
Work in progress, split across agents from here:
- core uses installer, packageurl-python, dep-logic, license-expression
  and packaging.direct_url; build-system requires them for the self-SBOM
- validate caps components at 5,000, raised with --max-components
- the sdist revision file is written as bytes, so it holds LF on Windows
- bench/ benchmark harness scaffolding
A cap of zero rejects every SBOM with components and a negative one rejects
all of them, and the cap also sets the size limit. argparse now reports
anything but a positive integer as a usage error.
python-sbom spdx crashed with a traceback on a missing file, invalid JSON,
input that is not UTF-8, deep nesting, or a timestamp that is no ISO date. It
now reports each as an error, and refuses input over the size validate
accepts at its default component cap, so the JSON parser never holds more
than validate lets it.
The hook cached the revision for the whole process, so a build after a commit
in the same process recorded the earlier commit and dirty state. Each build
now reads it in initialize, before it writes the artifact, and finalize
reuses that value.
initialize wrote the revision file into a directory from tempfile.mkdtemp
that nothing removed, so every sdist build left one behind. finalize now
removes it once the sdist holds the file. hatchling skips finalize when the
build fails, so the directory then goes with the hook, at the latest at exit.
The resolver of the content provider costs a third of a millisecond on top of
hatchling, so it loads with the hook instead of twice behind a suppression.
finalize's build_data now carries one suppression per line, each under 120
columns.
The self-build hook delegated only finalize, so python-sbom's sdist carried no
.python-sbom-revision and a wheel built from it named no commit. One helper
now builds the SbomBuildHook for the whole build, and both initialize and
finalize go to it.
The embed test and the retagged wheel test each asserted three behaviours;
fixtures now run the command or build once per test, and each test checks
one. The archive compression test takes its flagged archive from a fixture
instead of patching bytes in its body.
The parser imported _validate._document, and with it _formats._spdx, only
to show the --max-components default. The cap and the size limit it sets
now live in _validate._limits, which imports nothing.
SettingsError and WheelError, and the ValueError and ImportError bases,
had no caller that told them apart from SbomError. Every error is now an
SbomError; MissingExtraError stays, as validation lets it through where it
reports every other error per document.
The diagram missed _errors and version, drew the hook's lazy import of
_generate as a load-time one, and lacked the CLI's imports of _wheel and,
for types, _model. The text still described one pipeline, the generate
package docstring named backends, the contributing guide pointed at the
removed _cli/_commands.py, and the threat model claimed spdx caps nesting.
proselint read the diagram's model --> errors after wheel --> model as a
repeated word, and covdefaults excludes only the bare TYPE_CHECKING guard,
so the API test's typing.TYPE_CHECKING import counted as uncovered.
packageurl only lowercases a pypi name and turns underscores into dashes,
so jaraco.context and jaraco_context got two purls, and a build tool's
bom-ref, built from its normalized name, disagreed with its own purl
(tool:pkg:pypi/boolean-py vs pkg:pypi/boolean.py). Names now go through
canonicalize_name first, as before the switch to packageurl. The pinned
SBOM changes only in the project's purl and the serial derived from it.
license-expression and its SPDX index took 9-10 ms of every cold validate,
though only --allow-license reads them.
The census pushed every value of the document onto its stack. Only lists
and objects go there now; numbers are judged as their container is read,
and the depth check moves to the container, so a value still nests too deep
exactly when it did. The census of virtualenv's zipapp SBOM drops from
about 12 to 8 ms.
Each distribution's WHEEL went through the email parser for one field, and
each Requires-Dist line was parsed and its marker evaluated again for every
distribution declaring it. Root-Is-Purelib is now read from the header
lines directly, agreeing with the email parser on 104,668 WHEEL files, and
a requirement's verdict is cached for the process.
The SBOM followed the compression most of the archive's members used, so
virtualenv's zipapp, which stores all of them, took it stored, megabytes
of JSON. It now always goes in deflated at level 9, with the fixed time and
attributes as before, and the other members keep their bytes.
The archive command now only parses options and prints, like embed; the
orchestrator reads, builds and writes, and the archive's document is built
by a pure model function as the wheel's is. The pipeline module folds into
embed, its only user.
The built wheel's .whl members are read by _wheel and handed to the source
tree reader, so embed no longer rebuilds the source tree to rename them.
Keeps the order the archive path asked content providers in before the model
split, names the digest lookup for what it returns, and lets the benchmark
find the bundled wheel reader in _wheel.
Two .dist-info in one directory left one distribution out of the archive SBOM
and gave its files to the other, so that layout is now refused. Identical
bundled wheel copies took one path in the built wheel and broke the build on a
repeated bom-ref; each copy now takes its own. A .whl at the top level of an
archive crashed generation; it is now described as a bundled wheel under the
root component.
Fixes the second external review and restructures for review. The built
artifact is now the only source of the bytes an SBOM describes: one reader keeps
it open for the whole build, and bundled wheels are found in it by digest, so a
source wheel changed or left out since the build is refused. One module defines
what an artifact holds and ships and checks references, so generation,
validation and the license policy judge the same set: top-level archive wheels
meet the policy and get their vendored edges, and a repeated dependency entry is
refused. A provider module another project already imported is refused.

The wheel and archive pipelines now read alike: open the artifact, read it into
a tree, map the tree to an Artifact with a pure function, assemble, write. Zip
reading and writing are separate modules, one helper hashes, and the archive
SBOM file name, bundled order and bundled marking each live in one place.

Also: archive refuses to write over its own input, SPDX ids carry the full
digest, RECORD rows match by parsed path, the scheduled check runs, malformed
tools fail cleanly, archive ownership looks up ancestors, and
docs/contributing.rst names what each layer guarantees and the test that holds
it.
@gaborbernat
gaborbernat marked this pull request as ready for review November 8, 2026 02:27
@gaborbernat
gaborbernat merged commit 683e004 into main Nov 8, 2026
24 checks passed
@gaborbernat
gaborbernat deleted the add-sbom-hook branch November 8, 2026 02:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant