Skip to content

feat: allow opt-in for new backends - #409

Open
b-long wants to merge 57 commits into
masterfrom
feature/replacing-pybindgen
Open

b-long wants to merge 57 commits into
masterfrom
feature/replacing-pybindgen

Conversation

@b-long

@b-long b-long commented Sep 19, 2026 •

Copy link
Copy Markdown
Member

Relates-to: #408

Adds opt-in alternatives to pybindgen, selected with GOPY_BACKEND. The default stays pybindgen; its only output change is the Makefile fix below.

Backend Needs gopy exe Status
capi nothing beyond Python yes full suite passes
cffi cffi via capi full suite passes
pybind11 pybind11, C++ compiler via capi full suite passes
nanobind nanobind, C++ compiler via capi full suite passes

Each backend runs in its own CI job on Linux, macOS and Windows (Python 3.11 and 3.12) without pybindgen installed. A benchmark job compares per-call time and memory across backends.

Notes

  • capi reuses the default Go shim and build but writes the C module itself, matching pybindgen's argument handling and errors. It's the simplest way to drop pybindgen.
  • cffi, pybind11 and nanobind share a plain-C Go shim that avoids the CPython C API. Functions needing a raw PyObject* are skipped with ignoring function not supported by GOPY_BACKEND=<name>: ....
  • gopy exe builds with capi when one of those three is selected: an executable embeds the interpreter through the C API anyway, and capi binds everything they do.
  • Default backend fix: the Makefile gopy gen writes for pybindgen and capi linked against a separate <pkg>_go library with no rpath, so it didn't import as-is. cgo now builds one shared library, as gopy build does. TestMakefile runs for every backend.
  • gopy pkg and gopy exe now restore the working directory when done.
  • Copyright headers in touched files now read 2015-present.

Test coverage

Project coverage rises from 74.31% to 89.34% (+15.0 points), with 93.16% patch coverage (Codecov). New tests: _examples/callbacks, additions to _examples/pyerrors and _examples/slices, and Go unit tests in bind/.

About 5 of those points come from setting partials_as_hits in codecov.yml to match go tool cover; master scores 79.4% under that rule, so ~10 points are new coverage.

@b-long b-long mentioned this pull request Sep 19, 2026
Comment thread main_test.go Resolved
Comment thread cmd_build.go Resolved
@b-long

This comment was marked as outdated.

@codecov

codecov Bot commented Nov 1, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.16940% with 50 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.34%. Comparing base (67e890d) to head (5afc101).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
cmd_build.go 80.00% 18 Missing ⚠️
bind/gen_func.go 86.66% 8 Missing ⚠️
bind/cxxbuild.go 95.34% 6 Missing ⚠️
bind/plainc.go 93.58% 5 Missing ⚠️
bind/backend.go 93.02% 3 Missing ⚠️
bind/shim.go 94.82% 3 Missing ⚠️
bind/callback.go 97.97% 2 Missing ⚠️
bind/gen_slice.go 93.33% 1 Missing ⚠️
bind/printer.go 66.66% 1 Missing ⚠️
cmd_exe.go 75.00% 1 Missing ⚠️
... and 2 more
Additional details and impacted files
@@             Coverage Diff             @@
##           master     #409       +/-   ##
===========================================
+ Coverage   74.31%   89.34%   +15.03%     
===========================================
  Files          25       33        +8     
  Lines        4653     4929      +276     
===========================================
+ Hits         3458     4404      +946     
+ Misses        960      525      -435     
+ Partials      235        0      -235     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@b-long
b-long marked this pull request as ready for review November 3, 2026 02:06
@b-long
b-long requested a review from rcoreilly November 3, 2026 02:06
@b-long

b-long commented Nov 3, 2026

Copy link
Copy Markdown
Member Author

If anyone has time, I'd love a code review on this PR as it is fairly large. That said, I have increased test coverage significantly, and I am happy to field bugs after this is merged and released.

Please let me know if you find something that you'd like to see fixed, and thank you for any/all feedback. 🙇

cc/ @mspiller @matejsp @satarsa @rcoreilly

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant