Repository navigation
Support Apple Silicon CPU inference while preserving Linux x86 behavior - #1
Merged
Merged
Conversation
There was a problem hiding this comment.
🔵 Needs a closer look
It is a cross-platform portability change touching build configuration, linker behavior, and platform-specific syscalls whose macOS build/runtime correctness cannot be independently verified in this environment and warrants human confirmation.
No open issues 🎉
What changed in this PR
This PR enables native CPU (fp32) inference on Apple Silicon (macOS arm64) while keeping Linux x86_64 behavior byte-identical. It does so by gating x86-only compiler flags, an ELF-only linker option, and Linux-specific syscalls//proc reads behind platform conditionals, preferring OpenBLAS headers over Accelerate on macOS, accepting the macOS ecCodes wheel layout, and adding documentation plus a two-platform CI workflow. It is purely a portability/build change — no model arithmetic, CUDA kernels, or tolerances are touched.
Changes:
- Restrict AVX2/FMA/F16C to x86 arches, prefer OpenBLAS
cblas.hon macOS, accepteccodeslib/lib, and gate the GNU-Wl,--disable-new-dtags,-rpathoption to non-Apple Unix. - Compile the
MADV_POPULATE_READprefetch path only on Linux (macOS uses the page-touch fallback) and read macOS RSS via Mach APIs with correct byte→KiB/GB conversions forru_maxrss. - Add
docs/apple-silicon.md, a README note, a small CPU/GRIB test requirements file, and aCPU portabilityCI matrix (Linux x86_64 + macOS arm64) running builds, unit tests, synthetic parity, and a repeated-inference smoke test.
| File | Description |
|---|---|
CMakeLists.txt |
Gates x86 SIMD flags by processor, prefers OpenBLAS headers on macOS, adds eccodeslib/lib, and scopes the ELF rpath linker option to non-Apple Unix. |
src/varda/format.cpp |
Compiles the MADV_POPULATE_READ prefetch branch on Linux only; macOS falls back to page touching. |
apps/varda-run.cpp |
Reads macOS RSS via Mach task_info/getrusage with byte conversions; Linux /proc reporting unchanged. |
.github/workflows/cpu-portability.yml |
New CI matrix building CPU+GRIB and running tests/parity/smoke on Linux x86_64 and macOS arm64. |
python/tests/requirements-cpu.txt |
Minimal numpy + ecCodes/eckit wheels for synthetic parity, pinned to existing reference-lock versions. |
docs/apple-silicon.md |
New build/verification guide for macOS arm64. |
README.md |
Brief Apple Silicon support note pointing to the new doc. |
Review effort: Balanced
ofuhrer
marked this pull request as draft
October 23, 2026 18:42
ofuhrer
marked this pull request as ready for review
October 23, 2026 18:44
frazane
pushed a commit
that referenced
this pull request
Oct 24, 2026
Adds native Apple Silicon GPU inference through `--device metal --precision fp32`, using MPS matrix multiplication and Metal compute shaders for graph attention, normalization, GELU, residuals and bounding. Builds on the Apple Silicon CPU support merged in #1 and targets `main`. The backend uses shared reusable buffers and destination-row chunking. LayerNorm cooperates across a SIMD group, queries its width at runtime, and computes centered, compensated FP32 moments to avoid cancellation in nearly constant rows with large offsets. Command buffers batch up to 64 encoded operations, preserving completion waits before host reads and propagating GPU errors. Shader source is embedded in the executable; no PyTorch dependency or CPU compute fallback. Host rollout, the GRIB forecast CLI and the setup wrapper accept Metal. `VARDA_METAL=OFF` removes Apple GPU framework dependencies; Linux defaults to OFF and never enables Objective-C++. Existing CPU/CUDA numerical kernels and tolerance files are unchanged. FP16 remains explicitly unsupported. Validation on Apple M1: - 22 GPU model parity runs: six existing models with default/37-row chunks, plus three independently generated models covering 7/35/96 channels, one hidden node, chunks down to one row and 8,201 data nodes crossing the default chunk boundary. Existing FP32 tolerances pass; worst activation `proc.1.out` in model d: `rel_absmax` 2.72e-6, `max_abs` 4.18e-5, relative tolerance 1e-5. Hooked and hook-free outputs are bitwise equal. - A new CTest directly checks the production LayerNorm shader against double-precision mathematical references: 384 cases, including large-offset low-variance inputs and output guards. Worst `rel_absmax` 3.39e-7, `max_abs` 1.62e-6 at tolerance 1e-5. The original shader fails 90 cases; the revised shader passes all. - All six integration tests pass, including repeated inference, four-step host rollout with forcings compared against CPU, empty graphs, precision/chunk rejection and the additional model cases. CI enables Metal API validation for model parity/integration and both API/shader validation for the direct production kernel test. The full suite also passes with both validation layers on the physical M1. Global shader instrumentation changes MPS GEMM results on the hosted Paravirtual GPU (confirmed by an API-only versus instrumented comparison), so hosted full-inference checks disable that instrumentation; the direct kernel test keeps it enabled. - Metal-enabled build: 9/9 unit tests and 18/18 CPU parity checks. Metal-disabled build: 8/8 unit tests. Ruff and actionlint pass. Earlier checks also verified CMake 3.24.3/4.4.4 and byte-identical CPU outputs for all six models on Linux x86_64 against #1; current native Linux/macOS CI runs on each push. Two alternating 30-call FP32 microbenchmark rounds on M1, discarding the first three calls, measured warm mean forward times of 40.8 -> 25.9 ms (synthetic e) and 53.2 -> 31.1 ms (synthetic d) after parallel normalization and bounded batching. Repeats are bitwise stable. These synthetic timings do not establish full-Varda speed. Limits: FP32 only, host-driven rollout, no Metal graph capture. The full checkpoint and real-data GRIB forecasts still need qualification on a Mac with more unified memory and real reference assets; the 16 GB test Mac cannot hold the full model. CUDA execution was not retested. GPU-less CI runners explicitly report GPU execution as skipped. [Linux x86_64 and macOS arm64 CI passed at 317c63e](https://github.com/frazane/varda.cpp/actions/runs/40015739565). The macOS job executed GPU checks on an Apple Paravirtual device.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Apple Silicon CPU builds currently encounter x86-only compiler flags, Accelerate headers paired with OpenBLAS, and an ELF-only linker option. This adds native macOS arm64 CPU/GRIB support while retaining the existing Linux x86 compiler flags, linker options, prefetch path and numerical kernels.
MADV_POPULATE_READvalue to Darwin.ru_maxrssfrom bytes; Linux memory reporting stays unchanged.Validation:
rel_absmax1.17e-6 within the existing tolerances, repeated inference bitwise equal. Clean builds with CMake 3.24.3 and 4.4.4.4b491a0519), in the same Debian x86_64 container under emulation: both versions pass all 8 unit tests and 18 synthetic checks; output files for all six synthetic models are byte-identical. The x86 compiler flags are identical.actionlintwith ShellCheck).Scope: CPU fp32 only; no Metal/MPS backend, CUDA kernel changes, or tolerance changes. Full Varda forecasts and real-data parity have not been run on the 16 GB Mac. Existing tests skip optional checkpoint, recorded-forcing and real-GRIB fixture checks when those assets are absent. CUDA execution has not been retested.