Structured delta algebra and execution research.
The goal is not to make a CPU imitate a GPU kernel-by-kernel. The goal is to replace dense recomputation with exact state transitions whose work scales with the intrinsic structure of the change.
- exact normalized weighted-sum updates with cancellation-gated rebasing;
- sparse × low-rank delta algebra and closed crossover rule;
- factor propagation through linear maps without dense materialization;
- exact structured RMSNorm and LayerNorm rewrites;
- local operator composition, connected-component dirty cones, and geometry-aware tile scheduling;
- factor-ledger compaction without dense reconstruction;
- a cost router for full vs sparse vs sparse×low-rank execution;
- a
DirtyGraphCompilerthat compiles dirty support into optimal local tiles; - exact cached global-reduction / projected-reduction delta consumers;
- FP64 sufficient-statistics state for moments, Gram/covariance and normal equations;
- shared Woodbury state for solve/logdet/inverse-action fanout;
- implicit row/column low-rank updates for pairwise kernels;
- Cholesky update/downdate and Schur block insert/delete routes;
- sparse DFT and dynamic butterfly-DAG reference paths;
- a typed representation-switching delta IR;
- a consumer-aware CPU↔GPU hardware router calibrated against resident V100 baselines;
- output-demand routing (
edit support × requested outputs); - cacheless local reconstruction and lifetime/residency planning;
- lazy normalized query reconciliation;
- lazy Gram state with streaming Solve/Logdet consumers;
- whole-graph representation/device shortest-path planning;
- an Origami Temporal Delta Machine for token-to-token state transitions;
- exact temporal Linear, RMSNorm→multi-projection and quantized activation-code updates;
- a delta-native input-direction weight layout and compressed-codec callback ABI;
- certified MoE top-k routing;
- flat and tensor-train compiled partition-function attention;
- cross-lane/speculative low-rank factorization;
- a shipped Origami/Beast temporal-delta census CLI;
- Remath semantic auto-compilation into temporal rewrite families;
- Remath portability/observability integration receipts under
tools/remath/; - regression tests for every exact rewrite;
- machine-readable result records.
src/wirkung_math/— executable algebra/runtime primitivesexperiments/— numbered reproducible experimentstests/— correctness suitedocs/ALGEBRA.md— indexed algebra with code pointersdocs/ARCHITECTURE.md— runtime modeldocs/RESULTS.md— chronological measured resultsdocs/HARDWARE_ROUTER.md— measured CPU↔GPU routing rulesdocs/FINAL_SYNTHESIS_2026-11-05.md— final breakthrough map and overlooked leversdocs/ORIGAMI_TEMPORAL_DELTA_MACHINE_2026-11-05.md— transformer state-machine architecturedocs/ORIGAMI_DELTA_NATIVE_LAYOUT_2026-11-05.md— direction-major codec layout and 160–365 tok/s architecture budgetsdocs/ORIGAMI_TEMPORAL_IP_CLAIM_MAP_2026-11-05.md— technical invention-disclosure mapdocs/REAL_MODEL_TEMPORAL_VALIDATION_2026-11-06.md— real SmolLM2/TinyStories decode validation and measured boundariestools/remath/— leak-safe Remath portability integrationresults/— machine-readable imported/generated recordsscripts/origami_temporal_census.py— actual Origami/Beast temporal geometry censusscripts/run_and_record.py— reproducible experiment recorder
python -m pip install -e '.[dev]'
pytest -q
python scripts/run_all.py
python scripts/run_and_record.py --out results/my_run.jsonFor sequence length n, edit count e, width d, rank r:
- dense full update:
n²d - direct sparse update:
ned - sparse × low-rank update:
nr(e+d)
The factorized route beats direct sparse exactly when
r < ed/(e+d).
The temporal transformer extension adds two more independent axes:
- changed activation/code directions;
- outputs/queries actually requested.
Its core execution law is:
work ∝ temporal information change × requested output × surviving structure.
Direct resident-V100 records include:
- nonlinear local residual graph: 235.17× at 8192²;
- moments + Gram: 1152.60× for 64 changed rows at 33,554,432×64 FP64;
- supplied one-row full run: 1627.18×;
- composed Gram + 16-RHS Solve + Logdet: 225.93×;
- RBF→Solve: CPU structured route still wins at 16 changed points.
Origami Temporal Delta Machine controlled prototypes include:
- direction-major exact RMSNorm→Linear: 60–121× for 4–32 changed directions at d=2560;
- exact certified Beast-sized MoE router: 18.67×;
- exact compiled partition attention: 201.10× on a 131k-context discrete-signature workload;
- quantized SwiGLU→down: 11.73× when 0.25% of activation codes change;
- 80-lane shared temporal low rank: 8.60× at rank 1.
The measured Beast ctx512 operation budget is 134.13 ms/token (~7.46 tok/s). Composition of controlled temporal routes yields architecture targets of 163–365 tok/s. origami_temporal_census.py replaces route assumptions with actual Origami layer distributions.
The repository contains direct resident-V100 comparisons, not only CPU-vs-CPU baselines. The router is consumer-aware: requiring a dense intermediate and consuming a structured delta are different execution problems.
The next hardware phase diagram is:
CPU_delta ↔ GPU_delta ↔ GPU_full.
The temporal runtime has now been exercised on two public causal decoders rather than synthetic operators only:
- SmolLM2-135M-Instruct — 134.5M parameters, 30 Llama layers;
- TinyStories-1M — 3.75M parameters, 8 GPT-Neo layers.
Measured conclusions:
- raw FP32 / high-bit coordinate deltas are dense on ordinary checkpoints;
- simple fixed temporal PCA, static short-block response VQ, absolute signed-digit planes, and loose vocabulary Cauchy bounds do not provide a profitable universal route;
- deterministic response-quantized integer accumulators make temporal and full compiled arithmetic identical;
- a complete TinyStories 2-bit event executor gives 1.043× end-to-end temporal/full speedup inside identical compiled semantics;
- a mixed per-operator Sigma-Delta compiler preserves a separate 128/128-token FP32 greedy sequence exactly, but ~90.6% event density leaves it below the full FP32 crossover;
- real low-bit block transitions can repeat above 90% on selected TinyStories operators;
- exact vocabulary demand can collapse 50,257 rows to a median 2 final candidates, although the tiny 64-wide head remains below the hardware crossover for staged certification.
The resulting compiler target is therefore per-operator precision × event representation × hardware route, with deterministic full execution as the cost/quality fallback.