Compiler execution and export roadmap¶
Engineering decisions recorded on 2026-10-21. Planned work does not expand the qualified backend surface. The shared execution roadmap and baseline protocol describe cross-package ownership and measurements.
Keep the public numerical contract¶
Retain Python model interfaces and namespace discovery from input arrays. Keep NumPy, JAX and CPU Torch qualification. Bounded A100 JAX/Torch/CuPy core checks and JAX/Torch statistical checks are separately recorded in the device receipts. Their scope does not qualify every operator, solver, estimator or device. Sparse operators, compilation, differentiation and device support have separate contracts.
Generated vectorized expression functions are already cached without per-expression
runtime eval. Measure normalization, compilation and evaluator layouts before
choosing an acceleration target. Optional Numba needs a supported generated
function signature; decorating an arbitrary dynamic Python closure is insufficient.
An explicitly requested unavailable/unsupported accelerator must produce a clear
result, rather than silently appearing to use that accelerator.
Compiler-owned work¶
| Work | Scope and dependency |
|---|---|
| Structural model identity | Schema-1 canonical manifests and references are implemented, including ordered layouts, typed IR, observables, constants and compilation options. Forward consumers must carry this separately from runtime/prior/design identity and compare independently expected provenance. |
| Derivative exports | Reuse checked dependency structure. State and parameter derivatives and observable derivatives have distinct shapes and meanings. Unsupported IR operations fail explicitly; qualify small dense values/actions independently before general sparse or matrix-free claims. |
| IR export | Versioned typed serialization and a bounded Julia emitter with explicit supported rational/expression semantics. Export remains separate from normal evaluation. |
| Optional Numba lowering | Explicit CPU NumPy lowering generates one packed flat-array signature with float32/64 contract checks. Cold compilation, packing and memory remain part of adoption decisions; no solver default changes. |
| Rust/PyO3 IR experiment | Deferred private implementation of normalization/tree passes, preserving Python IR/public contracts; enter after compile-time profiles or external-runtime consumers justify it. |
Julia export serves optional external structural-identifiability tools; it does not start a full Julia port. A future C/LLVM emitter is another separately scoped consumer, not an automatic consequence of serializing the IR. No Rust rewrite of the small Python array protocol is planned.
Reuse existing artifacts¶
Scaling-gauge declarations, bounded detection, verified outputs and anchored coordinate maps already exist. Inference consumers should reuse them rather than reimplement detection. Reported gauge rank or remaining parameter-coordinate count does not establish structural identifiability. Initial conditions, observation and noise parameters can change the statistical dimension.
Latent observables already compile separately from statistical measurement noise. OP Engine owns numerical observation evaluation/reduction; OP Infer owns shared sampling/log-density, censoring and prior semantics. Keep this boundary when qualifying a complete compiled simulation-to-observation workflow.
Sequence¶
The NumPy evaluator profile now identifies repeated general broadcasts of already-shaped buffers and records a targeted read-only-view optimization. Both source variants use the existing hierarchical oracle and a shared harness. Keep these RHS measurements separate from complete solver, statistical pipeline and training performance. Optional Numba and Torch compilation require distinct contracts. The optional NumPy lowering now records packing, cold/warm and memory costs; Torch sequential compilation remains separate.
First reconcile open work and run the existing hierarchical compiler baseline. Develop structural identity and forward-consumer contracts early. Derivative and IR export work can proceed alongside shared observation/prior interfaces. Qualify JAX execution and explicit GPU cases before selecting additional native kernels. Revisit Rust normalization only after those measurements and contracts stabilize.
External-tool reproducibility¶
The downstream evidence on #326 includes rank disagreements and timeouts from manually transcribed external models. Their cause remains unresolved. A generated export should remove the transcription ambiguity, not assert that an external result is correct.
Preserve stable axis-expanded names and aliases, ordered coordinates, initial conditions, known time-varying inputs versus unknown parameters/constants, and observables with their measures. Unsupported operations must have explicit diagnostics. Bind the schema/emitter version and supported-subset contract to the structural identity from #328.
On independent synthetic fixtures, compare exported expression values and state/parameter/observable derivatives against the same compiled IR. Retain tool versions, assumptions, exact/numerical reference definitions, witness points, tolerances, disagreements and censored timeouts. Different rank notions or known input assumptions must not be treated as the same calculation. The optional Julia consumer remains interoperability work alongside forward/provenance interfaces.
Portable interfaces delivered¶
Portable IR and bounded scalar derivatives now provide typed structural interchange, conservative dependencies, native dense derivative matrices and Julia RHS/observable expressions from the same compiler IR. General sparse storage, operator/history and external structural-tool qualification remain separate extensions. This establishes interfaces, not a reason to select a Rust normalization implementation.
Dependency structure additionally exports bounded conservative state and input-buffer reads, grouped observable layouts and immutable snapshots of existing checked block proofs without numerical derivative construction. Nonlocal selectors and reductions retain their coupling. Operators, history and gauge-fixed execution have explicit unsupported coverage. Dense symbolic actions remain unchanged. Optional native JAX/Torch state actions now supply RHS and shaped-observable products with checked CPU transforms and explicitly rejected operator/history/gauge execution. Measured direct actions have higher whole-worker peak RSS and slower median warm timings on the final ring fixture; consumers must measure complete solves before adoption. This establishes an input boundary for engine operator/Krylov work without changing solver defaults.