feat(codegen): structured WIT records + compiling wit-bindgen crate from AADL (#319) by avrabe · Pull Request #320 · pulseengine/spar

avrabe · 2026-07-08T17:18:27Z

Closes the WIT/Rust codegen gaps reported in #319 (items 1, 3, 4, 6, 7). Before this, spar codegen degraded a record-typed data implementation to an opaque list<u8> and emitted a Rust skeleton with no wit-bindgen wiring that never compiled. Now it emits structured WIT records and a per-process crate that compiles as a whole and builds to wasm32-wasip2.

What lands

REQ-CODEGEN-WIT-RECORDS-001 — structured records + world⟷Guest wiring

data implementation → WIT record { field: type }, recursively, dependency-first (items 1, 6).
Scalar fields map to precise WIT scalars honoring signedness/float; a field that can't be a no_alloc scalar is a hard error naming the field (user-chosen policy).
Each thread's WIT world exports the lifecycle its Dispatch_Protocol implies (Periodic → compute; else initialize/compute/finalize), replacing the opaque {thread}: func() (item 7).
A per-process binding lib.rs wires the world to Rust: wit_bindgen::generate! + impl Guest + export! (item 4).

REQ-CODEGEN-WIT-RECORDS-002 — the whole crate compiles + sound no-alloc oracle

Every classifier-typed port resolves to a real Rust type via the same DataShape the WIT records use; records become structs in a per-crate types module (can't drift from the WIT).
Per-thread files are relocated under the crate and mod-wired; the Guest delegates to each per-thread component (not todo!()).
No-alloc oracle is at the WIT level, not the wasm symbol level: a probe proved cabi_realloc is unconditional wit-bindgen-rt glue (present in both alloc and no-alloc components), so it can't discriminate. list/string are the only unbounded ABI types, so the oracle asserts an all-scalar/record model's WIT has none, with an opaque-port model as the negative control (it must contain list<u8>).

Verification (compiler-enforced, not string-matched)

emitted_crate_cargo_checks — emits the full workspace and cargo checks the whole crate; a mismatched Guest method fails E0046/E0407.
emitted_crate_builds_to_wasm32_wasip2 — builds the record + multi-process crate to the real deployment target, warning-clean.
Both are #[ignore] (they shell out to cargo), so a new codegen-oracle CI job installs wasm32-wasip2 and runs -- --ignored — otherwise cargo nextest/cargo test skip them and the gate would be empty.

Scope / deferrals (tracked, not silent)

No-alloc guarantee holds for all-scalar/record ports; an opaque top-level port deliberately keeps the list<u8> fallback (out of the guarantee).
Deferred data plane: populating ports from the imported WIT functions + persistent component state (Guest methods are free fns).
REQ-CODEGEN-VERIFY-WORKSPACE-001 (filed): spar codegen --verify test/build/all emits harnesses + a root build.rs that are inert in the generated virtual workspace — pre-existing; flagged so --verify all isn't read as producing a runnable harness.

Falsification

If a generated Guest method name diverges from its WIT world export, or a record-typed port references an undefined Rust type, the codegen-oracle job fails to compile the emitted crate. If an all-scalar/record model emits any list/string in its WIT, the no-alloc oracle fails.

🤖 Generated with Claude Code

…ord (REQ-CODEGEN-WIT-RECORDS-001) P0 of #319 item 1/3/6. Adds a red-before-green oracle: a `data implementation` with scalar subcomponents (EepromSnapshot.Impl = u32+u32+u8) referenced by a port must generate a WIT `record { addr: u32, value: u32, flags: u8 }`, not an opaque `type ... = list<u8>` blob (which needs cabi_realloc at the ABI boundary, defeating no_alloc). Currently RED — wit_gen emits list<u8>. Confirmed the classifier resolves as EepromSnapshot.Impl and the fixture instantiates cleanly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…REQ-CODEGEN-WIT-RECORDS-001) P1 of #319 items 1/3/6. Flips the red oracle from #316^ green: an AADL `data implementation` with scalar subcomponents now generates a typed WIT `record` instead of an opaque `list<u8>` blob (which would force cabi_realloc at the ABI boundary, defeating no_alloc). data implementation EepromSnapshot.Impl { addr: Word32; value: Word32; flags: Byte8; } => record eeprom-snapshot-impl { addr: u32, value: u32, flags: u8 } Mechanism: - spar-hir-def: new `GlobalScope::resolve_data_shape` + `data_size_bytes` (resolver.rs) resolve a `data` classifier to a typed `DataShape` — an impl with subcomponents becomes a `Record` (fields resolved recursively), a data type carrying `Data_Size` becomes a `Scalar`, else `Opaque`. Reuses the same classifier + Data_Size resolution the instance builder already trusts. - spar-codegen: `generate`/`generate_wit` now take `&GlobalScope`; wit_gen resolves each referenced type's shape and emits a `record` (scalar fields), a scalar alias, or the legacy `list<u8>` fallback. `wit_ident` now breaks camelCase/PascalCase word boundaries (`EepromSnapshot` -> `eeprom-snapshot`, #319 item 6) instead of collapsing them. Oracle (green, hardened): the generated record is asserted field-by-field AND fed to wit-parser to prove it is valid, bindable WIT. Existing behavior preserved (opaque -> list<u8>, scalar Data_Size -> uN); golden integration + kebab-case tests unchanged. Verified workspace-wide (cargo test --workspace --all-targets --no-run) — caught + fixed the codegen bench call site too. Scoped to P1: flat scalar records. Non-representable fields (non-power-of-2 size, nested records, opaque) keep the legacy list<u8> for now; the hard-error diagnostic (user-chosen) + nested records + Dispatch_Protocol lifecycle + wit-bindgen wiring are P2/P3. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…-WIT-RECORDS-001) P2a of #319. Implements the user-chosen no_alloc policy: a record field that cannot be encoded without heap allocation makes codegen REFUSE (return an error) rather than silently emit an allocating `list<u8>`. - New `CodegenError { errors: Vec<String> }` (Display + Error); `generate` and `generate_wit` are now fallible. The CLI prints the diagnostic and exits 1 ("refuse to generate"). - `record_fields` is now three-way: `Err(msgs)` for genuinely non-encodable fields (a non-power-of-two Data_Size, or an opaque type with neither size nor fields) — every offending field reported, not just the first; `Ok(None)` for a nested-record field (representable but not yet emitted — kept as the legacy list<u8> fallback so a *supportable* type isn't wrongly rejected, P2b); `Ok(Some(rows))` for an all-scalar record. Oracle (new, green): a record with a 3-byte field and an opaque field → `generate_wit` returns Err naming both `odd` and `blob`, and does NOT flag the representable `good: u32`. Existing record + golden + kebab tests unchanged. Verified workspace-wide (cargo test --workspace --all-targets --no-run). Deliberately scoped: only record FIELDS hard-error. Top-level opaque port types keep the legacy list<u8> (unchanged behavior). Nested records (P2b), signedness/float mapping (P2c), and wit-bindgen wiring + Dispatch_Protocol lifecycle (P3) remain. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…-CODEGEN-WIT-RECORDS-001) P2c of #319. Scalar record fields (and top-level scalar port types) now honor the Data Modeling Annex representation instead of always emitting unsigned: - `Data_Representation => Float` -> f32 (4B) / f64 (8B) - `Number_Representation => Signed` -> s8 / s16 / s32 / s64 - unspecified -> unsigned u8 / u16 / u32 / u64 (unchanged default) `DataShape::Scalar` gains a `ScalarKind` (Unsigned/Signed/Float); the resolver's `scalar_of` reads `Data_Representation`/`Number_Representation` off the data type's property associations (last `::` segment, so `Data_Model::Float` matches), defaulting to integer/unsigned. `scalar_wit(bytes, kind)` maps accordingly; a sub-32-bit float (no WIT f8/f16) is non-representable and hard-errors as a record field, per P2a. Oracle (green): a record { temp: FloatWord[4B,Float], offset: SignedWord[4B,Signed], count: Ushort[2B] } generates `record { temp: f32, offset: s32, count: u16 }` and binds under wit-parser. Existing unsigned mappings (clock64->u64, flag8->u8) unchanged. Verified workspace-wide. Note: field named `offset` not `delta` — `delta` is a spar reserved keyword (DELTA_KW) and, used as a subcomponent identifier, silently drops all following declarations with no diagnostic (a separate spar parser-robustness gap). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…ODEGEN-WIT-RECORDS-001) P2b of #319 — completes P2. A record field whose type is itself a `data implementation` now becomes a nested WIT record, emitted dependency-first and referenced by name, instead of falling back to `list<u8>`. data implementation Packet.Impl { head: data Header.Impl; body: data Word32; } => record header-impl { hi: u16, lo: u8 } record packet-impl { head: header-impl, body: u32 } - `DataShape::Record` carries its source `type_name` so nested records can be named consistently (resolver sets it from the classifier). - `flatten_record` recursively collects every distinct record definition into a dependency-first list (post-order; `seen` dedups shared nested types) and references nested records by their WIT type name. Non-encodable fields still hard-error (P2a) recursively through nesting. Oracle (green): nested `header-impl` is emitted BEFORE the outer `packet-impl` that references it (`head: header-impl`), no `list<u8>`, binds under wit-parser. Existing flat-record / hard-error / signedness tests unchanged. Verified workspace-wide. P2 complete (hard-error + signedness/float + nested records). Remaining: P3 — wit-bindgen Guest/export! wiring + Dispatch_Protocol lifecycle + the compile/zero-cabi_realloc acceptance oracle. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…ycle (REQ-CODEGEN-WIT-RECORDS-001) Closes #319 items 4 & 7 (WIT<->Rust disconnect; opaque world export). The generated interface wiring is compiler-enforced, not string-hoped. - lib.rs: shared Lifecycle / lifecycle_for(dispatch) / dispatch_protocol(inst,idx) helpers — the single source the WIT world exports, the Rust trait/Guest, and the test harness all derive from (Periodic -> compute only; else initialize/compute/finalize). - wit_gen: each thread exports its Dispatch_Protocol lifecycle as {instance}-{method} funcs, replacing the opaque single {thread}: func(). wit_ident is now pub(crate) so rust_gen derives exactly-matching names. - rust_gen::generate_process_bindings: emit crates/{proc}/src/lib.rs with wit_bindgen::generate! + impl Guest (todo!() stub bodies) + export!, and make the per-thread {Struct}Component trait lifecycle-derived. workspace_gen no longer emits the placeholder crate lib.rs (bindings own it). - test_gen: harness is now lifecycle-aware — a Periodic thread's harness calls and defines tests for `compute` only, never the initialize/finalize the trait no longer has. Keeps generated test output coherent with the trimmed trait. - Oracle tests/lifecycle_bindings.rs: fast string-alignment tests + a compiler-enforced #[ignore] gate (emitted_crate_cargo_checks) on a RECORD-bearing, MULTI-PROCESS, mixed-dispatch model — it emits the full workspace and runs `cargo check`. wit-bindgen derives the Guest trait from the world, so a name-mangling/keyword/collision bug (or a wit-package/path collision across the two member crates) is a compile error (verified GREEN; a deliberately renamed Guest method fails E0046/E0407). - Export names use the SUBCOMPONENT INSTANCE name (unique per composite); both generators read child.name so world and Guest agree by construction. - artifacts: add REQ-CODEGEN-WIT-RECORDS-001 (referenced by 5 prior commits but never created) + successor REQ-CODEGEN-WIT-RECORDS-002 (whole-crate compile + wasm cabi_realloc symbol check) + TEST-CODEGEN-WIT-RECORDS + TEST-CODEGEN-WIT-BINDINGS. Remaining (REQ-CODEGEN-WIT-RECORDS-002): define Rust *Impl record structs + module-wire the per-thread files so the WHOLE crate compiles (Guest delegates to per-thread components instead of todo!()), and the wasm zero-cabi_realloc symbol check (structurally wasm-toolchain-gated). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…T oracle (REQ-CODEGEN-WIT-RECORDS-002) Closes #319 item 3. The generated per-process crate now compiles as a whole — not just the binding lib.rs — and the no_alloc guarantee has a sound oracle. - rust_gen: resolve each classifier-typed port to a real Rust type via the SAME `DataShape` wit_gen uses — `rust_scalar`/`data_shape_to_rust_type` (scalar -> u8..u64/i8..i64/f32/f64; record -> a `crate::types` struct; opaque -> Vec<u8>). New `generate_types_module` emits `crates/{proc}/src/types.rs` with the record structs (nested-first, deduped, derived from the same shape as the WIT records so they can't drift). Per-thread files are RELOCATED under the crate (`crates/{proc}/src/{thread}.rs`) and `mod`-wired from the binding lib.rs, whose `Guest` now DELEGATES each export to the per-thread `{Struct}Component` (construct-on-call) instead of `todo!()`. A crate-level `#![allow(unsafe_op_in_unsafe_fn)]` keeps the edition-2024 wit-bindgen glue warning-clean. - build_gen: the contract build.rs now reads the relocated `crates/{proc}/src/{thread}.rs` sources. - NO-ALLOC ORACLE is at the WIT level, not the wasm symbol level. A probe showed `cabi_realloc` is UNCONDITIONAL wit-bindgen-rt glue — exported and called in both a scalar-record and a `list<u8>` component — so a symbol check cannot discriminate. `list`/`string` are the only unbounded canonical-ABI types, so the oracle asserts the WIT of an all-scalar/record model has none, with an opaque-port model as the negative control (it deliberately keeps the P2a `list<u8>` fallback and MUST contain it). - Oracles (tests/lifecycle_bindings.rs): no_alloc WIT check + negative control (fast); emitted_crate_cargo_checks now compiles the WHOLE crate; new emitted_crate_builds_to_wasm32_wasip2 (#[ignore], stronger evidence) — all GREEN. - golden_test: updated for the new per-thread file location. SCOPE: the no-alloc guarantee holds only for models whose ports are all scalar/record; an opaque top-level port is out of it by design. Deferred (data plane): populating ports from the imported WIT functions + persistent component state (Guest methods are free fns). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…-RECORDS-001/002) `cargo nextest`/`cargo test` do not run #[ignore] tests, so the compiler-enforced codegen oracles — emitted_crate_cargo_checks (world⟷Guest alignment) and emitted_crate_builds_to_wasm32_wasip2 (whole-crate compile + no-alloc ABI) — were never gating in CI: a contributor could break generated-crate compilation or world/Guest alignment and CI would stay green. Add a dedicated `codegen-oracle` job that installs the wasm32-wasip2 target and runs `-- --ignored`, making the merge gate for these features non-empty. Note the CI coverage in the two TEST artifacts so `status: passing` no longer over-implies default-`cargo test` coverage. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…REQ-CODEGEN-VERIFY-WORKSPACE-001) Surfaced during REQ-CODEGEN-WIT-RECORDS-002: `spar codegen --verify test/build/all` emits harnesses + a timing-contract build.rs that are structurally inert in the generated virtual workspace (test harnesses use `super::` outside any crate member; the root build.rs never runs because a `[workspace]`-only manifest has no build script). Both predate REQ-002. Filed as a proposed requirement so `--verify all` is not read as producing a runnable verification harness until the harnesses/contract are relocated into the per-process crates. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

… -D warnings) Clippy `doc list item overindented` on the module doc comment's ordered list. Align the list markers and continuations so `cargo clippy --workspace --all-targets -- -D warnings` passes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

codecov · 2026-07-08T17:34:02Z

Codecov Report

❌ Patch coverage is 91.80328% with 75 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
crates/spar-codegen/src/rust_gen.rs	79.47%	47 Missing ⚠️
crates/spar-codegen/src/wit_gen.rs	97.90%	10 Missing ⚠️
crates/spar-codegen/src/lib.rs	78.04%	9 Missing ⚠️
crates/spar-hir-def/src/resolver.rs	89.70%	7 Missing ⚠️
crates/spar-cli/src/main.rs	33.33%	2 Missing ⚠️

📢 Thoughts on this report? Let us know!

github-actions · 2026-07-08T17:34:14Z

Rivet verification gate

✅ 20/20 passed

	count
Passed	20
Failed	0
Skipped (no steps)	0

Filter: (and (= type "feature") (or (has-tag "v093") (has-tag "v0100")))

Failed artifacts

(none)

_{Updated automatically by tools/post_verification_comment.py. Source of truth: artifacts/verification.yaml.}

RUSTSEC-2026-0204 (published 2026-07-06): invalid pointer dereference in the `fmt::Pointer` impl for crossbeam-epoch's `Atomic`/`Shared`. Transitive via salsa/rayon; lockfile-only bump to the patched 0.9.20. `cargo audit` clean. Pre-existing on main (not introduced by this branch); fixed here so #320's Security Audit gate is green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…Q-CODEGEN-WIT-RECORDS-001) `spar_codegen::generate` took `scope` and became fallible (`-> Result`) in P1/P2a, but the fuzz target still called the 2-arg `generate(&inst, &config)` and treated the result as non-Result. The `fuzz/` crate has its own `[workspace]` (cargo-fuzz convention), so `cargo test --workspace` never compiled it — the break only surfaced when CI's fuzz-smoke job built the target. Thread `scope` and handle the `Result` with `if let Ok(..)` (an `Err` is a legitimate no_alloc refusal, not a crash, so it must not be unwrapped). Verified: `cargo check --manifest-path fuzz/Cargo.toml` compiles clean (the macOS ASan-link step is environment-only; CI builds x86_64-unknown-linux-gnu). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Follows the RUSTSEC-2026-0204 bump: cargo-vet requires the new version to be audited or exempted. Move the existing crossbeam-epoch exemption from 0.9.18 to 0.9.20 (safe-to-deploy). `cargo vet --locked` succeeds (4 fully audited, 207 exempted). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

v0.24.0 — "Structured WIT records + a compiling wit-bindgen component crate". One feature area shipped since v0.23.0 (PR #320, closes #319 items 1/3/4/6/7): - REQ-CODEGEN-WIT-RECORDS-001 (verified) — spar codegen now emits structured, no_alloc-representable WIT from real AADL: a `data implementation` becomes a WIT `record` (recursively, dependency-first), scalar fields map to precise WIT scalars honouring signedness/float (a non-encodable field is a HARD ERROR, not a silent `list<u8>`), `wit_ident` preserves PascalCase word boundaries, each thread's `world` exports its `Dispatch_Protocol`-derived lifecycle, and a per-process binding crate wires the world to Rust (`generate!` + `impl Guest` + `export!`). Because wit-bindgen derives the `Guest` trait from the world, a `cargo check` of the emitted crate is a COMPILER-ENFORCED alignment oracle. - REQ-CODEGEN-WIT-RECORDS-002 (verified) — the WHOLE generated per-process crate now compiles: classifier-typed ports resolve to real Rust types (records -> structs in a per-crate `types` module derived from the SAME DataShape as the WIT records), per-thread files are relocated under the crate and module-wired, and the `Guest` delegates to each per-thread component. The no_alloc guarantee has a SOUND oracle at the WIT level (`list`/`string` absence) after a probe showed the wasm `cabi_realloc` symbol is unconditional wit-bindgen-rt glue that cannot discriminate; an opaque-port model is the negative control. The emitted crate also builds to wasm32-wasip2. A dedicated `codegen-oracle` CI job runs the #[ignore] compile oracles so they actually gate. Honesty / scope (stated, not footnoted): the no-alloc guarantee holds only for all-scalar/record ports (an opaque top-level port keeps the `list<u8>` fallback by design). DEFERRED to successor requirements: the data plane (populating ports from imported WIT functions + persistent component state — Guest methods are free fns), and REQ-CODEGEN-VERIFY-WORKSPACE-001 (`spar codegen --verify test/build` still emits artifacts that are inert in the generated virtual workspace). Falsification: if a generated `Guest` method name diverges from its WIT world export, or a record-typed port references an undefined Rust type, the `codegen-oracle` job fails to compile the emitted crate; if an all-scalar/record model emits any `list`/`string` in its WIT, the no-alloc oracle fails. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

avrabe and others added 10 commits July 2, 2026 21:36

avrabe and others added 3 commits July 8, 2026 20:35

avrabe enabled auto-merge (squash) July 8, 2026 19:59

avrabe merged commit 9962aa9 into main Jul 8, 2026
20 checks passed

avrabe deleted the feat/codegen-wit-records branch July 8, 2026 21:46

avrabe mentioned this pull request Jul 8, 2026

chore(release): bump to v0.24.0 #322

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(codegen): structured WIT records + compiling wit-bindgen crate from AADL (#319)#320

feat(codegen): structured WIT records + compiling wit-bindgen crate from AADL (#319)#320
avrabe merged 13 commits into
mainfrom
feat/codegen-wit-records

avrabe commented Jul 8, 2026

Uh oh!

codecov Bot commented Jul 8, 2026

Uh oh!

github-actions Bot commented Jul 8, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

avrabe commented Jul 8, 2026

What lands

Verification (compiler-enforced, not string-matched)

Scope / deferrals (tracked, not silent)

Falsification

Uh oh!

codecov Bot commented Jul 8, 2026

Codecov Report

Uh oh!

github-actions Bot commented Jul 8, 2026

Rivet verification gate

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant