Add streaming one-wesolowski compaction APIs

[hoffmang9]

Summary

• add a new c_bindings/fast_wrapper API surface for streaming one-wesolowski proof generation when y_ref is known up front
• include incremental GetBlock and memory-budgeted (k, l) tuning to improve compaction-worker throughput/memory behavior
• add minimal embedding/build support (vdf_fast_pairindex, quiet_mode, fastlib target, PIC/PIE options) and document the path in docs/bluebox_compaction.md

Test plan

• Verify commit signature on the branch tip (git log -1 --show-signature)
• Confirm branch diff against origin/main only includes intended PR1 files
• Build/test on target platforms (Linux x86_64, macOS Intel, macOS ARM, Windows)

Made with Cursor

---

Note

High Risk
Touches core VDF execution (repeated_square callbacks, counter slotting, stdout behavior) and adds substantial new proving logic and build targets, so regressions could affect correctness/performance or embedded concurrency.

Overview
Adds a new src/c_bindings/fast_wrapper C API for fast one-wesolowski proving, including a streaming mode that requires y_ref up front (plus an optional incremental GetBlock implementation), a memory-budgeted (k,l) tuner, progress callbacks, and optional per-thread debug stats/parameter introspection; also adds a batch API for multiple streaming jobs.

Updates the core VDF loop to support embedding/multi-worker use: introduces quiet_mode to suppress stdout, assigns per-thread vdf_fast_pairindex() slots for fast counters, and adds OnBatchStart/OnBatchReplay callback hooks to allow streaming bucket updates to be rolled back when the fast path replays a batch.

Improves build/CI support by adding a fastlib static library target (libchiavdf_fastc.a), optional PIC/PIE flags and asm compilation rules in Makefile.vdf-client, ensures cmake is present on macOS GitHub runners, and documents the compaction path in docs/bluebox_compaction.md.

^{Written by Cursor Bugbot for commit 707b2f4. This will update automatically on new commits. Configure here.}

[hoffmang9]

@Ealrann I'm hoping to cleanly upstream your changes so you can rely on chiavdf directly and not a fork. See the plan here:
https://gist.github.com/hoffmang9/6a848ff22f7cd29f4b6507600b099a5c

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.}

[cursor]

Pairindex modulo can divide by zero

High Severity

vdf_fast_pairindex() computes slot % kSlots where kSlots comes from sizeof(master_counter). With the new CHIA_VDF_FAST_COUNTER_SLOTS macro, setting it to 0 makes master_counter length 0, and the modulo becomes undefined behavior (likely crash) in fast VDF mode.

Additional Locations (1)

• src/threading.h#L568-L575

 

[Ealrann]

Oh, yes the plan is perfect. I'll definitely update the wesoforge client to use the main chiavdf.
It's ok if you keep enable_threads=true indeed, I'll use repeated_square_fast_single_thread instead.

Note: the plan mentions "Trick 2 (unreleased, on side branch)", but in fact trick 2 was properly merged on bbr branch (it's the commit Ealrann@f3c73bf). This one is important because it allows the group optimisation.
Commit Ealrann@445cb0d is also very important because it fixes a memory leak (also in bbr branch), it was problematic on high core count servers.