cargo-pgrx: build Postgres in parallel during cargo pgrx init#2291
Open
philippemnoel wants to merge 2 commits into
Open
cargo-pgrx: build Postgres in parallel during cargo pgrx init#2291philippemnoel wants to merge 2 commits into
cargo pgrx init#2291philippemnoel wants to merge 2 commits into
Conversation
`cargo pgrx init` already accepts `-j`/`--jobs` and uses it to size a `jobslot::Client`, which makes the jobserver auth available to child `make` processes via `MAKEFLAGS`. However, the `make world-bin` and `make install-world-bin` invocations were never given `-j` on their argv, so GNU make ignored the jobserver and ran serially — making `cargo pgrx init` slower than it needed to be on multi-core hosts. This pulls the existing job-count resolution into an `Init::resolved_jobs()` helper, reuses it to seed the jobserver, and passes `-jN` to both `make` invocations so GNU make actually parallelizes. With no `--jobs` flag, the default is `std::thread::available_parallelism()`, matching the previous jobserver-only behavior in spirit but now applied in practice.
Contributor
Author
|
The "verify package can build / ubuntu" run will be fixed once #2292 is merged |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
cargo pgrx initalready takes-j/--jobsand uses it to size ajobslot::Client, which exposes the jobserver auth to childmakeprocesses viaMAKEFLAGS. However, neithermake world-binnormake install-world-binwas given-jon its own argv, so GNU make ignored the jobserver and ran serially. Result:cargo pgrx initbuilt Postgres single-threaded even on big hosts.This PR:
Init::resolved_jobs()(--jobs->available_parallelism()->1)-jNto bothmakeinvocations so GNU make actually parallelizesNo new flags; the existing
--jobs/-jkeeps working and now has an observable effect on the Postgres build, not just the jobserver token pool.Test plan
cargo check -p cargo-pgrxpassescargo pgrx initrebuild is measurably faster on a multi-core host (verified locally by author)cargo pgrx init -j 1produces a serial build (verified locally by author)