[WIP] sea: generate single executable directly with Node.js binary #61167
+484,502
−961
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is not yet ready for full review, as it lacks more documentation and tests. For now, consider this as a POC for more feedback, especially the name of the command.
Instead of relying on a WASM build of postject to perform the
injection, add LIEF as dependency and generate the SEA directly
from core via a new CLI option --build-sea which takes the SEA
config. This simplifies SEA generation for users and makes it
easier to debug/maintain the SEA building process.
For the time being, backward compatibility with the postject-based SEA building process will be maintained, until there's motivation to break it (e.g. for optimizations)
The new process is simplified to as follows - no more knowledge about the blob location in the binary needed:
This idea was discussed a while back (brought up by @marco-ippolito) at the collaboration summit to improve the UX of SEA building process, which currently requires users to use an external tool (i.e. postject) to perform the injection, and know about the layout of the target binary. For most users, the details are probably not very useful. Moving the injection process into core simplifies the process.
Bringing it into core may also make it easier to implement some of the ideas brought up back in 2022 that would require close coordination with core to implement (e.g. trimming ICU data post-build to reduce binary size).
I've been helping out with the SEA feature from time to time and as I see it, apart from UX improvement, I found the current WASM-based tool somewhat difficult to debug when I was trying to fix nodejs/postject#105. Also, the WASM build is significantly slower than a native build, which adds friction to the debugging process.
At the summit @RaisinTen mentioned one concern about the potential binary size increase. With this POC the binary size is only increased by 5-6 MB on macOS/Linux, and ~3MB on Windows, which seems acceptable (as a reference, this is smaller than the decrease we recently got from compiling V8 with default hidden visibility #56290 (comment))
In addition, I think the LIEF library may also be useful for other purposes e.g. demangling the names in the V8 prof profiles, which is a current bottleneck when trying to analyze logs via
--prof-processby calling out tonmandc++filt.Commits
test: use fixture directories for sea tests
Instead of copying and writing files on the fly for SEA tests,
put the fixtures into a directory and copy them into tmpdir
for testing. This allows easier reproduction and debugging
when they do fail - we can just copy the entire fixture directory
and test directly from there.
deps: add tools and scripts to pull LIEF as a dependency
deps: add LIEF as a dependency
sea: split sea binary manipulation code
Split the sea binary manipulation code to a seperate file so that
adding more low-level binary manipulation code doesn't clobber the
higher-level code.
sea: add --build-sea to generate SEA directly with Node.js binary
Instead of relying on a WASM build of postject to perform the
injection, add LIEF as dependency and generate the SEA directly
from core via a new CLI option --build-sea which takes the SEA
config. This simplifies SEA generation for users and makes it
easier to debug/maintain the SEA building process.
test: migrate to --build-sea in existing SEA tests
Only leave a smoking test for the postject-based workflow
in test-single-executable-application.js