You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This addresses #628. Current formula sets 50 Mb of RAM for a single linking action which is too low and leads to bazel scheduling too many actions in parallel. This is especially painful in containerized environments (such as kubernetes) where such a behavior can easily cause OOM Killer to kill build pods.
With the new formula, first, default value is slightly higher (256 Mb) and, second, we scale more aggressively with the increasing number of inputs
This seems to be a drastic increase. Back when we came up with the numbers we did an extensive analysis to come up with the number. I wonder how it changed so signficantly. (cc @wilwell)
This seems to be a drastic increase. Back when we came up with the numbers we did an extensive analysis to come up with the number. I wonder how it changed so signficantly. (cc @wilwell)
This setting is less relevant for RBE environments where resources are managed at the cluster level (AFAIK, Google mainly relies on RBE so will barely notice this issue if ever).
Our case:
For builds on 6 GB RAM CI workers, it is critical. Bazel immediately claims 1.5 GB for its in-memory cache, but eventually needs 3 GB to handle the full action graph. Since heavy linking actions (C++/Rust) compete for the remaining RAM, the system often hits OOM limits. Using resource_set allows Bazel to throttle concurrent linking actions based on available memory, which is more scalable than manually hard-coding --jobs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This addresses #628. Current formula sets 50 Mb of RAM for a single linking action which is too low and leads to bazel scheduling too many actions in parallel. This is especially painful in containerized environments (such as kubernetes) where such a behavior can easily cause OOM Killer to kill build pods.
With the new formula, first, default value is slightly higher (256 Mb) and, second, we scale more aggressively with the increasing number of inputs