fix(datanode): raise default data volume so search cache fits; document data tiering#112
Open
monrax wants to merge 2 commits into
Open
fix(datanode): raise default data volume so search cache fits; document data tiering#112monrax wants to merge 2 commits into
monrax wants to merge 2 commits into
Conversation
…ering enabled
The default datanode.config.nodeSearchCacheSize (10gb) exceeded the default
datanode data volume (8Gi, ~7gb usable). node_search_cache_size reserves disk on
the datanode data volume, so once an S3/path_repo snapshot repository is
configured the datanode fails preflight ("not enough usable space for the node
search cache") and CrashLoops. Plain installs are unaffected because the check
only runs when a repository is enabled.
- values.yaml: datanode.persistence.data.size 8Gi -> 20Gi so the default cache
fits alongside hot index data.
- NOTES.txt: warn at install/upgrade time when tiering is enabled and the cache
does not fit the data volume (skipped when an existingClaim is used).
Co-Authored-By: Claude Opus 4.8 (1M context) <[EMAIL_ADDRESS_REDACTED]>
- README: add an "Enable Data Tiering" usage section covering the S3 client config, the AWS https/path-style overrides, and the cache-vs-data-volume sizing note; add ToC entries for Geolocation and Data Tiering; fix a broken TLS anchor; use absolute URLs for repo doc links so they resolve on ArtifactHub. - docs/data-tiering.md: new guide with AWS S3 and MinIO walkthroughs, handing off to go2docs for the Enterprise warm-tier UI workflow. Co-Authored-By: Claude Opus 4.8 (1M context) <[EMAIL_ADDRESS_REDACTED]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The default
datanode.config.nodeSearchCacheSize(10gb) exceeded the default Data Node data volume (8Gi, ~7gb usable). Sincenode_search_cache_sizereserves space on the Data Node data volume (disk, not memory), enabling an S3 (orpath_repo) snapshot repository makes the Data Node fail preflight withnot enough usable space for the node search cacheand crash-loop. Plain installs are unaffected as the cache validation only runs when a repository is configured. This PR aligns the defaults and documents data tiering setup.Details
values.yaml:datanode.persistence.data.size8Gi->20Giso the default10gbcache fits alongside hot index data.NOTES.txt: install/upgrade-time warning when data tiering is enabled and the cache does not fit the data volume (skipped when anexistingClaimis used).README.md: new Enable Data Tiering usage section (S3 client config, AWShttps/path_style_access=falseoverrides, cache-vs-volume sizing note); ToC entries for Geolocation and Data Tiering; fixed a broken TLS anchor; repo doc links switched to absolute URLs so they resolve on ArtifactHub.docs/data-tiering.md: new guide with AWS S3 and MinIO walkthroughs, linking to go2docs for the Enterprise warm-tier UI workflow.PR Checklist
Please check the items that apply to your change.
Testing Checklist
Static Validation
helm lint ./charts/grayloghelm template graylog ./charts/graylog --validateInstallation
helm install graylog ./charts/graylogkubectl rollout status statefulset/grayloghelm test graylogFunctional (if applicable)
Upgrade (if applicable)
Specific to this PR
CrashLoopBackOffon the old8Gidefault +10gbcache (not enough usable space... only 7gb available)1/1 Runningon the new20Gidefault--dry-run=client)Notes for reviewers
🤖 Generated with Claude Code