fix: initialize large-binary S3 bucket at startup to prevent NoSuchBucketException#4241
Merged
kunwp1 merged 4 commits intoapache:mainfrom Mar 19, 2026
Merged
fix: initialize large-binary S3 bucket at startup to prevent NoSuchBucketException#4241kunwp1 merged 4 commits intoapache:mainfrom
kunwp1 merged 4 commits intoapache:mainfrom
Conversation
Contributor
Author
|
I suggest @mengw15 to review this PR as he filed the issue. |
Contributor
|
@mengw15 Please review it. |
Contributor
|
@aicam Please do another review. |
aicam
requested changes
Mar 19, 2026
Contributor
aicam
left a comment
aicam
left a comment
There was a problem hiding this comment.
I think all S3 related operations should be in File Service for consistency
amber/src/main/scala/org/apache/texera/web/TexeraWebApplication.scala
Outdated
Show resolved
Hide resolved
common/workflow-core/src/main/scala/org/apache/texera/service/util/LargeBinaryManager.scala
Outdated
Show resolved
Hide resolved
Contributor
Author
|
@aicam Addressed. Can you review it again? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this PR?
Previously, the
texera-large-binariesS3 bucket was created lazily (only when a workflow first used alarge-binaryoperator). If nolarge-binaryhad ever been used, the bucket would not exist. This caused a NoSuchBucketException warning every timeWorkflowService.clearExecutionResourcesran at workflow initialization, since it unconditionally attempted to delete objects from the bucket.This fix mirrors the existing pattern used for the dataset bucket in
FileService. TheS3StorageClient.createBucketIfNotExistis now called once duringFileServicestartup, ensuring the bucket always exists before any workflow execution or cleanup is attempted. The redundant per-callcreateBucketIfNotExistthat was previously insideLargeBinaryManager.create()has also been removed, as it is no longer needed.Any related issues, documentation, discussions?
Fixes #4239
How was this PR tested?
Check if the error message disappears after running any workflow
Was this PR authored or co-authored using generative AI tooling?
Claude-4.6