Add S3 addressing style configuration support#3367
Conversation
- Add adressing_style field to S3Config - Add S3_ADDRESSING_STYLE config entry with support for legacy and YAML config - Configure S3 client to use virtual addressing style when specified
|
Thank you for opening this pull request! 🙌 These tips will help get your PR across the finish line:
|
|
Why do you need addressing style? The yaml config entry will break flytectl in struct mode as that config is not there |
|
@ranjit-parva thanks for your contribution. |
Virtual Host-style and Path-style addressing differ in how the bucket name is positioned within the URL, either as a subdomain or as part of the path. Virtual Host-style (e.g., https://bucket.s3.amazonaws.com) is the current AWS standard and is required for all buckets created after September 2020 to ensure better traffic routing and scalability. Path-style (e.g., https://s3.amazonaws.com/bucket) is considered legacy but remains widely used in local development environments like LocalStack or MinIO for easier DNS configuration. By default path sytle is picked by boto. This config enables users to set virtual host style. |
…ends (#703) ## Summary - Adds `addressing_style` field to the `S3` config class, readable via the `FLYTE_AWS_S3_ADDRESSING_STYLE` env var - When set to `virtual`, passes `virtual_hosted_style_request=True` to obstore so URLs are constructed as `https://<bucket>.<endpoint>/<key>` instead of `https://<endpoint>/<bucket>/<key>` - Replicates [flyteorg/flytekit#3367](flyteorg/flytekit#3367) into flyte-sdk (v2) Signed-off-by: M. Adil Fayyaz <62440954+AdilFayyaz@users.noreply.github.com>
|
Congrats on merging your first pull request! 🎉 |
Related PR:
flyteorg/stow#20