Skip to content

feat(gcp): refactor GCP modules into composer + submodules (full) #233

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
micheledaddetta-databricks wants to merge 14 commits into
base: main
Choose a base branch
from
Draft

feat(gcp): refactor GCP modules into composer + submodules (full) #233

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
2bfd9bd
docs: add design spec for GCP modules refactor
micheledaddetta-databricks May 14, 2026
d3e7b87
docs: add implementation plan for GCP modules refactor
micheledaddetta-databricks May 14, 2026
a4ce05a
build: add Makefile recursion for modules/gcp/ submodules
micheledaddetta-databricks May 14, 2026
bc0312c
feat(gcp/network): VPC create/existing/hub + fixtures
micheledaddetta-databricks May 14, 2026
4cd9b5b
feat(gcp/private-connectivity): PSC + firewall + fixtures
micheledaddetta-databricks May 14, 2026
2834e04
feat(gcp/account): mws_* resources + fixtures
micheledaddetta-databricks May 14, 2026
1094c23
feat(gcp/dns): hub + spoke private zones + fixture
micheledaddetta-databricks May 14, 2026
ce76df1
feat(gcp/databricks-workspace): composer + preconditions + fixtures
micheledaddetta-databricks May 14, 2026
334cc32
refactor(gcp/service-account): relocate from modules/gcp-sa-provisioning
micheledaddetta-databricks May 14, 2026
085fa2b
refactor(gcp/unity-catalog): relocate from modules/gcp-unity-catalog
micheledaddetta-databricks May 14, 2026
4b617d8
docs(gcp): terraform-docs READMEs for new submodules
micheledaddetta-databricks May 14, 2026
9a07265
refactor(examples/gcp): migrate to new composer + add existing-vpc
micheledaddetta-databricks May 22, 2026
640d4ef
chore(gcp): remove deprecated modules and junk directories
micheledaddetta-databricks May 22, 2026
39954d9
docs: refresh GCP examples and modules in top-level README
micheledaddetta-databricks May 22, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 12 additions & 6 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,9 +62,11 @@ The folder `examples` contains the following Terraform implementation examples :
| AWS | [aws-databricks-uc-bootstrap](examples/aws-databricks-uc-bootstrap/) | AWS UC |
| AWS | [aws-remote-backend-infra](examples/aws-remote-backend-infra/) | Simple example on remote backend |
| AWS | [aws-workspace-config](examples/aws-workspace-config/) | Configure workspace objects |
| GCP | [gcp-sa-provisionning](examples/gcp-sa-provisionning/) | Provisionning of the identity with the permissions required to deploy on GCP. |
| GCP | [gcp-basic](examples/gcp-basic/) | Workspace Deployment with managed vpc |
| GCP | [gcp-byovpc](examples/gcp-byovpc/) | Workspace Deployment with customer-managed vpc |
| GCP | [gcp-sa-provisioning](examples/gcp-sa-provisioning/) | Provisioning the identity (service account) with permissions required to deploy on GCP |
| GCP | [gcp-basic](examples/gcp-basic/) | Workspace deployment with Databricks-managed VPC |
| GCP | [gcp-byovpc](examples/gcp-byovpc/) | Workspace deployment with customer-managed VPC (Terraform creates the VPC) |
| GCP | [gcp-existing-vpc](examples/gcp-existing-vpc/) | Workspace deployment into a pre-existing VPC |
| GCP | [gcp-with-psc-exfiltration-protection](examples/gcp-with-psc-exfiltration-protection/) | Workspace with PrivateLink (PSC), private DNS, and restricted egress (hub-and-spoke topology) |
### Modules

The folder `modules` contains the following Terraform modules :
Expand All @@ -89,9 +91,13 @@ The folder `modules` contains the following Terraform modules :
| AWS | [aws-workspace-with-firewall](modules/aws-workspace-with-firewall/) | Provisioning AWS Databricks E2 with an AWS Firewall |
| AWS | [aws-exfiltration-protection](modules/aws-exfiltration-protection/) | An implementation of [Data Exfiltration Protection on AWS](https://www.databricks.com/blog/2021/02/02/data-exfiltration-protection-with-databricks-on-aws.html) |
| AWS | aws-workspace-with-private-link | Coming soon |
| GCP | [gcp-sa-provisionning](modules/gcp-sa-provisionning/) | Provisions the identity (SA) with the correct permissions |
| GCP | [gcp-workspace-basic](modules/gcp-workspace-basic/) | Provisions a workspace with managed VPC |
| GCP | [gcp-workspace-byovpc](modules/gcp-workspace-byovpc/) | Workspace with customer-managed VPC. |
| GCP | [gcp/databricks-workspace](modules/gcp/databricks-workspace/) | Composer that orchestrates network, PSC, account, and DNS submodules based on scenario flags |
| GCP | [gcp/network](modules/gcp/network/) | VPC, subnet, router, NAT, peering, and shared-VPC binding (create or data-source lookup) |
| GCP | [gcp/private-connectivity](modules/gcp/private-connectivity/) | PSC endpoints (frontend, backend, hub-transit) and restricted-egress firewall rules |
| GCP | [gcp/account](modules/gcp/account/) | All databricks_mws_* resources: networks, workspaces, vpc_endpoint, private_access_settings |
| GCP | [gcp/dns](modules/gcp/dns/) | Private DNS zones (gcp.databricks.com, gcr.io, googleapis.com, pkg.dev) for restricted-egress workspaces |
| GCP | [gcp/service-account](modules/gcp/service-account/) | Service account with the IAM permissions required to provision Databricks workspaces |
| GCP | [gcp/unity-catalog](modules/gcp/unity-catalog/) | Metastore, GCS bucket, storage credential, external location, and default catalog |
### CI/CD pipelines

The `cicd-pipelines` folder contains the following implementation examples of pipeline:
Expand Down
Loading