Skip to content

chore(infra): retire self-hosted MinIO — DO Spaces is canonical (supersedes #4)#16

Merged
mastermanas805 merged 1 commit into
masterfrom
chore/retire-self-hosted-minio-2026-05-20
May 21, 2026
Merged

chore(infra): retire self-hosted MinIO — DO Spaces is canonical (supersedes #4)#16
mastermanas805 merged 1 commit into
masterfrom
chore/retire-self-hosted-minio-2026-05-20

Conversation

@mastermanas805
Copy link
Copy Markdown
Member

Summary

DO Spaces (nyc3.digitaloceanspaces.com, bucket instant-shared) has been the
active object-store backend in production since 2026-05-11. The self-hosted
MinIO Deployment in the instant-data k8s namespace is no longer in the
request path for POST /storage/new.

Live evidence (pulled before this PR):

$ kubectl get secret -n instant instant-secrets -o yaml | grep -i -E 'OBJECT_STORE'
OBJECT_STORE_BACKEND: shared-key                  # alias → do-spaces (factory NormalizeBackend)
OBJECT_STORE_ENDPOINT: nyc3.digitaloceanspaces.com
OBJECT_STORE_BUCKET: instant-shared
OBJECT_STORE_REGION: nyc3
OBJECT_STORE_PUBLIC_URL: https://nyc3.digitaloceanspaces.com
OBJECT_STORE_SECURE: true

This PR retires the local MinIO manifests and replaces every MINIO_* env
injection on production k8s deployments with OBJECT_STORE_* sourced from
instant-secrets / instant-infra-secrets.

Supersedes PR #4

The original retirement work shipped as PR #4 on 2026-05-11 but stalled past
its 24h merge window (anonymous-storage TTLs) and then went stale after the
broad manifest reconciliation that landed in 1b65d9c k8s: sync app/worker/ provisioner manifests with live prod state on master. Rather than rebase a
3-week-old PR over a now-different base, this PR re-does the work cleanly
against current master (commit 1b65d9c). Close PR #4 in favor of this one.

Anonymous-storage TTL caveat from PR #4 is no longer relevant — any MinIO-
backed anonymous tokens minted on 2026-05-11 have long since expired (24h TTL,
9 days ago).

Files deleted

  • k8s/data/minio.yaml — Deployment, PVC (minio-data, 10Gi), cluster
    Service, and a ClusterIP-mode external Service
  • k8s/data/minio-bucket-init.yaml — one-shot Job that created the
    instant-shared bucket via mc mb --ignore-existing
  • k8s/data/minio-secret.yaml — local-dev MinIO root creds Secret
    (minio-secrets in instant-data namespace)

Manifests still referencing MINIO_* (cleaned in this PR)

File Change
k8s/app.yaml (instant-api) Removed optional MINIO_ROOT_USER / MINIO_ROOT_PASSWORD secret refs. OBJECT_STORE_* was already wired in the storage-abstraction work; no new env keys needed here.
k8s/worker/deployment.yaml (instant-worker) Replaced the storage_bytes-scanner MINIO_* block with OBJECT_STORE_* env refs sourced from instant-infra-secrets (which already has the keys populated live).
k8s/provisioner/deployment.yaml (instant-provisioner) Removed the MINIO_* block. Also fixes a dangling reference to a minio-secrets Secret in the instant-infra namespace that was never defined in-tree (the in-repo Secret lived in instant-data).
k8s/configmap.yaml Removed MINIO_ENDPOINT and MINIO_BUCKET_NAME.
k8s/secrets.yaml (template) Removed MINIO_ROOT_USER / MINIO_ROOT_PASSWORD template keys; added OBJECT_STORE_BACKEND/ENDPOINT/PUBLIC_URL/REGION/ACCESS_KEY/SECRET_KEY/BUCKET/SECURE template keys for DO Spaces.

Server-side dry-run (clean)

$ kubectl apply --dry-run=server -f k8s/app.yaml -f k8s/configmap.yaml \
    -f k8s/provisioner/deployment.yaml -f k8s/worker/deployment.yaml \
    -f k8s/secrets.yaml
Warning: spec.template.spec.containers[0].env[25]: hides previous definition
         of "E2E_TEST_TOKEN", which may be dropped when using apply
deployment.apps/instant-api configured (server dry run)
service/instant-api unchanged (server dry run)
configmap/instant-config configured (server dry run)
deployment.apps/instant-provisioner configured (server dry run)
deployment.apps/instant-worker configured (server dry run)
service/instant-worker created (server dry run)
secret/instant-secrets configured (server dry run)

The E2E_TEST_TOKEN and service/instant-worker created lines are
pre-existing on master, unrelated to this change.

Post-merge operator action

Per CLAUDE.md rule 15, infra has no auto-apply. The new APPLY-CHECKLIST.md
"MinIO retirement" section documents the post-merge cleanup. Summary:

kubectl get deploy,pvc,svc,job,secret -n instant-data -l app=minio
kubectl delete -n instant-data deploy/minio pvc/minio-data \
                 svc/minio svc/minio-external \
                 job/minio-bucket-init secret/minio-secrets --ignore-not-found
kubectl get pods -n instant-data | grep -i minio   # should print nothing

Rollback plan

Revert the merge commit and re-apply the deleted manifests from history:

git revert <merge-sha>
git push origin master
git show <revert-sha>~1 -- k8s/data/minio*.yaml | kubectl apply -f -
kubectl patch secret instant-secrets -n instant --type merge \
  -p "{\"data\":{\"OBJECT_STORE_BACKEND\":\"$(printf minio | base64)\"}}"
kubectl rollout restart -n instant deploy/instant-api

Storage data isn't lost in either direction — the PVC was on local-path
storage in Rancher Desktop only; DO Spaces holds the actual production
object data and is the active backend.

Coverage block (CLAUDE.md rule 17)

Symptom:       Self-hosted MinIO still deployed in instant-data while
               prod traffic is on DO Spaces
Enumeration:   rg -n -i 'minio' k8s/ --type yaml
Sites found:   8 (3 manifest files + 5 env-injection blocks across
               app/configmap/worker/provisioner/secrets)
Sites touched: 8 (3 deleted + 5 migrated to OBJECT_STORE_*)
Coverage test: kubectl apply --dry-run=server clean across all
               mutated manifests
Live verified: Operator runs APPLY-CHECKLIST.md 'MinIO retirement'
               section commands post-merge; prod request path
               verified pre-PR via OBJECT_STORE_BACKEND env on live
               instant-secrets (DO Spaces, nyc3, instant-shared).

🤖 Generated with Claude Code

…rsedes #4)

DO Spaces (nyc3.digitaloceanspaces.com, bucket instant-shared) has
been the active object-store backend in production since 2026-05-11.
The self-hosted MinIO Deployment in instant-data is no longer in the
request path for /storage/new — verified via live cluster:

  OBJECT_STORE_BACKEND=shared-key  (alias → do-spaces)
  OBJECT_STORE_ENDPOINT=nyc3.digitaloceanspaces.com
  OBJECT_STORE_BUCKET=instant-shared

This commit retires the local MinIO manifests and replaces every
MINIO_* env injection on production k8s deployments with OBJECT_STORE_*
sourced from instant-secrets / instant-infra-secrets.

Supersedes PR #4 (2026-05-11), which had drifted from current master
after the broad manifest reconciliation work landed.

Manifests deleted:
  - k8s/data/minio.yaml             (Deployment + PVC + ClusterIP + NodePort)
  - k8s/data/minio-bucket-init.yaml (one-shot Job creating instant-shared)
  - k8s/data/minio-secret.yaml      (local-dev MinIO root creds)

Env-var injection migrated MINIO_* → OBJECT_STORE_* in:
  - k8s/app.yaml             (instant-api) — removed MINIO_ROOT_USER/PASSWORD
                              optional refs; OBJECT_STORE_* keys already
                              wired via prior storage-abstraction work.
  - k8s/worker/deployment.yaml (instant-worker) — storage_bytes scanner
                              now reads OBJECT_STORE_* from instant-infra-secrets.
  - k8s/provisioner/deployment.yaml (instant-provisioner) — storage_bytes
                              scanner block migrated. Also removes the
                              dangling reference to a 'minio-secrets'
                              Secret in the instant-infra namespace that
                              was never defined in-tree (the in-repo
                              Secret lived in instant-data).
  - k8s/configmap.yaml — removed MINIO_ENDPOINT / MINIO_BUCKET_NAME.
  - k8s/secrets.yaml (template) — removed MINIO_ROOT_USER/PASSWORD
                              template keys; added OBJECT_STORE_*
                              template keys for DO Spaces.

Docs:
  - k8s/APPLY-CHECKLIST.md — new 'MinIO retirement' section with the
    post-merge operator cleanup commands (kubectl delete deploy/minio
    pvc/minio-data svc/{minio,minio-external} job/minio-bucket-init
    secret/minio-secrets in instant-data). Per CLAUDE.md rule 15
    (infra has no auto-apply), the operator runs this manually after
    merge.

Verification:
  - kubectl apply --dry-run=server -f k8s/{app,configmap,secrets,
    provisioner/deployment,worker/deployment}.yaml — all clean (no
    schema errors; pre-existing E2E_TEST_TOKEN hides-previous warning
    is unrelated to this change).
  - Live confirmation prod is on DO Spaces: instant-secrets has
    OBJECT_STORE_BACKEND=shared-key (alias to do-spaces),
    OBJECT_STORE_ENDPOINT=nyc3.digitaloceanspaces.com.

Coverage block:
  Symptom:       Self-hosted MinIO still deployed in instant-data while
                 prod traffic is on DO Spaces
  Enumeration:   rg -n -i 'minio' k8s/ --type yaml
  Sites found:   8 (3 manifest files + 5 env-injection blocks across
                 app/configmap/worker/provisioner/secrets)
  Sites touched: 8 (3 deleted + 5 migrated to OBJECT_STORE_*)
  Coverage test: kubectl apply --dry-run=server clean across all
                 mutated manifests (no resolveError on missing minio-
                 secrets, no env-var redefinition warnings beyond the
                 pre-existing E2E_TEST_TOKEN one)
  Live verified: Operator runs APPLY-CHECKLIST.md 'MinIO retirement'
                 section commands post-merge; prod request path
                 verified pre-PR via OBJECT_STORE_BACKEND env on live
                 instant-secrets (DO Spaces, nyc3, instant-shared).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@mastermanas805 mastermanas805 force-pushed the chore/retire-self-hosted-minio-2026-05-20 branch from 8efe402 to cf16b4f Compare May 21, 2026 16:06
@mastermanas805 mastermanas805 merged commit 958f7d3 into master May 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant