chore(infra): retire self-hosted MinIO after DO Spaces flip

[mastermanas805]

Summary

DO Spaces is now the active object-store backend (verified live: POST /storage/new returns Spaces credentials, PUT+GET round-trip works). The self-hosted MinIO Deployment in the instant-data k8s namespace is no longer in the request path. This PR retires the local MinIO manifests and replaces every MINIO_* env injection on production k8s deployments with OBJECT_STORE_* sourced from instant-secrets / instant-infra-secrets.

DO NOT MERGE UNTIL 2026-05-12 (24h after this PR opens)

Anonymous storage tokens have a 24h TTL. Any anonymous tokens minted against MinIO in the last 24h will 404 if this Deployment is removed before TTL expires. Merging before 2026-05-12 would break those still-live tokens. Authenticated (claimed) storage resources are not at risk — they were re-minted against DO Spaces at flip time. Hold this PR for at least 24h, then merge.

Files deleted

• k8s/data/minio.yaml — Deployment, PVC (minio-data, 10Gi), cluster Service, and NodePort Service (30900 API / 30901 console)
• k8s/data/minio-bucket-init.yaml — one-shot Job that created the instant-shared bucket via mc mb --ignore-existing
• k8s/data/minio-secret.yaml — local-dev MinIO root creds Secret (minio-secrets in instant-data namespace)

Manifests still referencing MINIO_* (cleaned in this PR)

File	Change
k8s/app.yaml (instant-api)	Removed optional MINIO_ROOT_USER / MINIO_ROOT_PASSWORD secret refs. API now reads OBJECT_STORE_* from instant-secrets via internal/config/config.go.
k8s/worker/deployment.yaml (instant-worker)	Replaced the storage_bytes-scanner MINIO_* block with OBJECT_STORE_* env refs sourced from instant-infra-secrets.
k8s/provisioner/deployment.yaml (instant-provisioner)	Removed the MINIO_* block used for StorageBytes queries. Also fixes a dangling reference to a minio-secrets Secret in the instant-infra namespace that was never defined in-tree (the in-repo Secret lived in instant-data).
k8s/configmap.yaml	Removed MINIO_ENDPOINT and MINIO_BUCKET_NAME.
k8s/secrets.yaml (template)	Removed MINIO_ROOT_USER / MINIO_ROOT_PASSWORD template keys; added OBJECT_STORE_BACKEND/ENDPOINT/PUBLIC_URL/ACCESS_KEY/SECRET_KEY/BUCKET/REGION/SECURE template keys for DO Spaces.

Not touched (intentional)

• k8s/data/minio-ingress.yaml — this file is untracked locally and not part of this PR. It serves s3.instanode.dev -> the minio Service and will 404 once the Deployment is removed. A follow-up should delete it (or repoint to DO Spaces). Flagged so it's not forgotten.
• api/internal/config/config.go — keeps the legacy MINIO_* fallback so local docker-compose dev (where a developer might still have MINIO_* in .env) keeps working unchanged. The fallback chain is documented in-place; production no longer relies on it.

Post-merge checklist

Once this PR merges, run on the production cluster (and on any local Rancher Desktop clusters with the old MinIO still deployed):

# Confirm what's there first
kubectl get deploy,pvc,svc,job -n instant-data -l app=minio

# Remove the MinIO workload + storage + services + bootstrap Job
kubectl delete -n instant-data deploy/minio pvc/minio-data svc/minio svc/minio-external
kubectl delete -n instant-data job/minio-bucket-init --ignore-not-found
kubectl delete -n instant-data secret/minio-secrets --ignore-not-found

# If MinIO was deployed as a StatefulSet on prod (not the case in this repo's
# manifest, but worth checking), also:
# kubectl delete -n instant-data statefulset/minio

# Verify nothing left
kubectl get pods -n instant-data | grep -i minio   # should print nothing

# Sanity-check that the storage hot path is unaffected
curl -s -X POST http://<api-host>/storage/new | jq '.endpoint, .bucket'
# Expect a DO Spaces endpoint (e.g. nyc3.digitaloceanspaces.com), not a
# minio.instant-data.svc.cluster.local hostname.

Rollback plan

If DO Spaces shows any unforeseen issues after merge:

1. Revert this PR (or git revert <merge-sha> and push to master).
2. Re-apply the resurrected manifests:

   kubectl apply -f k8s/data/minio-secret.yaml
   kubectl apply -f k8s/data/minio.yaml
   kubectl apply -f k8s/data/minio-bucket-init.yaml

3. Roll the API and worker so they pick the MINIO_* env vars back up:

   kubectl rollout restart -n instant deploy/instant-api
   kubectl rollout restart -n instant-infra deploy/instant-worker
   kubectl rollout restart -n instant-infra deploy/instant-provisioner

4. Flip OBJECT_STORE_BACKEND=minio-admin in instant-secrets and restart instant-api to switch the credential-issuance strategy back to per-customer IAM users.

Storage data isn't lost in either direction — the PVC was on local-path storage in Rancher Desktop only; DO Spaces holds the actual production object data and was already the active backend at PR-open time.

Test plan

• Hold this PR open until 2026-05-12 (24h after open) — anonymous storage TTLs.
• Verify kubectl apply -f k8s/ on a fresh Rancher Desktop cluster succeeds without the MinIO manifests.
• Verify POST /storage/new returns DO Spaces credentials (endpoint, bucket, region) after the merge.
• Verify the worker storage_bytes scanner logs read OBJECT_STORE_* (or fail-open warn-log if scanner not yet ported).
• Run post-merge checklist above on production cluster.

Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com

[mastermanas805]

Rebase attempt 2026-05-20 hit conflicts. Still RELEVANT — k8s/data/minio.yaml + minio-bucket-init.yaml + minio-secret.yaml still exist on master, and configmap.yaml still has MinIO refs. Needs manual rebase against 1b65d9c (today's manifest reconciliation work likely caused the conflicts). Recommend opening a fresh follow-up PR against current master if reviewer effort here is too high.

Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com

[mastermanas805]

Superseded by #16 (rebased against current master after the broad manifest reconciliation landed in 1b65d9c). The new PR redoes this retirement cleanly with the post-2026-05-20 OBJECT_STORE_* abstraction wired in. Closing in favor of #16.

#16