Skip to content

Commit 0c8fe79

Browse files
intel352claude
andcommitted
docs: plugin IaC registration, multi-store secrets, per-env infra resolution
Update dsl-reference (docs/ and cmd/wfctl/embedded) with secretStores section, updated secrets section (defaultStore, per-secret store field, multi-store example), and expanded infrastructure section with per-env resolution strategies (container/provision/existing with connection config). Update WFCTL.md with 11-screen wizard reference, Ctrl+G navigation, and secrets setup command documentation. Update CHANGELOG.md with all new features from this implementation cycle. Create plugin-manifest-guide.md for plugin authors to declare infrastructure requirements. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 9a886f8 commit 0c8fe79

File tree

5 files changed

+456
-35
lines changed

5 files changed

+456
-35
lines changed

CHANGELOG.md

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -10,12 +10,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
1010
### Added
1111

1212
- **`wfctl dev`** (`cmd/wfctl/dev.go`, `dev_compose.go`, `dev_process.go`, `dev_k8s.go`, `dev_expose.go`): local development cluster management. Subcommands: `up`, `down`, `logs`, `status`, `restart`. Three modes: docker-compose (default), process (`--local`, with hot-reload via fsnotify), and minikube (`--k8s`). Exposure integrations: Tailscale Funnel, Cloudflare Tunnel, ngrok (`--expose`). Auto-detects `environments.local.exposure.method` when `--expose` is omitted.
13-
- **`wfctl wizard`** (`cmd/wfctl/wizard.go`, `wizard_models.go`): interactive Bubbletea TUI wizard for project setup. Nine screens: project info → services → infrastructure → environments → deployment → secrets → CI/CD → review → write. Generates a complete `app.yaml` and optionally triggers `wfctl ci init`. Navigates with Enter/Esc/Tab/Space/arrows.
13+
- **`wfctl wizard`** (`cmd/wfctl/wizard.go`, `wizard_models.go`): interactive Bubbletea TUI wizard for project setup. Eleven screens: project info → services → infrastructure → infra resolution (per-env strategy) → environments → deployment → secret stores → secret routing → secret values → CI/CD → review → write. New screens vs prior version: per-environment infra resolution (container/provision/existing with connection details for "existing"), named secret store configuration with add/remove flow, per-secret store routing (← → to assign), and bulk hidden secret input with Ctrl+G auto-generation for keys/tokens. Generates a complete `app.yaml` including `secretStores:` and per-secret `store:` routing.
14+
- **`wfctl secrets setup`** (`cmd/wfctl/secrets_setup.go`): standalone interactive command to set all secrets for a given environment. Reads `secrets.entries` from config, resolves store per secret (env override → per-secret store → defaultStore → legacy provider), prompts with hidden terminal input, and supports `--auto-gen-keys` flag to auto-generate random hex values for names ending in `_KEY`, `_SECRET`, `_TOKEN`, or `_SIGNING`.
15+
- **Plugin manifest `moduleInfraRequirements`** (`config/plugin_manifest.go`): `PluginManifestFile` struct with `moduleInfraRequirements` map, `ModuleInfraSpec`, and `InfraRequirement`. Allows plugin authors to declare infrastructure dependencies (type, name, Docker image, ports, secrets, providers) per module type.
16+
- **Multi-store secrets** (`config/secrets_config.go`, `config/config.go`): `SecretStoreConfig`, `SecretStores` map on `WorkflowConfig`, `DefaultStore` on `SecretsConfig`, and `Store` field on `SecretEntry` for per-secret store routing.
17+
- **Per-environment infra resolution** (`config/infra_resolution.go`): `InfraEnvironmentResolution` with `strategy` (container/provision/existing), `dockerImage`, `port`, `provider`, `config`, and `connection` (host/port/auth). Added `Environments` map to `InfraResourceConfig`.
18+
- **`SecretsProvider.Check()`** (`cmd/wfctl/secrets_providers.go`): `SecretState` enum (Set/NotSet/NoAccess/FetchError/Unconfigured) and `Check()` method on the interface with `envProvider` implementation. `SecretStatus` now includes `Store` and `State` fields.
19+
- **Multi-store secret resolution** (`cmd/wfctl/secrets_resolve.go`): `ResolveSecretStore` (priority: env override → per-secret store → defaultStore → legacy provider → "env"), `getProviderForStore` (maps SecretStores config to providers), `buildSecretStatuses` (access-aware status for `wfctl secrets list`).
20+
- **`detect_infra_needs` plugin manifest integration** (`cmd/wfctl/plugin_infra.go`, `mcp/scaffold_tools.go`): `LoadPluginManifests` and `DetectPluginInfraNeeds` scan local plugin directories for `plugin.json` manifests and surface module-type infra requirements. MCP tool gains optional `plugins_dir` parameter.
1421

1522
### Documentation
1623

17-
- `docs/WFCTL.md`: added `dev up/down/logs/status/restart` reference (flags, mode table, exposure methods), `wizard` reference (screen list, navigation keys).
18-
- `CHANGELOG.md`: entry for wfctl dev + wizard.
24+
- `docs/WFCTL.md`: updated `wizard` reference (11 screens, new navigation keys); added `secrets setup` command reference with flag table and examples; updated `secrets list` description to mention multi-store routing.
25+
- `docs/dsl-reference.md` + `cmd/wfctl/dsl-reference-embedded.md`: expanded `infrastructure` fields with per-env resolution strategies, connection config, and extended example; added `secretStores:` section with example and relationships; updated `secrets:` section with `defaultStore`, per-secret `store` field, multi-store example, and `secrets setup` CLI command.
26+
- `docs/plugin-manifest-guide.md`: new guide for plugin authors on declaring infrastructure requirements in `plugin.json` via `moduleInfraRequirements`.
1927

2028

2129

cmd/wfctl/dsl-reference-embedded.md

Lines changed: 128 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -668,7 +668,27 @@ These top-level sections describe infrastructure-as-code resources that the engi
668668
- `capabilities` (array) — resource capability definitions
669669

670670
### `infrastructure` Fields
671-
A map of infrastructure resource definitions. Structure is plugin-defined.
671+
672+
The `infrastructure:` section declares infrastructure resources the application depends on. Each resource can specify per-environment resolution strategies.
673+
674+
- `infrastructure.resources` (array) — list of infrastructure resource declarations. Each entry:
675+
- `name` (string, required) — unique resource name
676+
- `type` (string, required) — resource type (e.g., `postgresql`, `redis`, `nats`, `s3-bucket`)
677+
- `provider` (string) — IaC provider to use for provisioning (e.g., `aws`, `gcp`, `azure`, `digitalocean`)
678+
- `config` (map) — resource-specific configuration
679+
- `environments` (map) — per-environment resolution strategies. Each key is an environment name and the value is an `InfraEnvironmentResolution` object:
680+
- `strategy` (string, required) — how to resolve this resource in this environment:
681+
- `container` — run a container locally (for local/CI environments)
682+
- `provision` — provision via IaC plugin (for staging/production)
683+
- `existing` — connect to an already-running instance
684+
- `dockerImage` (string) — container image to use when `strategy: container`
685+
- `port` (int) — container port when `strategy: container`
686+
- `provider` (string) — override IaC provider for this environment
687+
- `config` (map) — environment-specific resource configuration
688+
- `connection` (object) — connection details when `strategy: existing`:
689+
- `host` (string, required) — hostname or IP
690+
- `port` (int) — port number
691+
- `auth` (string) — authentication reference (e.g., a secret name)
672692

673693
### `sidecars` Fields
674694
- `sidecars` (array) — list of sidecar container definitions
@@ -699,6 +719,43 @@ platform:
699719
ports:
700720
- container_port: 8080
701721
722+
infrastructure:
723+
resources:
724+
- name: db
725+
type: postgresql
726+
provider: aws
727+
config:
728+
instanceClass: db.t3.micro
729+
environments:
730+
local:
731+
strategy: container
732+
dockerImage: postgres:16
733+
port: 5432
734+
staging:
735+
strategy: provision
736+
provider: aws
737+
production:
738+
strategy: provision
739+
provider: aws
740+
config:
741+
instanceClass: db.r6g.large
742+
- name: cache
743+
type: redis
744+
environments:
745+
local:
746+
strategy: container
747+
dockerImage: redis:7
748+
port: 6379
749+
staging:
750+
strategy: existing
751+
connection:
752+
host: redis.internal.staging.example.com
753+
port: 6379
754+
auth: REDIS_PASSWORD
755+
production:
756+
strategy: provision
757+
provider: aws
758+
702759
sidecars:
703760
- name: redis-cache
704761
type: redis
@@ -715,6 +772,8 @@ sidecars:
715772
- `platform.context` module type references the `platform` section org/environment values
716773
- Sidecars are deployed alongside the application container but are not addressable as workflow modules
717774
- `infrastructure` resources are provisioned by IaC plugins before application start
775+
- `infrastructure.resources[*].environments` per-env strategies control `wfctl dev up` container lifecycle
776+
- Plugins declare their infra requirements in `plugin.json` via `moduleInfraRequirements`
718777

719778
---
720779

@@ -877,33 +936,70 @@ environments:
877936

878937
---
879938

939+
<!-- section: secretStores -->
940+
## Secret Stores
941+
942+
The optional `secretStores:` section declares named secret storage backends. This enables routing different secrets to different backends (e.g., application secrets in environment variables, payment keys in AWS Secrets Manager).
943+
944+
### Fields
945+
946+
- `secretStores.<name>` (object) — a named store. Fields:
947+
- `provider` (string, required) — backend provider: `env`, `vault`, `aws-secrets-manager`, `gcp-secret-manager`
948+
- `config` (map) — provider-specific configuration (e.g., Vault address, AWS region)
949+
950+
### Example
951+
952+
```yaml
953+
secretStores:
954+
primary:
955+
provider: env
956+
payment-vault:
957+
provider: aws-secrets-manager
958+
config:
959+
region: us-east-1
960+
```
961+
962+
### Relationship to Other Sections
963+
964+
- `secrets.defaultStore` references a named store from `secretStores`
965+
- `secrets.entries[*].store` routes an individual secret to a specific store
966+
- `environments[*].secretsProvider` overrides the store name for all secrets in that environment
967+
968+
---
969+
880970
<!-- section: secrets -->
881971
## Secrets
882972

883-
The optional `secrets:` section declares the application's secret management configuration: which provider to use, rotation policy, and what secrets the application needs.
973+
The optional `secrets:` section declares the application's secret management configuration: which stores to use, rotation policy, and what secrets the application needs.
884974

885975
### Fields
886976

887-
- `secrets.provider` (string, required) — secrets provider name. Currently supported: `env`
888-
- `secrets.config` (map) — provider-specific configuration
977+
- `secrets.defaultStore` (string) — name of the default store from `secretStores`. When set, all secrets without an explicit `store` field use this store.
978+
- `secrets.provider` (string) — legacy single-provider name (use `defaultStore` + `secretStores` for new configs). Supported: `env`, `vault`, `aws-secrets-manager`, `gcp-secret-manager`
979+
- `secrets.config` (map) — provider-specific configuration (used with legacy `provider` field)
889980
- `secrets.rotation` (object) — default rotation policy:
890981
- `enabled` (bool) — enable automatic rotation
891982
- `interval` (string) — rotation interval (e.g., `30d`, `7d`)
892983
- `strategy` (string) — rotation strategy (`dual-credential`, `graceful`, `immediate`)
893984
- `secrets.entries` (array) — declared secrets the application needs. Each entry:
894985
- `name` (string, required) — secret name (typically an env var name)
895986
- `description` (string) — human-readable description
987+
- `store` (string) — name of a specific store from `secretStores`; overrides `defaultStore` and environment override
896988
- `rotation` (object) — per-secret rotation override (same fields as `secrets.rotation`)
897989

898-
### Example
990+
### Example (multi-store)
899991

900992
```yaml
993+
secretStores:
994+
primary:
995+
provider: env
996+
payment-vault:
997+
provider: aws-secrets-manager
998+
config:
999+
region: us-east-1
1000+
9011001
secrets:
902-
provider: env
903-
rotation:
904-
enabled: true
905-
interval: 30d
906-
strategy: dual-credential
1002+
defaultStore: primary
9071003
entries:
9081004
- name: DATABASE_URL
9091005
description: PostgreSQL connection string
@@ -915,22 +1011,42 @@ secrets:
9151011
strategy: graceful
9161012
- name: STRIPE_SECRET_KEY
9171013
description: Stripe payment API key
1014+
store: payment-vault
1015+
```
1016+
1017+
### Example (single provider, legacy)
1018+
1019+
```yaml
1020+
secrets:
1021+
provider: env
1022+
rotation:
1023+
enabled: true
1024+
interval: 30d
1025+
strategy: dual-credential
1026+
entries:
1027+
- name: DATABASE_URL
1028+
description: PostgreSQL connection string
1029+
- name: JWT_SECRET
1030+
description: JWT signing key
9181031
```
9191032

9201033
### CLI Commands
9211034

9221035
- `wfctl secrets detect --config app.yaml` — scan config for secret-like values
9231036
- `wfctl secrets set DATABASE_URL --value "..."` — set a secret in the provider
9241037
- `wfctl secrets set TLS_CERT --from-file ./certs/server.crt` — set from file
925-
- `wfctl secrets list --config app.yaml` — list declared secrets and status
1038+
- `wfctl secrets list --config app.yaml` — list declared secrets and their store routing
9261039
- `wfctl secrets validate --config app.yaml` — verify all secrets are set
9271040
- `wfctl secrets init --provider env --env staging` — initialize provider config
9281041
- `wfctl secrets rotate DATABASE_URL --env production` — trigger rotation
9291042
- `wfctl secrets sync --from staging --to production` — sync secret structure
1043+
- `wfctl secrets setup --env local` — interactively set all secrets for an environment
1044+
- `wfctl secrets setup --env production --auto-gen-keys` — set secrets, auto-generating key/token values
9301045

9311046
### Relationship to Other Sections
9321047

933-
- `environments[*].secretsProvider` overrides the top-level provider per environment
1048+
- `secretStores` must be declared before referencing store names in `secrets.defaultStore` or `secrets.entries[*].store`
1049+
- `environments[*].secretsProvider` overrides the store for all secrets in that environment
9341050
- `environments[*].secretsPrefix` is prepended to secret names when resolving in that environment
9351051
- `ci.deploy.environments` can reference secrets from the `secrets:` section
9361052

docs/WFCTL.md

Lines changed: 34 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1326,7 +1326,7 @@ wfctl secrets set TLS_CERT --from-file ./certs/server.crt
13261326

13271327
#### `secrets list`
13281328

1329-
List all declared secrets and their set/unset status.
1329+
List all declared secrets, their store routing, and access-aware set/unset status. For multi-store configs, each secret shows which store it resolves to and whether the store is accessible.
13301330

13311331
```bash
13321332
wfctl secrets list --config app.yaml
@@ -1364,6 +1364,26 @@ Copy secret structure from one environment to another (Tier 2 implementation).
13641364
wfctl secrets sync --from staging --to production
13651365
```
13661366

1367+
#### `secrets setup`
1368+
1369+
Interactively set all secrets declared in the config for a given environment. Prompts for each secret's value with hidden terminal input. Secrets in inaccessible stores are skipped. Use `--auto-gen-keys` to automatically generate random values for secrets whose names end in `_KEY`, `_SECRET`, `_TOKEN`, or `_SIGNING`.
1370+
1371+
```
1372+
wfctl secrets setup [options]
1373+
```
1374+
1375+
| Flag | Default | Description |
1376+
|------|---------|-------------|
1377+
| `--env` | `local` | Target environment name |
1378+
| `--config` | `app.yaml` | Workflow config file |
1379+
| `--auto-gen-keys` | `false` | Auto-generate random values for key/token/secret-named entries |
1380+
1381+
```bash
1382+
wfctl secrets setup --env local
1383+
wfctl secrets setup --env production --auto-gen-keys
1384+
wfctl secrets setup --env staging --config config/staging.yaml
1385+
```
1386+
13671387
---
13681388

13691389
### `git connect`
@@ -1787,12 +1807,15 @@ No flags. The wizard runs in the terminal and collects:
17871807
1. **Project info** — name and description
17881808
2. **Services** — single-service or multi-service (comma-separated names)
17891809
3. **Infrastructure** — PostgreSQL, Redis cache, NATS message queue (checkboxes)
1790-
4. **Environments** — local, staging, production (checkboxes)
1791-
5. **Deployment** — provider per environment (Docker Compose, Kubernetes, AWS ECS)
1792-
6. **Secrets** — secrets provider (env vars, Vault, AWS Secrets Manager, GCP Secret Manager)
1793-
7. **CI/CD** — generate CI bootstrap and select platform (GitHub Actions, GitLab CI)
1794-
8. **Review** — preview generated YAML
1795-
9. **Write** — save to `app.yaml`
1810+
4. **Infra resolution** — per-environment strategy for each selected infrastructure resource (container/provision/existing); if "existing", prompts for host:port
1811+
5. **Environments** — local, staging, production (checkboxes)
1812+
6. **Deployment** — provider per environment (Docker Compose, Kubernetes, AWS ECS)
1813+
7. **Secret stores** — define named stores (env, Vault, AWS Secrets Manager, GCP Secret Manager); Space to add, Delete to remove, Enter to continue
1814+
8. **Secret routing** — assign each required secret to a store (← → to change store)
1815+
9. **Secret values** — enter values for required secrets with hidden input; Ctrl+G auto-generates random values for keys/tokens
1816+
10. **CI/CD** — generate CI bootstrap and select platform (GitHub Actions, GitLab CI)
1817+
11. **Review** — preview generated YAML
1818+
12. **Write** — save to `app.yaml`
17961819

17971820
**Navigation:**
17981821

@@ -1801,8 +1824,10 @@ No flags. The wizard runs in the terminal and collects:
18011824
| `Enter` | Advance to next screen / confirm |
18021825
| `Esc` | Go back to previous screen |
18031826
| `Tab` | Move focus between fields |
1804-
| `Space` | Toggle checkbox / select option |
1827+
| `Space` | Toggle checkbox / add store |
18051828
| `` / `` | Move cursor in lists |
1829+
| `` / `` | Change strategy or store selection |
1830+
| `Ctrl+G` | Auto-generate a random secret value (bulk secrets screen) |
18061831
| `Ctrl+C` | Quit without saving |
18071832

18081833
**Example:**
@@ -1811,6 +1836,7 @@ No flags. The wizard runs in the terminal and collects:
18111836
wfctl wizard
18121837
# Follow the interactive prompts to generate app.yaml
18131838
wfctl validate app.yaml
1839+
wfctl secrets setup --env local # set secret values interactively
18141840
wfctl dev up
18151841
```
18161842

0 commit comments

Comments
 (0)