diff --git a/README.md b/README.md index 753da73..f157e3d 100644 --- a/README.md +++ b/README.md @@ -30,18 +30,35 @@ Create a `tasks.yml` file: repository: http://github.com/cakephp/inflector.cakephp.org ``` +JSON5 is also supported: write the same recipe as `tasks.json` and pass `--tasks tasks.json`, or just drop the file in the working directory and docket will pick it up automatically when `tasks.yml` / `tasks.yaml` are absent. See "Task file formats" below for the dispatch rules. + Run it: ```shell -# from the same directory as the tasks.yml +# from the same directory as the tasks.yml (or tasks.json) docket apply ``` -Running `docket` with no subcommand prints the available commands. Use `docket init` to scaffold a starter `tasks.yml`, `docket apply` to execute a task file, `docket fmt` to canonically format a task file, `docket plan` to preview the changes a task file would make without mutating any state, `docket validate` to check a task file's schema and templates without contacting the server, or `docket version` to print the binary's version. +Running `docket` with no subcommand prints the available commands. Use `docket init` to scaffold a starter task file, `docket apply` to execute a task file, `docket fmt` to canonically format a task file, `docket plan` to preview the changes a task file would make without mutating any state, `docket validate` to check a task file's schema and templates without contacting the server, or `docket version` to print the binary's version. All five commands accept either YAML or JSON5 surface syntax. + +### Task file formats + +Docket reads task files in either YAML or JSON5. Format is selected by file extension: + +| Extension | Parser | +|-----------|--------| +| `.yml`, `.yaml` | `gopkg.in/yaml.v3` | +| `.json`, `.json5` | titanous JSON5 (a strict superset of JSON) | + +JSON5 adds three things over plain JSON that are useful in a recipe: `// line` and `/* block */` comments, trailing commas in arrays and objects, and unquoted keys when they are valid identifiers. Existing JSON files parse unchanged. + +When `--tasks` is omitted, docket probes the working directory in this order: `tasks.yml`, `tasks.yaml`, `tasks.json`. The first one that exists wins. If none are present the run errors with the candidate list so the typo is obvious. With `--tasks `, format is detected from the path's extension; unknown extensions default to YAML so a path like `recipe.txt` keeps its pre-#218 behaviour. + +The same JSON5 recipe in YAML and in JSON5 produces an identical play / task structure - sigil `{{ .var }}` templates, expr predicates, every envelope key, and every task type behave the same way. `docket fmt` round-trips comments in both formats. ### Multi-play recipes -`tasks.yml` is a YAML list of plays. Each play has its own `name`, optional `tags`, optional `when:`, optional `inputs:`, and a `tasks:` list. The executor walks every play in source order, so a single recipe can describe multiple coordinated apps or services in one file. +A docket recipe is a list of plays. Each play has its own `name`, optional `tags`, optional `when:`, optional `inputs:`, and a `tasks:` list. The executor walks every play in source order, so a single recipe can describe multiple coordinated apps or services in one file. The examples below use YAML; JSON5 recipes have the same shape (a top-level array of objects) - see "Task file formats" above. ```yaml --- @@ -345,13 +362,18 @@ Execution rules: ### Scaffolding with `init` -`docket init` writes a starter `tasks.yml` from an embedded template. It is offline only: no Dokku server contact, no `git` subprocess. The default scaffold ships four tasks (`dokku_app`, `dokku_config`, `dokku_domains`, `dokku_git_sync`) wrapped in a single play with `app` and `repo` inputs, and round-trips cleanly through `docket validate`. +`docket init` writes a starter task file from an embedded template. It is offline only: no Dokku server contact, no `git` subprocess. The default scaffold ships four tasks (`dokku_app`, `dokku_config`, `dokku_domains`, `dokku_git_sync`) wrapped in a single play with `app` and `repo` inputs, and round-trips cleanly through `docket validate`. + +The output format is inferred from the `--output` extension: `tasks.json` / `tasks.json5` writes a JSON5 scaffold (with `// ...` comments demonstrating the comment syntax), anything else writes the YAML scaffold. Stdout (`--output -`) defaults to YAML. ```shell # Use cwd basename as the app and remote.origin.url from ./.git/config as the repo docket init -# Stream the rendered YAML to stdout for piping +# Same scaffold, JSON5 surface syntax +docket init --output tasks.json + +# Stream the rendered scaffold to stdout for piping docket init --output - ``` @@ -360,7 +382,7 @@ The flags are: | Flag | Effect | |------|--------| | (default) | Write `./tasks.yml`; refuse if the file exists | -| `--output ` | Write to a specific path; `-` writes to stdout | +| `--output ` | Write to a specific path; `-` writes to stdout. Format is inferred from the extension (`.json` / `.json5` -> JSON5, otherwise YAML). | | `--force` | Overwrite an existing file | | `--name ` | Override the play and `app` input default (defaults to the cwd basename) | | `--repo ` | Override the `repo` input default (defaults to `remote.origin.url` in `./.git/config`, if present) | @@ -368,16 +390,25 @@ The flags are: ### Formatting recipes with `fmt` -`docket fmt` is a canonical formatter for `tasks.yml`, in the spirit of `gofmt`. It parses with `gopkg.in/yaml.v3`'s `Node` API so head, line, and foot comments round-trip; reorders task envelope and play keys into a stable order; normalises indentation to a 2-space step; and inserts blank lines between top-level plays and between top-level task entries. The default rewrites `./tasks.yml` in place; `--check` and `--diff` are read-only modes. The CLI flags compose, modeled after `black` / `ruff format`. +`docket fmt` is a canonical formatter for task files, in the spirit of `gofmt`. It works for both YAML and JSON5: format is detected per file from the path's extension (`.yml` / `.yaml` use the `gopkg.in/yaml.v3` Node API, `.json` / `.json5` use docket's in-tree JSON5 formatter). Both formatters share the same canonical key order so a YAML recipe and its JSON5 twin lay out identically. + +For YAML, head / line / foot comments survive via yaml.v3's Node API. For JSON5, comments survive via a comment-aware in-tree AST + emitter (line `// ...` comments above a member, beside a member on the same line, or as foot comments inside a container before its closing brace; block `/* ... */` comments are preserved at the same anchors). Both surfaces reorder task envelope and play keys into a stable order, normalise indentation to a 2-space step, and insert blank lines between top-level plays and between top-level task entries. The default rewrites the named file in place; `--check` and `--diff` are read-only modes. The CLI flags compose, modeled after `black` / `ruff format`. ```shell -# Rewrite ./tasks.yml in place. +# Rewrite ./tasks.yml in place. With no positional argument, fmt +# probes tasks.yml -> tasks.yaml -> tasks.json (same default-lookup +# rule as apply / plan / validate). docket fmt +# Format a JSON5 recipe in place +docket fmt tasks.json + # CI gate: print the diff and exit 1 if anything is not canonical. docket fmt --check --diff -# Read from stdin, write canonical to stdout. +# Read from stdin, write canonical to stdout. Stdin format is sniffed +# from the first non-trivia byte: a leading [ or { signals JSON5, +# anything else parses as YAML. cat tasks.yml | docket fmt - ``` @@ -423,7 +454,7 @@ The flags are: | Flag | Effect | |------|--------| -| `--tasks ` | Use a specific task file (default `./tasks.yml`) | +| `--tasks ` | Use a specific task file (YAML or JSON5). When omitted, docket probes `tasks.yml` -> `tasks.yaml` -> `tasks.json`. | | `--verbose` | After each task line, echo every resolved Dokku command the task ran on a `→`-prefixed continuation line, in invocation order. Tasks that loop over inputs (e.g. `dokku_buildpacks` adding several URLs) emit one continuation per call. Commands are masked against the global sensitive value set. Ignored when `--json` is set; the JSON output already includes the resolved commands. | | `--json` | Suppress the human formatter and emit one JSON-lines event per `play_start`, `task`, or `summary` to stdout. Sensitive values mask to `***`. See "JSON output" below for the schema. | | `--vars-file ` | Load input values from a YAML or JSON file. Repeatable; later files override earlier files for the same key. CLI `--name=value` flags always win. See "Layered input variables with `--vars-file`" below. | @@ -543,7 +574,7 @@ The flags are: | Flag | Effect | |------|--------| -| `--tasks ` | Use a specific task file (default `./tasks.yml`) | +| `--tasks ` | Use a specific task file (YAML or JSON5). When omitted, docket probes `tasks.yml` -> `tasks.yaml` -> `tasks.json`. | | `--json` | Suppress the human formatter and emit one JSON-lines event per `play_start`, `task`, or `summary` to stdout. Sensitive values mask to `***`. See "JSON output" below for the schema. | | `--detailed-exitcode` | Exit `0` when no drift is detected, `2` when at least one task reports drift, `1` on read or probe error. Errors win over drift. Without this flag, plan exits `0` regardless of drift. Mirrors the `git diff --exit-code` / `terraform plan -detailed-exitcode` convention. | | `--vars-file ` | Load input values from a YAML or JSON file. Repeatable; later files override earlier files for the same key. CLI `--name=value` flags always win. See "Layered input variables with `--vars-file`" below. | @@ -599,9 +630,12 @@ Exit codes are `0` when no problems are found and `1` otherwise. Five flags are A task file can also be specified via flag, and may be a file retrieved via http: ```shell -# alternate path +# alternate path (YAML) docket apply --tasks path/to/task.yml +# JSON5 task file +docket apply --tasks path/to/tasks.json + # html file docket apply --tasks http://dokku.com/docket/example.yml ``` diff --git a/commands/apply.go b/commands/apply.go index c476775..59f75d4 100644 --- a/commands/apply.go +++ b/commands/apply.go @@ -19,6 +19,7 @@ type ApplyCommand struct { command.Meta tasksFile string + tasksFormat string verbose bool json bool host string @@ -50,7 +51,8 @@ func (c *ApplyCommand) Examples() map[string]string { appName := os.Getenv("CLI_APP_NAME") return map[string]string{ "Apply tasks from the default tasks.yml": fmt.Sprintf("%s %s", appName, c.Name()), - "Apply tasks from a specific file": fmt.Sprintf("%s %s --tasks path/to/task.yml", appName, c.Name()), + "Apply tasks from a specific YAML file": fmt.Sprintf("%s %s --tasks path/to/task.yml", appName, c.Name()), + "Apply tasks from a JSON5 file": fmt.Sprintf("%s %s --tasks path/to/tasks.json", appName, c.Name()), "Apply tasks from a remote URL": fmt.Sprintf("%s %s --tasks http://dokku.com/docket/example.yml", appName, c.Name()), "Override a task input": fmt.Sprintf("%s %s --name lollipop", appName, c.Name()), } @@ -70,7 +72,7 @@ func (c *ApplyCommand) ParsedArguments(args []string) (map[string]command.Argume func (c *ApplyCommand) FlagSet() *flag.FlagSet { f := c.Meta.FlagSet(c.Name(), command.FlagSetClient) - f.StringVar(&c.tasksFile, "tasks", "tasks.yml", "a yaml file containing a task list") + f.StringVar(&c.tasksFile, "tasks", "", "task file (YAML or JSON5) containing a task list. When omitted, docket probes tasks.yml -> tasks.yaml -> tasks.json in the current directory.") f.BoolVar(&c.verbose, "verbose", false, "echo the resolved dokku command for each task as a continuation line. Values from inputs declared `sensitive: true` and from task struct fields tagged `sensitive:\"true\"` are masked as `***`. Ignored when --json is set; the JSON output already includes the resolved commands.") f.BoolVar(&c.json, "json", false, "emit one JSON-lines event per play/task/summary instead of human-readable output. Schema is keyed by `version: 1`; sensitive values mask to `***`.") f.StringVar(&c.host, "host", "", "remote dokku host as [user@]host[:port]; equivalent to DOKKU_HOST. Routes every dokku invocation through ssh.") @@ -84,13 +86,13 @@ func (c *ApplyCommand) FlagSet() *flag.FlagSet { f.BoolVar(&c.listTasks, "list-tasks", false, "print the resolved task plan and exit without running. Honors --play / --tags / --skip-tags and shows expanded loop iterations and [skipped] markers for when:-skipped tasks.") f.StringVar(&c.startAtTask, "start-at-task", "", "skip every task before the matched name; the matched task and successors run normally. Filter order: --start-at-task -> --tags/--skip-tags -> per-task when: at execution. The name search walks every play in source order, narrowed by --play.") - taskFile := getTaskYamlFilename(os.Args) + taskFile, format := resolveTaskFileFromArgs(os.Args) data, err := os.ReadFile(taskFile) if err != nil { return f } - arguments, err := registerInputFlags(f, data) + arguments, err := registerInputFlags(f, data, format) if err != nil { return f } @@ -103,7 +105,7 @@ func (c *ApplyCommand) AutocompleteFlags() complete.Flags { return command.MergeAutocompleteFlags( c.Meta.AutocompleteFlags(command.FlagSetClient), complete.Flags{ - "--tasks": complete.PredictFiles("*.yml"), + "--tasks": complete.PredictFiles(taskFileAutocompleteGlob), "--verbose": complete.PredictNothing, "--json": complete.PredictNothing, "--host": complete.PredictAnything, @@ -137,6 +139,14 @@ func (c *ApplyCommand) Run(args []string) int { resolvedHost := resolveSshFlags(c.host, c.sudo, c.acceptNewHostKeys) + resolvedPath, resolvedFormat, err := resolveTaskFilePath(c.tasksFile) + if err != nil { + c.Ui.Error(fmt.Sprintf("read error: %v", err)) + return 1 + } + c.tasksFile = resolvedPath + c.tasksFormat = resolvedFormat + data, err := os.ReadFile(c.tasksFile) if err != nil { c.Ui.Error(fmt.Sprintf("read error: %v", err)) @@ -160,7 +170,7 @@ func (c *ApplyCommand) Run(args []string) int { userSet := userSetKeys(flags, varsFileKeys, c.arguments) - plays, err := tasks.GetPlays(data, context, userSet) + plays, err := tasks.GetPlaysWithFormat(data, c.tasksFormat, context, userSet) if err != nil { c.Ui.Error(fmt.Sprintf("task error: %v", err)) return 1 diff --git a/commands/apply_args.go b/commands/apply_args.go index c518613..2437467 100644 --- a/commands/apply_args.go +++ b/commands/apply_args.go @@ -117,20 +117,38 @@ func isFalseString(s string) bool { } func getTaskYamlFilename(s []string) string { + path, _ := resolveTaskFileFromArgs(s) + return path +} + +// resolveTaskFileFromArgs walks os.Args-style argv, finds a --tasks +// value, and returns it together with its detected format. When --tasks +// is not present the function probes defaultTaskFileCandidates in order +// and returns the first one that exists; if none exist the legacy +// default ("tasks.yml") is returned so the downstream os.ReadFile error +// path still fires with the familiar message. Format is keyed by file +// extension; see detectTaskFileFormat. +func resolveTaskFileFromArgs(s []string) (string, string) { for i, arg := range s { if arg == "--tasks" { if len(s) > i+1 { - return s[i+1] + path := s[i+1] + return path, detectTaskFileFormat(path) } } if taskFile, found := strings.CutPrefix(arg, "--tasks="); found { - return taskFile + return taskFile, detectTaskFileFormat(taskFile) } } - return "tasks.yml" + for _, candidate := range defaultTaskFileCandidates { + if _, err := os.Stat(candidate); err == nil { + return candidate, detectTaskFileFormat(candidate) + } + } + return defaultTaskFileCandidates[0], detectTaskFileFormat(defaultTaskFileCandidates[0]) } -func getInputVariables(data []byte) (map[string]*tasks.Input, error) { +func getInputVariables(data []byte, format string) (map[string]*tasks.Input, error) { vars := make(map[string]interface{}) render, err := sigil.Execute(data, vars, "tasks") if err != nil { @@ -142,15 +160,16 @@ func getInputVariables(data []byte) (map[string]*tasks.Input, error) { return map[string]*tasks.Input{}, fmt.Errorf("render error: %v", err.Error()) } - return parseInputYaml(out) + return parseInputDocument(out, format) } // registerInputFlags reads the task file inputs and registers a flag for each // dynamic input on the given FlagSet. It returns the Argument map keyed by -// input name so the caller can collect values after flags.Parse. -func registerInputFlags(f *flag.FlagSet, data []byte) (map[string]*Argument, error) { +// input name so the caller can collect values after flags.Parse. format is +// "yaml" or "json5"; the empty string is treated as YAML. +func registerInputFlags(f *flag.FlagSet, data []byte, format string) (map[string]*Argument, error) { arguments := make(map[string]*Argument) - inputs, err := getInputVariables(data) + inputs, err := getInputVariables(data, format) if err != nil { return arguments, err } @@ -196,10 +215,13 @@ func registerInputFlags(f *flag.FlagSet, data []byte) (map[string]*Argument, err return arguments, nil } -func parseInputYaml(data []byte) (map[string]*tasks.Input, error) { +// parseInputDocument decodes data as a Recipe in the given on-disk +// format and returns the merged input map keyed by input name. format +// is "yaml" or "json5"; empty / unknown values default to YAML. +func parseInputDocument(data []byte, format string) (map[string]*tasks.Input, error) { inputs := make(map[string]*tasks.Input) - t := tasks.Recipe{} - if err := yaml.Unmarshal(data, &t); err != nil { + t, err := tasks.UnmarshalRecipe(data, format) + if err != nil { return inputs, err } @@ -217,6 +239,13 @@ func parseInputYaml(data []byte) (map[string]*tasks.Input, error) { return inputs, nil } +// parseInputYaml is the YAML-only back-compat wrapper kept so existing +// callers (and the unit tests under apply_args_test.go) do not need to +// learn the format dispatch. +func parseInputYaml(data []byte) (map[string]*tasks.Input, error) { + return parseInputDocument(data, tasks.FormatYAML) +} + // SetFromVarsFile coerces value to the Argument's declared Type and writes // it through the underlying typed pointer that registerInputFlags allocated. // The resulting state is indistinguishable from a CLI flag at the same value diff --git a/commands/apply_args_test.go b/commands/apply_args_test.go index 742f23d..9cc1ee5 100644 --- a/commands/apply_args_test.go +++ b/commands/apply_args_test.go @@ -173,6 +173,51 @@ func TestArgumentGetValue(t *testing.T) { }) } +func TestParseInputDocumentJSON5(t *testing.T) { + data := []byte(`[ + { + inputs: [ + // primary input + { name: "app", default: "myapp", description: "Application name", required: true, type: "string" }, + { name: "port", default: "8080", description: "Port number", type: "int" }, + ], + tasks: [], + }, +] +`) + inputs, err := parseInputDocument(data, tasks.FormatNameJSON5) + if err != nil { + t.Fatalf("parseInputDocument failed: %v", err) + } + if len(inputs) != 2 { + t.Fatalf("expected 2 inputs, got %d", len(inputs)) + } + if app, ok := inputs["app"]; !ok || app.Default != "myapp" || !app.Required || app.Type != "string" { + t.Errorf("'app' input mismatched: %+v ok=%v", app, ok) + } + if port, ok := inputs["port"]; !ok || port.Default != "8080" || port.Type != "int" { + t.Errorf("'port' input mismatched: %+v ok=%v", port, ok) + } +} + +func TestGetInputVariablesJSON5(t *testing.T) { + data := []byte(`[ + { + inputs: [ + { name: "app", default: "myapp" }, + ], + tasks: [], + }, +]`) + inputs, err := getInputVariables(data, tasks.FormatNameJSON5) + if err != nil { + t.Fatalf("getInputVariables: %v", err) + } + if app, ok := inputs["app"]; !ok || app.Default != "myapp" { + t.Errorf("'app' input mismatched: %+v ok=%v", app, ok) + } +} + func TestParseInputYamlValidInputs(t *testing.T) { data := []byte(`--- - inputs: @@ -318,7 +363,7 @@ func TestGetInputVariablesValid(t *testing.T) { required: true tasks: [] `) - inputs, err := getInputVariables(data) + inputs, err := getInputVariables(data, tasks.FormatYAML) if err != nil { t.Fatalf("getInputVariables failed: %v", err) } @@ -341,7 +386,7 @@ func TestGetInputVariablesTemplateError(t *testing.T) { - name: {{ .broken tasks: [] `) - _, err := getInputVariables(data) + _, err := getInputVariables(data, tasks.FormatYAML) if err == nil { t.Fatal("expected error for bad template syntax") } diff --git a/commands/fmt.go b/commands/fmt.go index 45c58c8..989b218 100644 --- a/commands/fmt.go +++ b/commands/fmt.go @@ -38,7 +38,7 @@ func (c *FmtCommand) Name() string { } func (c *FmtCommand) Synopsis() string { - return "Formats a tasks.yml file canonically" + return "Formats a tasks file canonically (YAML or JSON5)" } func (c *FmtCommand) Help() string { @@ -48,13 +48,15 @@ func (c *FmtCommand) Help() string { func (c *FmtCommand) Examples() map[string]string { appName := os.Getenv("CLI_APP_NAME") return map[string]string{ - "Format ./tasks.yml in place": fmt.Sprintf("%s %s", appName, c.Name()), - "Check whether files are canonical": fmt.Sprintf("%s %s --check", appName, c.Name()), - "Print the diff without writing": fmt.Sprintf("%s %s --diff", appName, c.Name()), - "CI gate: print diff and fail on bad": fmt.Sprintf("%s %s --check --diff", appName, c.Name()), - "Read from stdin, write to stdout": fmt.Sprintf("cat tasks.yml | %s %s -", appName, c.Name()), - "Format every yaml under recipes/": fmt.Sprintf("%s %s 'recipes/*.yml'", appName, c.Name()), - "Force colorized diff in a pipe": fmt.Sprintf("%s %s --diff --color always", appName, c.Name()), + "Format ./tasks.yml in place": fmt.Sprintf("%s %s", appName, c.Name()), + "Format a JSON5 task file in place": fmt.Sprintf("%s %s tasks.json", appName, c.Name()), + "Check whether files are canonical": fmt.Sprintf("%s %s --check", appName, c.Name()), + "Print the diff without writing": fmt.Sprintf("%s %s --diff", appName, c.Name()), + "CI gate: print diff and fail on bad": fmt.Sprintf("%s %s --check --diff", appName, c.Name()), + "Read from stdin, write to stdout": fmt.Sprintf("cat tasks.yml | %s %s -", appName, c.Name()), + "Format every yaml under recipes/": fmt.Sprintf("%s %s 'recipes/*.yml'", appName, c.Name()), + "Format every JSON5 file under recipes/": fmt.Sprintf("%s %s 'recipes/*.json5'", appName, c.Name()), + "Force colorized diff in a pipe": fmt.Sprintf("%s %s --diff --color always", appName, c.Name()), } } @@ -63,7 +65,7 @@ func (c *FmtCommand) Arguments() []command.Argument { } func (c *FmtCommand) AutocompleteArgs() complete.Predictor { - return complete.PredictFiles("*.yml") + return complete.PredictFiles(taskFileAutocompleteGlob) } func (c *FmtCommand) ParsedArguments(args []string) (map[string]command.Argument, error) { @@ -132,13 +134,18 @@ func (c *FmtCommand) Run(args []string) int { // runStdin reads stdin, formats it, and writes the result to stdout in // the default mode. With --diff the diff goes to stdout; with --check // the exit code reflects whether the input was canonical. +// +// Stdin has no filename to drive format detection, so it sniffs the +// first non-trivia byte: a leading [ or { signals JSON5; anything else +// (including the typical `---` document marker or a leading comment- +// only YAML file) goes through the YAML formatter. func (c *FmtCommand) runStdin() int { src, err := io.ReadAll(os.Stdin) if err != nil { c.Ui.Error(fmt.Sprintf("read stdin: %v", err)) return 1 } - formatted, err := tasks.Format(src) + formatted, err := formatTaskFileBytes(src, sniffStdinFormat(src)) if err != nil { c.Ui.Error(fmt.Sprintf("format error: %v", err)) return 1 @@ -180,7 +187,7 @@ func (c *FmtCommand) formatPath(path string) int { return 1 } - formatted, err := tasks.Format(src) + formatted, err := formatTaskFileBytes(src, detectTaskFileFormat(path)) if err != nil { c.Ui.Error(fmt.Sprintf("%s: %v", path, err)) return 1 @@ -219,13 +226,54 @@ func (c *FmtCommand) formatPath(path string) int { return 0 } +// formatTaskFileBytes dispatches to the YAML or JSON5 formatter based +// on format. Centralised so the in-place and stdin paths stay byte- +// identical for the same logical operation. +func formatTaskFileBytes(src []byte, format string) ([]byte, error) { + if format == taskFileFormatJSON5 { + return tasks.FormatJSON5(src) + } + return tasks.Format(src) +} + +// sniffStdinFormat picks a format for stdin input. JSON5 input always +// starts (after optional whitespace and comments) with `[` or `{`; +// YAML recipes start with `-`, `---`, or a key, none of which collide. +// On ambiguity the function defaults to YAML so existing pipelines +// keep their pre-#218 behaviour. +func sniffStdinFormat(src []byte) string { + for i := 0; i < len(src); i++ { + c := src[i] + if c == ' ' || c == '\t' || c == '\r' || c == '\n' { + continue + } + if c == '/' && i+1 < len(src) && (src[i+1] == '/' || src[i+1] == '*') { + // Skip a leading line/block comment - JSON5 idiom. + return taskFileFormatJSON5 + } + if c == '[' || c == '{' { + return taskFileFormatJSON5 + } + return taskFileFormatYAML + } + return taskFileFormatYAML +} + // expandPaths resolves the positional arguments to a sorted, deduped -// list of file paths. An empty argument list expands to "tasks.yml". +// list of file paths. An empty argument list expands to the first +// existing default candidate (tasks.yml -> tasks.yaml -> tasks.json), +// falling back to "tasks.yml" so the downstream read step produces the +// familiar "no such file" error message when nothing is present. // Each argument is passed through filepath.Glob; literal paths that // do not match the glob syntax flow through unchanged. func expandPaths(args []string) ([]string, error) { if len(args) == 0 { - return []string{"tasks.yml"}, nil + for _, candidate := range defaultTaskFileCandidates { + if _, err := os.Stat(candidate); err == nil { + return []string{candidate}, nil + } + } + return []string{defaultTaskFileCandidates[0]}, nil } seen := map[string]bool{} diff --git a/commands/fmt_test.go b/commands/fmt_test.go index 887d82b..10c953d 100644 --- a/commands/fmt_test.go +++ b/commands/fmt_test.go @@ -61,6 +61,68 @@ func TestFmtCommandHelpDoesNotPanic(t *testing.T) { _ = c.FlagSet() } +// JSON5 fixtures parallel to the YAML ones. Used to assert fmt picks +// the right formatter based on extension and that the canonical JSON5 +// shape matches what FormatJSON5 produces directly. +const messyTasksJSON5 = `[ + // a comment + { + tasks: [ + { + dokku_app: { app: "web" }, + name: "configure web", + }, + ], + }, +] +` + +const canonicalTasksJSON5 = `[ + // a comment + { + tasks: [ + { + name: "configure web", + dokku_app: { + app: "web", + }, + }, + ], + }, +] +` + +func TestFmtRewritesJSON5InPlace(t *testing.T) { + dir := t.TempDir() + path := filepath.Join(dir, "tasks.json") + if err := os.WriteFile(path, []byte(messyTasksJSON5), 0o644); err != nil { + t.Fatalf("seed write: %v", err) + } + c := newTestFmtCommand() + if exit := c.Run([]string{path}); exit != 0 { + t.Errorf("exit = %d, want 0", exit) + } + got, err := os.ReadFile(path) + if err != nil { + t.Fatalf("read: %v", err) + } + if string(got) != canonicalTasksJSON5 { + t.Errorf("file not canonicalised:\nwant:\n%s\ngot:\n%s", canonicalTasksJSON5, got) + } +} + +func TestFmtJSON5IdempotentOnCanonical(t *testing.T) { + dir := t.TempDir() + path := filepath.Join(dir, "tasks.json") + if err := os.WriteFile(path, []byte(canonicalTasksJSON5), 0o644); err != nil { + t.Fatalf("seed write: %v", err) + } + c := newTestFmtCommand() + if exit := c.Run([]string{"--check", path}); exit != 0 { + t.Errorf("--check on canonical JSON5 exit = %d, want 0", exit) + } +} + func TestFmtRewritesNonCanonicalInPlace(t *testing.T) { dir := t.TempDir() path := filepath.Join(dir, "tasks.yml") diff --git a/commands/init.go b/commands/init.go index 5e7e5f6..bc22582 100644 --- a/commands/init.go +++ b/commands/init.go @@ -17,7 +17,6 @@ import ( "github.com/josegonzalez/cli-skeleton/command" "github.com/posener/complete" flag "github.com/spf13/pflag" - yaml "gopkg.in/yaml.v3" ) // InitCommand scaffolds a starter tasks.yml from an embedded template. @@ -50,13 +49,14 @@ func (c *InitCommand) Help() string { func (c *InitCommand) Examples() map[string]string { appName := os.Getenv("CLI_APP_NAME") return map[string]string{ - "Scaffold tasks.yml using cwd defaults": fmt.Sprintf("%s %s", appName, c.Name()), - "Write a minimal one-task scaffold": fmt.Sprintf("%s %s --minimal", appName, c.Name()), - "Override the play and app name": fmt.Sprintf("%s %s --name web", appName, c.Name()), - "Override the git repository URL": fmt.Sprintf("%s %s --repo git@example.com:owner/repo.git", appName, c.Name()), - "Write to a specific path": fmt.Sprintf("%s %s --output path/to/tasks.yml", appName, c.Name()), - "Stream the rendered YAML to stdout": fmt.Sprintf("%s %s --output -", appName, c.Name()), - "Overwrite an existing file": fmt.Sprintf("%s %s --force", appName, c.Name()), + "Scaffold tasks.yml using cwd defaults": fmt.Sprintf("%s %s", appName, c.Name()), + "Scaffold a JSON5 tasks.json instead": fmt.Sprintf("%s %s --output tasks.json", appName, c.Name()), + "Write a minimal one-task scaffold": fmt.Sprintf("%s %s --minimal", appName, c.Name()), + "Override the play and app name": fmt.Sprintf("%s %s --name web", appName, c.Name()), + "Override the git repository URL": fmt.Sprintf("%s %s --repo git@example.com:owner/repo.git", appName, c.Name()), + "Write to a specific path": fmt.Sprintf("%s %s --output path/to/tasks.yml", appName, c.Name()), + "Stream the rendered scaffold to stdout": fmt.Sprintf("%s %s --output -", appName, c.Name()), + "Overwrite an existing file": fmt.Sprintf("%s %s --force", appName, c.Name()), } } @@ -86,7 +86,7 @@ func (c *InitCommand) AutocompleteFlags() complete.Flags { return command.MergeAutocompleteFlags( c.Meta.AutocompleteFlags(command.FlagSetClient), complete.Flags{ - "--output": complete.PredictFiles("*.yml"), + "--output": complete.PredictFiles(taskFileAutocompleteGlob), "--force": complete.PredictNothing, "--minimal": complete.PredictNothing, "--name": complete.PredictNothing, @@ -123,10 +123,19 @@ func (c *InitCommand) Run(args []string) int { } } + // Format is inferred from the --output extension: tasks.json / + // tasks.json5 -> JSON5, anything else -> YAML. Stdout (--output -) + // has no extension to inspect, so it falls through to YAML. + format := tasks.FormatYAML + if !toStdout { + format = detectTaskFileFormat(c.output) + } + rendered, err := renderInit(initOptions{ Name: c.name, Repo: c.repo, Minimal: c.minimal, + Format: format, }) if err != nil { c.Ui.Error(err.Error()) @@ -146,7 +155,7 @@ func (c *InitCommand) Run(args []string) int { return 1 } - taskCount, playCount, err := countTasks(rendered) + taskCount, playCount, err := countTasks(rendered, format) if err != nil { c.Ui.Error(fmt.Sprintf("internal error: rendered scaffold did not parse: %v", err)) return 1 @@ -166,11 +175,18 @@ type initOptions struct { Name string Repo string Minimal bool + // Format selects the on-disk shape of the rendered scaffold. Valid + // values are tasks.FormatYAML (default) and tasks.FormatNameJSON5; the + // empty string is treated as YAML so existing callers (and tests + // that drive renderInit directly) keep their behaviour. + Format string } // renderInit reads the right embedded template, parses it with custom // delimiters so sigil syntax in the body survives untouched, and returns -// the rendered bytes (including the leading `---\n` document marker). +// the rendered bytes. YAML scaffolds are prefixed with the `---\n` +// document marker; JSON5 scaffolds are emitted as a top-level array +// (the docket recipe shape) with no marker. // // Exposed at package scope so unit tests can drive it directly without // going through the cli-skeleton UI plumbing. @@ -180,10 +196,7 @@ func renderInit(opts initOptions) ([]byte, error) { name = "app" } - templateName := "default.yml.tmpl" - if opts.Minimal { - templateName = "minimal.yml.tmpl" - } + templateName := selectInitTemplate(opts.Format, opts.Minimal) raw, err := templates.FS.ReadFile(templateName) if err != nil { @@ -204,11 +217,26 @@ func renderInit(opts initOptions) ([]byte, error) { } var out bytes.Buffer - out.WriteString("---\n") + if !tasks.IsJSON5Format(opts.Format) { + out.WriteString("---\n") + } out.Write(body.Bytes()) return out.Bytes(), nil } +// selectInitTemplate returns the embedded template name for (format, +// minimal). YAML / unknown format -> .yml.tmpl, JSON5 -> .json5.tmpl. +func selectInitTemplate(format string, minimal bool) string { + suffix := "yml" + if tasks.IsJSON5Format(format) { + suffix = "json5" + } + if minimal { + return "minimal." + suffix + ".tmpl" + } + return "default." + suffix + ".tmpl" +} + // defaultName returns the basename of the working directory, or "app" if // the cwd cannot be derived to a usable name. func defaultName() string { @@ -255,12 +283,13 @@ func defaultRepo() string { return "" } -// countTasks parses rendered YAML and returns the total number of tasks -// across all plays plus the play count. Used for the "==> Created -// tasks.yml (N tasks, M plays)" summary line. -func countTasks(data []byte) (int, int, error) { - var recipe tasks.Recipe - if err := yaml.Unmarshal(data, &recipe); err != nil { +// countTasks parses rendered scaffold bytes and returns the total +// number of tasks across all plays plus the play count. Used for the +// "==> Created tasks.yml (N tasks, M plays)" summary line. format +// selects the parser; the empty string defaults to YAML. +func countTasks(data []byte, format string) (int, int, error) { + recipe, err := tasks.UnmarshalRecipe(data, format) + if err != nil { return 0, 0, err } total := 0 diff --git a/commands/init_test.go b/commands/init_test.go index 51bb773..82d90ce 100644 --- a/commands/init_test.go +++ b/commands/init_test.go @@ -266,6 +266,82 @@ func TestInitMinimalPassesValidate(t *testing.T) { } } +func TestInitDefaultJSON5PassesValidate(t *testing.T) { + out, err := renderInit(initOptions{Name: "demo", Format: tasks.FormatNameJSON5}) + if err != nil { + t.Fatalf("renderInit (json5): %v", err) + } + if problems := tasks.Validate(out, tasks.ValidateOptions{Format: tasks.FormatNameJSON5}); len(problems) != 0 { + t.Fatalf("JSON5 default scaffold should pass validate, got %+v", problems) + } + out2, err := renderInit(initOptions{Name: "demo", Repo: "git@example.com:foo/bar.git", Format: tasks.FormatNameJSON5}) + if err != nil { + t.Fatalf("renderInit (json5 with repo): %v", err) + } + if problems := tasks.Validate(out2, tasks.ValidateOptions{Format: tasks.FormatNameJSON5}); len(problems) != 0 { + t.Fatalf("JSON5 scaffold with --repo should pass validate, got %+v", problems) + } +} + +func TestInitMinimalJSON5PassesValidate(t *testing.T) { + out, err := renderInit(initOptions{Name: "demo", Minimal: true, Format: tasks.FormatNameJSON5}) + if err != nil { + t.Fatalf("renderInit (minimal json5): %v", err) + } + if problems := tasks.Validate(out, tasks.ValidateOptions{Format: tasks.FormatNameJSON5}); len(problems) != 0 { + t.Fatalf("minimal JSON5 scaffold should pass validate, got %+v", problems) + } +} + +func TestInitJSON5HasNoYAMLDocumentMarker(t *testing.T) { + out, err := renderInit(initOptions{Name: "demo", Format: tasks.FormatNameJSON5}) + if err != nil { + t.Fatalf("renderInit: %v", err) + } + if strings.HasPrefix(string(out), "---") { + t.Errorf("JSON5 scaffold should not start with YAML document marker:\n%s", out) + } + if !strings.HasPrefix(strings.TrimSpace(string(out)), "[") { + t.Errorf("JSON5 scaffold should start with [:\n%s", out) + } +} + +func TestInitJSON5RoundTripsThroughGetPlays(t *testing.T) { + out, err := renderInit(initOptions{Name: "api", Format: tasks.FormatNameJSON5}) + if err != nil { + t.Fatalf("renderInit: %v", err) + } + plays, err := tasks.GetPlaysWithFormat(out, tasks.FormatNameJSON5, map[string]interface{}{ + "app": "api", + "repo": "https://example.com/repo.git", + }, nil) + if err != nil { + t.Fatalf("GetPlaysWithFormat: %v", err) + } + if len(plays) != 1 { + t.Fatalf("plays = %d, want 1", len(plays)) + } + if got := len(plays[0].Tasks.Keys()); got != 4 { + t.Errorf("default JSON5 scaffold = %d tasks, want 4", got) + } +} + +func TestSelectInitTemplate(t *testing.T) { + cases := map[[2]string]string{ + {tasks.FormatYAML, "false"}: "default.yml.tmpl", + {tasks.FormatYAML, "true"}: "minimal.yml.tmpl", + {tasks.FormatNameJSON5, "false"}: "default.json5.tmpl", + {tasks.FormatNameJSON5, "true"}: "minimal.json5.tmpl", + {"", "false"}: "default.yml.tmpl", + } + for key, want := range cases { + minimal := key[1] == "true" + if got := selectInitTemplate(key[0], minimal); got != want { + t.Errorf("selectInitTemplate(%q, %v) = %q, want %q", key[0], minimal, got, want) + } + } +} + func TestInitDefaultParsesAsRecipe(t *testing.T) { out, err := renderInit(initOptions{Name: "api"}) if err != nil { diff --git a/commands/plan.go b/commands/plan.go index c4a39d9..83cd763 100644 --- a/commands/plan.go +++ b/commands/plan.go @@ -20,6 +20,7 @@ type PlanCommand struct { command.Meta tasksFile string + tasksFormat string json bool detailedExitCode bool host string @@ -49,7 +50,8 @@ func (c *PlanCommand) Examples() map[string]string { appName := os.Getenv("CLI_APP_NAME") return map[string]string{ "Plan tasks from the default tasks.yml": fmt.Sprintf("%s %s", appName, c.Name()), - "Plan tasks from a specific file": fmt.Sprintf("%s %s --tasks path/to/task.yml", appName, c.Name()), + "Plan tasks from a specific YAML file": fmt.Sprintf("%s %s --tasks path/to/task.yml", appName, c.Name()), + "Plan tasks from a JSON5 file": fmt.Sprintf("%s %s --tasks path/to/tasks.json", appName, c.Name()), "Plan tasks from a remote URL": fmt.Sprintf("%s %s --tasks http://dokku.com/docket/example.yml", appName, c.Name()), "Override a task input": fmt.Sprintf("%s %s --name lollipop", appName, c.Name()), } @@ -69,7 +71,7 @@ func (c *PlanCommand) ParsedArguments(args []string) (map[string]command.Argumen func (c *PlanCommand) FlagSet() *flag.FlagSet { f := c.Meta.FlagSet(c.Name(), command.FlagSetClient) - f.StringVar(&c.tasksFile, "tasks", "tasks.yml", "a yaml file containing a task list") + f.StringVar(&c.tasksFile, "tasks", "", "task file (YAML or JSON5) containing a task list. When omitted, docket probes tasks.yml -> tasks.yaml -> tasks.json in the current directory.") f.BoolVar(&c.json, "json", false, "emit one JSON-lines event per play/task/summary instead of human-readable output. Schema is keyed by `version: 1`; sensitive values mask to `***`.") f.BoolVar(&c.detailedExitCode, "detailed-exitcode", false, "exit 0 when no drift is detected, 2 when drift is detected, 1 on error. Without this flag plan exits 0 regardless of drift.") f.StringVar(&c.host, "host", "", "remote dokku host as [user@]host[:port]; equivalent to DOKKU_HOST. Routes every dokku invocation through ssh.") @@ -81,13 +83,13 @@ func (c *PlanCommand) FlagSet() *flag.FlagSet { f.StringVar(&c.play, "play", "", "plan only the play with this name (matches the play's `name:` field; auto-named plays use `play #N`)") f.BoolVar(&c.listTasks, "list-tasks", false, "print the resolved task plan and exit without contacting the server. Honors --play / --tags / --skip-tags and shows expanded loop iterations and [skipped] markers for when:-skipped tasks.") - taskFile := getTaskYamlFilename(os.Args) + taskFile, format := resolveTaskFileFromArgs(os.Args) data, err := os.ReadFile(taskFile) if err != nil { return f } - arguments, err := registerInputFlags(f, data) + arguments, err := registerInputFlags(f, data, format) if err != nil { return f } @@ -100,7 +102,7 @@ func (c *PlanCommand) AutocompleteFlags() complete.Flags { return command.MergeAutocompleteFlags( c.Meta.AutocompleteFlags(command.FlagSetClient), complete.Flags{ - "--tasks": complete.PredictFiles("*.yml"), + "--tasks": complete.PredictFiles(taskFileAutocompleteGlob), "--json": complete.PredictNothing, "--detailed-exitcode": complete.PredictNothing, "--host": complete.PredictAnything, @@ -146,6 +148,14 @@ func (c *PlanCommand) Run(args []string) int { resolvedHost := resolveSshFlags(c.host, c.sudo, c.acceptNewHostKeys) + resolvedPath, resolvedFormat, err := resolveTaskFilePath(c.tasksFile) + if err != nil { + c.Ui.Error(fmt.Sprintf("read error: %v", err)) + return 1 + } + c.tasksFile = resolvedPath + c.tasksFormat = resolvedFormat + data, err := os.ReadFile(c.tasksFile) if err != nil { c.Ui.Error(fmt.Sprintf("read error: %v", err)) @@ -169,7 +179,7 @@ func (c *PlanCommand) Run(args []string) int { userSet := userSetKeys(flags, varsFileKeys, c.arguments) - plays, err := tasks.GetPlays(data, context, userSet) + plays, err := tasks.GetPlaysWithFormat(data, c.tasksFormat, context, userSet) if err != nil { c.Ui.Error(fmt.Sprintf("task error: %v", err)) return 1 diff --git a/commands/task_file.go b/commands/task_file.go new file mode 100644 index 0000000..6452d36 --- /dev/null +++ b/commands/task_file.go @@ -0,0 +1,65 @@ +package commands + +import ( + "errors" + "fmt" + "os" + "path/filepath" + "strings" +) + +// Task file format identifiers used throughout the commands package and +// passed through to tasks.GetPlaysWithFormat / tasks.Validate via +// ValidateOptions.Format. Only these two values are valid; other strings +// are treated as YAML by the dispatchers. +const ( + taskFileFormatYAML = "yaml" + taskFileFormatJSON5 = "json5" +) + +// defaultTaskFileCandidates is the ordered list of filenames probed when +// --tasks is not given. The first one that exists in the working +// directory is used. The order matches the legacy default (tasks.yml) +// so behaviour does not change for existing recipes; .yaml and .json +// fall through to give JSON-native users a no-config setup. +var defaultTaskFileCandidates = []string{"tasks.yml", "tasks.yaml", "tasks.json"} + +// detectTaskFileFormat returns "json5" when path's extension is .json or +// .json5 (case-insensitive), and "yaml" otherwise. Unknown extensions +// default to YAML so explicit paths like `--tasks recipe.txt` keep the +// pre-#218 behaviour. HTTP URLs and other non-filesystem paths flow +// through filepath.Ext just fine because they still carry an extension. +func detectTaskFileFormat(path string) string { + switch strings.ToLower(filepath.Ext(path)) { + case ".json", ".json5": + return taskFileFormatJSON5 + default: + return taskFileFormatYAML + } +} + +// resolveTaskFilePath returns the path to use as the task file plus its +// detected format. When explicit is non-empty it is used as-is and the +// format is inferred from its extension; the file's existence is not +// checked here so the caller's os.ReadFile produces the canonical "no +// such file" error message. When explicit is empty the function probes +// defaultTaskFileCandidates in order and returns the first one that +// exists. If none exist the returned error names every candidate so the +// user can see which paths were tried. +func resolveTaskFilePath(explicit string) (string, string, error) { + if explicit != "" { + return explicit, detectTaskFileFormat(explicit), nil + } + for _, candidate := range defaultTaskFileCandidates { + if _, err := os.Stat(candidate); err == nil { + return candidate, detectTaskFileFormat(candidate), nil + } else if !errors.Is(err, os.ErrNotExist) { + return "", "", fmt.Errorf("stat %s: %w", candidate, err) + } + } + return "", "", fmt.Errorf("no task file found; looked for %s", strings.Join(defaultTaskFileCandidates, ", ")) +} + +// taskFileAutocompleteGlob is the shared file glob for the --tasks flag +// completion across apply / plan / validate / fmt / init. +const taskFileAutocompleteGlob = "*.{yml,yaml,json,json5}" diff --git a/commands/task_file_test.go b/commands/task_file_test.go new file mode 100644 index 0000000..a7766f4 --- /dev/null +++ b/commands/task_file_test.go @@ -0,0 +1,133 @@ +package commands + +import ( + "os" + "path/filepath" + "strings" + "testing" +) + +func TestDetectTaskFileFormat(t *testing.T) { + cases := map[string]string{ + "tasks.yml": taskFileFormatYAML, + "tasks.yaml": taskFileFormatYAML, + "tasks.YML": taskFileFormatYAML, + "tasks.json": taskFileFormatJSON5, + "tasks.JSON": taskFileFormatJSON5, + "tasks.json5": taskFileFormatJSON5, + "path/to/tasks.yml": taskFileFormatYAML, + "recipe.txt": taskFileFormatYAML, + "": taskFileFormatYAML, + } + for path, want := range cases { + if got := detectTaskFileFormat(path); got != want { + t.Errorf("detectTaskFileFormat(%q) = %q, want %q", path, got, want) + } + } +} + +func TestResolveTaskFilePathExplicit(t *testing.T) { + dir := t.TempDir() + path := filepath.Join(dir, "custom.json") + if err := os.WriteFile(path, []byte("[]"), 0o644); err != nil { + t.Fatalf("write: %v", err) + } + gotPath, gotFormat, err := resolveTaskFilePath(path) + if err != nil { + t.Fatalf("resolveTaskFilePath: %v", err) + } + if gotPath != path { + t.Errorf("path = %q, want %q", gotPath, path) + } + if gotFormat != taskFileFormatJSON5 { + t.Errorf("format = %q, want %q", gotFormat, taskFileFormatJSON5) + } +} + +func TestResolveTaskFilePathDefaultPrefersYAML(t *testing.T) { + dir := t.TempDir() + if err := os.WriteFile(filepath.Join(dir, "tasks.yml"), []byte("---\n"), 0o644); err != nil { + t.Fatalf("write yml: %v", err) + } + if err := os.WriteFile(filepath.Join(dir, "tasks.json"), []byte("[]"), 0o644); err != nil { + t.Fatalf("write json: %v", err) + } + withCwd(t, dir, func() { + path, format, err := resolveTaskFilePath("") + if err != nil { + t.Fatalf("resolveTaskFilePath: %v", err) + } + if path != "tasks.yml" { + t.Errorf("path = %q, want tasks.yml", path) + } + if format != taskFileFormatYAML { + t.Errorf("format = %q, want yaml", format) + } + }) +} + +func TestResolveTaskFilePathDefaultFallsThroughToJSON(t *testing.T) { + dir := t.TempDir() + if err := os.WriteFile(filepath.Join(dir, "tasks.json"), []byte("[]"), 0o644); err != nil { + t.Fatalf("write json: %v", err) + } + withCwd(t, dir, func() { + path, format, err := resolveTaskFilePath("") + if err != nil { + t.Fatalf("resolveTaskFilePath: %v", err) + } + if path != "tasks.json" { + t.Errorf("path = %q, want tasks.json", path) + } + if format != taskFileFormatJSON5 { + t.Errorf("format = %q, want json5", format) + } + }) +} + +func TestResolveTaskFilePathDefaultErrorsWhenNoneExist(t *testing.T) { + dir := t.TempDir() + withCwd(t, dir, func() { + _, _, err := resolveTaskFilePath("") + if err == nil { + t.Fatal("expected error when no candidate task file exists") + } + if !strings.Contains(err.Error(), "no task file found") { + t.Errorf("error = %q, want substring 'no task file found'", err.Error()) + } + }) +} + +func TestResolveTaskFileFromArgsUsesExplicitFlag(t *testing.T) { + path, format := resolveTaskFileFromArgs([]string{"docket", "apply", "--tasks", "custom.json"}) + if path != "custom.json" { + t.Errorf("path = %q, want custom.json", path) + } + if format != taskFileFormatJSON5 { + t.Errorf("format = %q, want json5", format) + } + + path, format = resolveTaskFileFromArgs([]string{"docket", "apply", "--tasks=other.yml"}) + if path != "other.yml" { + t.Errorf("path = %q, want other.yml", path) + } + if format != taskFileFormatYAML { + t.Errorf("format = %q, want yaml", format) + } +} + +// withCwd chdirs to dir for the duration of body and restores the +// original cwd afterwards. Centralised so the resolveTaskFilePath +// tests do not each handle the t.Cleanup dance. +func withCwd(t *testing.T, dir string, body func()) { + t.Helper() + prev, err := os.Getwd() + if err != nil { + t.Fatalf("getwd: %v", err) + } + if err := os.Chdir(dir); err != nil { + t.Fatalf("chdir %s: %v", dir, err) + } + t.Cleanup(func() { _ = os.Chdir(prev) }) + body() +} diff --git a/commands/templates/default.json5.tmpl b/commands/templates/default.json5.tmpl new file mode 100644 index 0000000..b9cbc90 --- /dev/null +++ b/commands/templates/default.json5.tmpl @@ -0,0 +1,53 @@ +[ + { + inputs: [ + { + name: "app", + default: "<<.Name>>", + description: "Application name", + }, + { + name: "repo", + <>default: "<<.Repo>>",<>required: true,<> + description: "Git repository URL", + }, + ], + tasks: [ + { + name: "create app", + dokku_app: { + app: "{{ .app }}", + state: "present", + }, + }, + + { + name: "configure env vars", + dokku_config: { + app: "{{ .app }}", + config: { + APP_ENV: "production", + LOG_LEVEL: "info", + }, + }, + }, + + { + name: "configure domains", + dokku_domains: { + app: "{{ .app }}", + domains: ["{{ .app }}.example.com"], + state: "set", + }, + }, + + { + name: "sync git repository", + dokku_git_sync: { + app: "{{ .app }}", + remote: "{{ .repo }}", + }, + }, + ], + }, +] diff --git a/commands/templates/minimal.json5.tmpl b/commands/templates/minimal.json5.tmpl new file mode 100644 index 0000000..835f67a --- /dev/null +++ b/commands/templates/minimal.json5.tmpl @@ -0,0 +1,13 @@ +[ + { + tasks: [ + { + name: "create app", + dokku_app: { + app: "<<.Name>>", + state: "present", + }, + }, + ], + }, +] diff --git a/commands/templates/templates.go b/commands/templates/templates.go index 1c55a77..9cc1192 100644 --- a/commands/templates/templates.go +++ b/commands/templates/templates.go @@ -7,5 +7,5 @@ package templates import "embed" -//go:embed *.yml.tmpl +//go:embed *.yml.tmpl *.json5.tmpl var FS embed.FS diff --git a/commands/validate.go b/commands/validate.go index 04e95cd..9234172 100644 --- a/commands/validate.go +++ b/commands/validate.go @@ -20,6 +20,7 @@ type ValidateCommand struct { command.Meta tasksFile string + tasksFormat string json bool strict bool varsFiles []string @@ -44,7 +45,8 @@ func (c *ValidateCommand) Examples() map[string]string { appName := os.Getenv("CLI_APP_NAME") return map[string]string{ "Validate the default tasks.yml": fmt.Sprintf("%s %s", appName, c.Name()), - "Validate a specific file": fmt.Sprintf("%s %s --tasks path/to/task.yml", appName, c.Name()), + "Validate a specific YAML file": fmt.Sprintf("%s %s --tasks path/to/task.yml", appName, c.Name()), + "Validate a JSON5 file": fmt.Sprintf("%s %s --tasks path/to/tasks.json", appName, c.Name()), "Emit JSON-lines problem events": fmt.Sprintf("%s %s --json", appName, c.Name()), "Flag required inputs without an override": fmt.Sprintf("%s %s --strict", appName, c.Name()), } @@ -64,20 +66,20 @@ func (c *ValidateCommand) ParsedArguments(args []string) (map[string]command.Arg func (c *ValidateCommand) FlagSet() *flag.FlagSet { f := c.Meta.FlagSet(c.Name(), command.FlagSetClient) - f.StringVar(&c.tasksFile, "tasks", "tasks.yml", "a yaml file containing a task list") + f.StringVar(&c.tasksFile, "tasks", "", "task file (YAML or JSON5) containing a task list. When omitted, docket probes tasks.yml -> tasks.yaml -> tasks.json in the current directory.") f.BoolVar(&c.json, "json", false, "emit one JSON-lines problem event per finding") f.BoolVar(&c.strict, "strict", false, "additionally flag required inputs that have no default and no CLI override, and check that --play / --start-at-task references resolve to real names in the file") f.StringArrayVar(&c.varsFiles, "vars-file", nil, "load input values from a YAML or JSON file (repeatable; later files override earlier; CLI --name=value flags always win). A .json extension parses as JSON; otherwise YAML.") f.StringVar(&c.play, "play", "", "(strict) verify the named play exists in the recipe (matches the play's `name:` field; auto-named plays use `play #N`)") f.StringVar(&c.startAtTask, "start-at-task", "", "(strict) verify a task with this name exists in the recipe; narrowed by --play when set") - taskFile := getTaskYamlFilename(os.Args) + taskFile, format := resolveTaskFileFromArgs(os.Args) data, err := os.ReadFile(taskFile) if err != nil { return f } - arguments, err := registerInputFlags(f, data) + arguments, err := registerInputFlags(f, data, format) if err != nil { return f } @@ -90,7 +92,7 @@ func (c *ValidateCommand) AutocompleteFlags() complete.Flags { return command.MergeAutocompleteFlags( c.Meta.AutocompleteFlags(command.FlagSetClient), complete.Flags{ - "--tasks": complete.PredictFiles("*.yml"), + "--tasks": complete.PredictFiles(taskFileAutocompleteGlob), "--json": complete.PredictNothing, "--strict": complete.PredictNothing, "--vars-file": complete.PredictFiles("*"), @@ -127,6 +129,21 @@ func (c *ValidateCommand) Run(args []string) int { return 1 } + resolvedPath, resolvedFormat, err := resolveTaskFilePath(c.tasksFile) + if err != nil { + if c.json { + c.emitJSONProblem(tasks.Problem{ + Code: "read_error", + Message: err.Error(), + }) + } else { + c.Ui.Error(fmt.Sprintf("read error: %v", err)) + } + return 1 + } + c.tasksFile = resolvedPath + c.tasksFormat = resolvedFormat + data, err := os.ReadFile(c.tasksFile) if err != nil { if c.json { @@ -152,6 +169,7 @@ func (c *ValidateCommand) Run(args []string) int { InputOverrides: overrides, PlayName: c.play, StartAtTask: c.startAtTask, + Format: c.tasksFormat, }) if c.json { diff --git a/go.mod b/go.mod index 5a7e975..43c75a0 100644 --- a/go.mod +++ b/go.mod @@ -42,6 +42,7 @@ require ( github.com/rs/zerolog v1.35.1 // indirect github.com/shopspring/decimal v1.4.0 // indirect github.com/spf13/cast v1.10.0 // indirect + github.com/titanous/json5 v1.0.0 // indirect golang.org/x/crypto v0.50.0 // indirect golang.org/x/sys v0.43.0 // indirect golang.org/x/term v0.42.0 // indirect diff --git a/go.sum b/go.sum index 33ffe57..dd5fa4f 100644 --- a/go.sum +++ b/go.sum @@ -119,6 +119,8 @@ github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/ github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU= github.com/stretchr/testify v1.8.1 h1:w7B6lhMri9wdJUVmEZPGGhZzrYTPvgJArz7wNPgYKsk= github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4= +github.com/titanous/json5 v1.0.0 h1:hJf8Su1d9NuI/ffpxgxQfxh/UiBFZX7bMPid0rIL/7s= +github.com/titanous/json5 v1.0.0/go.mod h1:7JH1M8/LHKc6cyP5o5g3CSaRj+mBrIimTxzpvmckH8c= golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= golang.org/x/crypto v0.0.0-20200414173820-0848c9571904/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= golang.org/x/crypto v0.0.0-20200820211705-5c72a883971a/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= diff --git a/tasks/format_json5.go b/tasks/format_json5.go new file mode 100644 index 0000000..86a9c97 --- /dev/null +++ b/tasks/format_json5.go @@ -0,0 +1,909 @@ +package tasks + +import ( + "bytes" + "fmt" + "strings" + "unicode" + "unicode/utf8" +) + +// FormatJSON5 returns the canonical JSON5 form of data. err is non-nil +// only on a parse error or when the round-trip equivalence guard fails. +// +// The formatter mirrors the YAML Format() contract, adapted to JSON5: +// +// - 2-space indentation. +// - Reorders play and task envelope keys per canonicalPlayKeys / +// canonicalEnvelopeKeys (task-type key last). Members not in the +// canonical list keep their relative order, appended afterwards. +// - Inserts a blank line between top-level plays and between top- +// level task entries inside a play's tasks list, matching the YAML +// formatter so a JSON5 recipe and its YAML twin look the same. +// - Always emits trailing commas on multiline objects and arrays to +// match JSON5 community convention. +// - Quotes string values with double quotes; quotes object keys only +// when they are not valid JSON5 identifiers. +// - Preserves line and block comments in their original anchor +// position (before a member, beside it on the same line, or after +// the last member of a container). +// - Re-parses the canonical output and aborts unless the original +// and canonical AST trees are structurally equivalent. Catches +// emitter edge cases before the caller writes anything to disk. +func FormatJSON5(data []byte) ([]byte, error) { + root, err := parseJSON5(data) + if err != nil { + return nil, fmt.Errorf("json5 parse error: %w", err) + } + + canonicaliseJSON5Recipe(root) + + var buf bytes.Buffer + emitJSON5(&buf, root, 0) + out := buf.Bytes() + + roundTrip, err := parseJSON5(out) + if err != nil { + return nil, fmt.Errorf("round-trip parse error: %w", err) + } + if !equivalentJSON5Nodes(root, roundTrip) { + return nil, fmt.Errorf("round-trip equivalence check failed; refusing to write") + } + + return out, nil +} + +// json5Node is the AST shape produced by parseJSON5. Kind discriminates +// the variant; fields are populated only for the relevant kind. +type json5Node struct { + Kind json5Kind + + // Object members; nil when Kind != json5Object. + Members []*json5Member + + // Array elements; nil when Kind != json5Array. + Elements []*json5Element + + // Raw scalar source (the original token, e.g. `"hello"`, `42`, + // `true`, `null`, or an unquoted identifier where JSON5 allows it). + // Kept as-is so number formats and string escapes round-trip. + Raw string + + // HeadComments are comments that appeared on lines above this + // node, preserved in source order. Used for top-level documents + // and for object/array comments that are not associated with any + // member. + HeadComments []string + + // FootComments are trailing comments inside a container after the + // last member / element but before the closing brace / bracket. + FootComments []string +} + +// json5Kind enumerates AST node variants. +type json5Kind int + +const ( + json5Object json5Kind = iota + json5Array + json5Scalar +) + +// json5Member is a single (key, value) pair inside an object node. +type json5Member struct { + Key string + Value *json5Node + HeadComments []string + LineComment string // trailing // ... or /* ... */ on the same line as the value +} + +// json5Element is a single value inside an array node. +type json5Element struct { + Value *json5Node + HeadComments []string + LineComment string +} + +// canonicaliseJSON5Recipe applies the same canonical-key ordering the +// YAML formatter applies. The top-level node is expected to be an array +// (the recipe shape); each element is a play whose keys are reordered, +// and each play's tasks: array entry has its envelope keys reordered. +func canonicaliseJSON5Recipe(root *json5Node) { + if root == nil || root.Kind != json5Array { + return + } + for _, play := range root.Elements { + if play.Value == nil || play.Value.Kind != json5Object { + continue + } + reorderJSON5Object(play.Value, canonicalPlayKeys) + tasksMember := findJSON5Member(play.Value, "tasks") + if tasksMember == nil || tasksMember.Value == nil || tasksMember.Value.Kind != json5Array { + continue + } + for _, task := range tasksMember.Value.Elements { + if task.Value == nil || task.Value.Kind != json5Object { + continue + } + reorderJSON5TaskEnvelope(task.Value) + } + } +} + +// reorderJSON5Object rebuilds Members so canonical keys appear in the +// given order, with any unrecognised keys appended in their original +// relative order. +func reorderJSON5Object(node *json5Node, priority []string) { + out := make([]*json5Member, 0, len(node.Members)) + used := make(map[int]bool, len(node.Members)) + for _, key := range priority { + for i, m := range node.Members { + if used[i] { + continue + } + if m.Key == key { + out = append(out, m) + used[i] = true + break + } + } + } + for i, m := range node.Members { + if used[i] { + continue + } + out = append(out, m) + } + node.Members = out +} + +// reorderJSON5TaskEnvelope reorders a task entry object so the task-type +// key (the single non-envelope key) comes last, with envelope keys +// ahead in canonicalEnvelopeKeys order. +func reorderJSON5TaskEnvelope(node *json5Node) { + out := make([]*json5Member, 0, len(node.Members)) + used := make(map[int]bool, len(node.Members)) + for _, key := range canonicalEnvelopeKeys { + for i, m := range node.Members { + if used[i] { + continue + } + if m.Key == key { + out = append(out, m) + used[i] = true + break + } + } + } + for i, m := range node.Members { + if used[i] { + continue + } + if envelopeKeySet[m.Key] { + out = append(out, m) + used[i] = true + } + } + for i, m := range node.Members { + if used[i] { + continue + } + out = append(out, m) + } + node.Members = out +} + +func findJSON5Member(node *json5Node, key string) *json5Member { + for _, m := range node.Members { + if m.Key == key { + return m + } + } + return nil +} + +// equivalentJSON5Nodes compares two AST trees structurally, ignoring +// comments. Object key order is not considered (canonicalisation +// reorders by design); arrays compare element-wise. +func equivalentJSON5Nodes(a, b *json5Node) bool { + if a == nil && b == nil { + return true + } + if a == nil || b == nil || a.Kind != b.Kind { + return false + } + switch a.Kind { + case json5Scalar: + return normaliseJSON5Scalar(a.Raw) == normaliseJSON5Scalar(b.Raw) + case json5Array: + if len(a.Elements) != len(b.Elements) { + return false + } + for i := range a.Elements { + if !equivalentJSON5Nodes(a.Elements[i].Value, b.Elements[i].Value) { + return false + } + } + return true + case json5Object: + if len(a.Members) != len(b.Members) { + return false + } + bIdx := make(map[string]*json5Node, len(b.Members)) + for _, m := range b.Members { + bIdx[m.Key] = m.Value + } + for _, m := range a.Members { + bv, ok := bIdx[m.Key] + if !ok { + return false + } + if !equivalentJSON5Nodes(m.Value, bv) { + return false + } + } + return true + } + return false +} + +// normaliseJSON5Scalar collapses representational differences that do +// not affect semantics: a single-quoted string and a double-quoted +// string with the same decoded body are equal; +5 and 5 are equal as +// numbers. Implemented as a best-effort string normalisation; this +// function is only used by the round-trip guard so false negatives +// surface as a refusal to write rather than corruption. +func normaliseJSON5Scalar(raw string) string { + if len(raw) == 0 { + return raw + } + switch raw[0] { + case '"', '\'': + decoded, ok := decodeJSON5String(raw) + if ok { + return "S:" + decoded + } + } + return strings.TrimPrefix(raw, "+") +} + +func decodeJSON5String(raw string) (string, bool) { + if len(raw) < 2 { + return "", false + } + quote := raw[0] + if raw[len(raw)-1] != quote { + return "", false + } + body := raw[1 : len(raw)-1] + var b strings.Builder + for i := 0; i < len(body); i++ { + c := body[i] + if c != '\\' { + b.WriteByte(c) + continue + } + i++ + if i >= len(body) { + return "", false + } + switch body[i] { + case 'n': + b.WriteByte('\n') + case 't': + b.WriteByte('\t') + case 'r': + b.WriteByte('\r') + case '"', '\'', '\\', '/': + b.WriteByte(body[i]) + case '\n': + // JSON5 line continuation; keep nothing. + default: + b.WriteByte('\\') + b.WriteByte(body[i]) + } + } + return b.String(), true +} + +// --------------------------------------------------------------------- +// Lexer +// --------------------------------------------------------------------- + +type json5TokKind int + +const ( + tokEOF json5TokKind = iota + tokLBrace + tokRBrace + tokLBracket + tokRBracket + tokColon + tokComma + tokString + tokNumber + tokIdent // unquoted identifier (also true / false / null) + tokLineComment + tokBlockComment +) + +type json5Tok struct { + Kind json5TokKind + Raw string // verbatim source slice + NewlineBefore bool // true if any newline preceded this token +} + +type json5Lexer struct { + src []byte + pos int + tokens []json5Tok +} + +func lexJSON5(src []byte) ([]json5Tok, error) { + l := &json5Lexer{src: src} + pendingNewline := false + for l.pos < len(l.src) { + c := l.src[l.pos] + if c == ' ' || c == '\t' || c == '\r' { + l.pos++ + continue + } + if c == '\n' { + l.pos++ + pendingNewline = true + continue + } + if c == '/' && l.pos+1 < len(l.src) && l.src[l.pos+1] == '/' { + start := l.pos + l.pos += 2 + for l.pos < len(l.src) && l.src[l.pos] != '\n' { + l.pos++ + } + l.tokens = append(l.tokens, json5Tok{Kind: tokLineComment, Raw: string(l.src[start:l.pos]), NewlineBefore: pendingNewline}) + pendingNewline = false + continue + } + if c == '/' && l.pos+1 < len(l.src) && l.src[l.pos+1] == '*' { + start := l.pos + l.pos += 2 + for l.pos+1 < len(l.src) && !(l.src[l.pos] == '*' && l.src[l.pos+1] == '/') { + l.pos++ + } + if l.pos+1 >= len(l.src) { + return nil, fmt.Errorf("unterminated block comment at offset %d", start) + } + l.pos += 2 + l.tokens = append(l.tokens, json5Tok{Kind: tokBlockComment, Raw: string(l.src[start:l.pos]), NewlineBefore: pendingNewline}) + pendingNewline = false + continue + } + + switch c { + case '{': + l.tokens = append(l.tokens, json5Tok{Kind: tokLBrace, Raw: "{", NewlineBefore: pendingNewline}) + l.pos++ + pendingNewline = false + continue + case '}': + l.tokens = append(l.tokens, json5Tok{Kind: tokRBrace, Raw: "}", NewlineBefore: pendingNewline}) + l.pos++ + pendingNewline = false + continue + case '[': + l.tokens = append(l.tokens, json5Tok{Kind: tokLBracket, Raw: "[", NewlineBefore: pendingNewline}) + l.pos++ + pendingNewline = false + continue + case ']': + l.tokens = append(l.tokens, json5Tok{Kind: tokRBracket, Raw: "]", NewlineBefore: pendingNewline}) + l.pos++ + pendingNewline = false + continue + case ':': + l.tokens = append(l.tokens, json5Tok{Kind: tokColon, Raw: ":", NewlineBefore: pendingNewline}) + l.pos++ + pendingNewline = false + continue + case ',': + l.tokens = append(l.tokens, json5Tok{Kind: tokComma, Raw: ",", NewlineBefore: pendingNewline}) + l.pos++ + pendingNewline = false + continue + case '"', '\'': + tok, err := l.readString(c) + if err != nil { + return nil, err + } + tok.NewlineBefore = pendingNewline + pendingNewline = false + l.tokens = append(l.tokens, tok) + continue + } + + if c == '-' || c == '+' || c == '.' || (c >= '0' && c <= '9') { + tok, err := l.readNumber() + if err != nil { + return nil, err + } + tok.NewlineBefore = pendingNewline + pendingNewline = false + l.tokens = append(l.tokens, tok) + continue + } + + if isIdentStart(rune(c)) { + tok := l.readIdent() + tok.NewlineBefore = pendingNewline + pendingNewline = false + l.tokens = append(l.tokens, tok) + continue + } + + return nil, fmt.Errorf("unexpected character %q at offset %d", c, l.pos) + } + l.tokens = append(l.tokens, json5Tok{Kind: tokEOF, NewlineBefore: pendingNewline}) + return l.tokens, nil +} + +func (l *json5Lexer) readString(quote byte) (json5Tok, error) { + start := l.pos + l.pos++ + for l.pos < len(l.src) { + c := l.src[l.pos] + if c == '\\' { + l.pos += 2 + continue + } + if c == quote { + l.pos++ + return json5Tok{Kind: tokString, Raw: string(l.src[start:l.pos])}, nil + } + l.pos++ + } + return json5Tok{}, fmt.Errorf("unterminated string starting at offset %d", start) +} + +func (l *json5Lexer) readNumber() (json5Tok, error) { + start := l.pos + if l.src[l.pos] == '+' || l.src[l.pos] == '-' { + l.pos++ + } + if l.pos+1 < len(l.src) && l.src[l.pos] == '0' && (l.src[l.pos+1] == 'x' || l.src[l.pos+1] == 'X') { + l.pos += 2 + for l.pos < len(l.src) && isHexDigit(l.src[l.pos]) { + l.pos++ + } + return json5Tok{Kind: tokNumber, Raw: string(l.src[start:l.pos])}, nil + } + for l.pos < len(l.src) { + c := l.src[l.pos] + if (c >= '0' && c <= '9') || c == '.' || c == 'e' || c == 'E' || c == '+' || c == '-' { + l.pos++ + continue + } + break + } + if l.pos == start { + return json5Tok{}, fmt.Errorf("invalid number at offset %d", start) + } + return json5Tok{Kind: tokNumber, Raw: string(l.src[start:l.pos])}, nil +} + +func (l *json5Lexer) readIdent() json5Tok { + start := l.pos + for l.pos < len(l.src) { + r, sz := utf8.DecodeRune(l.src[l.pos:]) + if !isIdentPart(r) { + break + } + l.pos += sz + } + return json5Tok{Kind: tokIdent, Raw: string(l.src[start:l.pos])} +} + +func isIdentStart(r rune) bool { + return r == '_' || r == '$' || unicode.IsLetter(r) +} + +func isIdentPart(r rune) bool { + return isIdentStart(r) || unicode.IsDigit(r) +} + +func isHexDigit(c byte) bool { + return (c >= '0' && c <= '9') || (c >= 'a' && c <= 'f') || (c >= 'A' && c <= 'F') +} + +// isJSON5IdentKey reports whether s is safe to emit unquoted as an +// object key. Keeps in step with the JSON5 spec's IdentifierName rule +// (subset chosen here for ASCII identifiers). +func isJSON5IdentKey(s string) bool { + if s == "" { + return false + } + for i, r := range s { + if i == 0 { + if !isIdentStart(r) { + return false + } + continue + } + if !isIdentPart(r) { + return false + } + } + return true +} + +// --------------------------------------------------------------------- +// Parser +// --------------------------------------------------------------------- + +type json5Parser struct { + tokens []json5Tok + pos int +} + +func parseJSON5(src []byte) (*json5Node, error) { + tokens, err := lexJSON5(src) + if err != nil { + return nil, err + } + p := &json5Parser{tokens: tokens} + headComments := p.consumeComments() + root, err := p.parseValue() + if err != nil { + return nil, err + } + if root != nil { + root.HeadComments = append(headComments, root.HeadComments...) + } + footComments := p.consumeComments() + if root != nil { + root.FootComments = append(root.FootComments, footComments...) + } + if p.peek().Kind != tokEOF { + return nil, fmt.Errorf("unexpected token %q after root value", p.peek().Raw) + } + return root, nil +} + +func (p *json5Parser) peek() json5Tok { + if p.pos < len(p.tokens) { + return p.tokens[p.pos] + } + return json5Tok{Kind: tokEOF} +} + +func (p *json5Parser) advance() json5Tok { + t := p.peek() + p.pos++ + return t +} + +// consumeComments consumes a run of comment tokens and returns them +// as raw strings. +func (p *json5Parser) consumeComments() []string { + var out []string + for { + t := p.peek() + if t.Kind != tokLineComment && t.Kind != tokBlockComment { + return out + } + out = append(out, t.Raw) + p.pos++ + } +} + +// consumeTrailingLineComment consumes one line/block comment that sits +// on the same line as the just-emitted value (no preceding newline). +// Returns "" if the next non-trivia token is not a same-line comment. +func (p *json5Parser) consumeTrailingLineComment() string { + t := p.peek() + if (t.Kind == tokLineComment || t.Kind == tokBlockComment) && !t.NewlineBefore { + p.pos++ + return t.Raw + } + return "" +} + +func (p *json5Parser) parseValue() (*json5Node, error) { + t := p.peek() + switch t.Kind { + case tokLBrace: + return p.parseObject() + case tokLBracket: + return p.parseArray() + case tokString, tokNumber, tokIdent: + p.advance() + return &json5Node{Kind: json5Scalar, Raw: t.Raw}, nil + } + return nil, fmt.Errorf("unexpected token %q while parsing value", t.Raw) +} + +func (p *json5Parser) parseObject() (*json5Node, error) { + if p.peek().Kind != tokLBrace { + return nil, fmt.Errorf("expected { at object start, got %q", p.peek().Raw) + } + p.advance() + node := &json5Node{Kind: json5Object} + for { + head := p.consumeComments() + if p.peek().Kind == tokRBrace { + p.advance() + node.FootComments = head + return node, nil + } + key, err := p.parseKey() + if err != nil { + return nil, err + } + if p.peek().Kind != tokColon { + return nil, fmt.Errorf("expected : after key %q, got %q", key, p.peek().Raw) + } + p.advance() + val, err := p.parseValue() + if err != nil { + return nil, err + } + member := &json5Member{Key: key, Value: val, HeadComments: head} + // Trailing comment on the same line as the value. + member.LineComment = p.consumeTrailingLineComment() + // Optional comma; trailing comma allowed. + if p.peek().Kind == tokComma { + p.advance() + if member.LineComment == "" { + member.LineComment = p.consumeTrailingLineComment() + } + } + node.Members = append(node.Members, member) + } +} + +func (p *json5Parser) parseArray() (*json5Node, error) { + if p.peek().Kind != tokLBracket { + return nil, fmt.Errorf("expected [ at array start, got %q", p.peek().Raw) + } + p.advance() + node := &json5Node{Kind: json5Array} + for { + head := p.consumeComments() + if p.peek().Kind == tokRBracket { + p.advance() + node.FootComments = head + return node, nil + } + val, err := p.parseValue() + if err != nil { + return nil, err + } + elem := &json5Element{Value: val, HeadComments: head} + elem.LineComment = p.consumeTrailingLineComment() + if p.peek().Kind == tokComma { + p.advance() + if elem.LineComment == "" { + elem.LineComment = p.consumeTrailingLineComment() + } + } + node.Elements = append(node.Elements, elem) + } +} + +func (p *json5Parser) parseKey() (string, error) { + t := p.advance() + switch t.Kind { + case tokString: + decoded, ok := decodeJSON5String(t.Raw) + if !ok { + return "", fmt.Errorf("invalid string key %q", t.Raw) + } + return decoded, nil + case tokIdent, tokNumber: + return t.Raw, nil + } + return "", fmt.Errorf("expected key, got %q", t.Raw) +} + +// --------------------------------------------------------------------- +// Emitter +// --------------------------------------------------------------------- + +const json5Indent = " " + +func emitJSON5(buf *bytes.Buffer, node *json5Node, depth int) { + emitJSON5HeadComments(buf, node.HeadComments, depth) + emitJSON5Value(buf, node, depth) + emitJSON5FootComments(buf, node.FootComments, depth) + if buf.Len() == 0 || buf.Bytes()[buf.Len()-1] != '\n' { + buf.WriteByte('\n') + } +} + +func emitJSON5Value(buf *bytes.Buffer, node *json5Node, depth int) { + emitJSON5ValueCtx(buf, node, depth, false) +} + +// emitJSON5ValueCtx is the value emitter aware of one extra context +// bit: tasksMember signals that the value being emitted is the value +// of a member named "tasks" / "block" / "rescue" / "always", so an +// array value gets blank-line separation between its top-level +// elements (matching the YAML formatter's tasks-list rule). +func emitJSON5ValueCtx(buf *bytes.Buffer, node *json5Node, depth int, tasksMember bool) { + if node == nil { + buf.WriteString("null") + return + } + switch node.Kind { + case json5Scalar: + buf.WriteString(canonicaliseScalarRaw(node.Raw)) + case json5Array: + emitJSON5Array(buf, node, depth, tasksMember) + case json5Object: + emitJSON5Object(buf, node, depth) + } +} + +func emitJSON5Array(buf *bytes.Buffer, node *json5Node, depth int, tasksMember bool) { + if len(node.Elements) == 0 && len(node.FootComments) == 0 { + buf.WriteString("[]") + return + } + if isInlineArray(node) { + buf.WriteByte('[') + for i, elem := range node.Elements { + if i > 0 { + buf.WriteString(", ") + } + emitJSON5Value(buf, elem.Value, depth) + } + buf.WriteByte(']') + return + } + buf.WriteByte('[') + buf.WriteByte('\n') + innerIndent := strings.Repeat(json5Indent, depth+1) + closingIndent := strings.Repeat(json5Indent, depth) + insertPlayBlankLine := depth == 0 + insertTaskBlankLine := tasksMember + for i, elem := range node.Elements { + if (insertTaskBlankLine || insertPlayBlankLine) && i > 0 { + buf.WriteByte('\n') + } + emitJSON5HeadComments(buf, elem.HeadComments, depth+1) + buf.WriteString(innerIndent) + emitJSON5Value(buf, elem.Value, depth+1) + buf.WriteByte(',') + if elem.LineComment != "" { + buf.WriteByte(' ') + buf.WriteString(elem.LineComment) + } + buf.WriteByte('\n') + } + emitJSON5FootComments(buf, node.FootComments, depth+1) + buf.WriteString(closingIndent) + buf.WriteByte(']') +} + +func emitJSON5Object(buf *bytes.Buffer, node *json5Node, depth int) { + if len(node.Members) == 0 && len(node.FootComments) == 0 { + buf.WriteString("{}") + return + } + buf.WriteByte('{') + buf.WriteByte('\n') + innerIndent := strings.Repeat(json5Indent, depth+1) + closingIndent := strings.Repeat(json5Indent, depth) + for _, m := range node.Members { + emitJSON5HeadComments(buf, m.HeadComments, depth+1) + buf.WriteString(innerIndent) + buf.WriteString(formatJSON5Key(m.Key)) + buf.WriteString(": ") + emitJSON5ValueCtx(buf, m.Value, depth+1, isTaskListMemberKey(m.Key)) + buf.WriteByte(',') + if m.LineComment != "" { + buf.WriteByte(' ') + buf.WriteString(m.LineComment) + } + buf.WriteByte('\n') + } + emitJSON5FootComments(buf, node.FootComments, depth+1) + buf.WriteString(closingIndent) + buf.WriteByte('}') +} + +// isTaskListMemberKey returns true for the canonical play / group +// keys that carry a list of task entries. The emitter inserts blank +// lines between elements of these arrays so the shape matches the +// YAML formatter's tasks-block rule. +func isTaskListMemberKey(key string) bool { + switch key { + case "tasks", "block", "rescue", "always": + return true + } + return false +} + +func emitJSON5HeadComments(buf *bytes.Buffer, comments []string, depth int) { + indent := strings.Repeat(json5Indent, depth) + for _, c := range comments { + buf.WriteString(indent) + buf.WriteString(c) + buf.WriteByte('\n') + } +} + +func emitJSON5FootComments(buf *bytes.Buffer, comments []string, depth int) { + emitJSON5HeadComments(buf, comments, depth) +} + +// isInlineArray returns true for arrays of scalars that should be +// rendered on a single line. Empty arrays handled separately. Keep +// scalar-only arrays one-line so `domains: ["a.example.com"]` renders +// the same way it does in the YAML formatter. +func isInlineArray(node *json5Node) bool { + if len(node.FootComments) > 0 { + return false + } + for _, e := range node.Elements { + if e == nil || e.Value == nil || e.Value.Kind != json5Scalar { + return false + } + if len(e.HeadComments) > 0 || e.LineComment != "" { + return false + } + } + return len(node.Elements) > 0 +} + +// formatJSON5Key chooses between an unquoted identifier (when valid) +// and a double-quoted string. Always quoting works too but the +// identifier form is the JSON5 community default for plain keys. +func formatJSON5Key(key string) string { + if isJSON5IdentKey(key) { + return key + } + return quoteJSON5String(key) +} + +// quoteJSON5String wraps s in double quotes, escaping the canonical +// JSON5 escape characters. Used when emitting object keys; string +// values keep their original quote form via canonicaliseScalarRaw. +func quoteJSON5String(s string) string { + var b strings.Builder + b.WriteByte('"') + for _, r := range s { + switch r { + case '\\': + b.WriteString(`\\`) + case '"': + b.WriteString(`\"`) + case '\n': + b.WriteString(`\n`) + case '\t': + b.WriteString(`\t`) + case '\r': + b.WriteString(`\r`) + default: + b.WriteRune(r) + } + } + b.WriteByte('"') + return b.String() +} + +// canonicaliseScalarRaw normalises a scalar's source form: single- +// quoted strings convert to double-quoted (idiomatic JSON5 output), +// other scalars (numbers, true, false, null, identifier-form values) +// pass through verbatim. Sigil templates inside strings are preserved +// because they live entirely inside the quoted body. +func canonicaliseScalarRaw(raw string) string { + if len(raw) == 0 { + return raw + } + if raw[0] == '\'' && raw[len(raw)-1] == '\'' { + decoded, ok := decodeJSON5String(raw) + if ok { + return quoteJSON5String(decoded) + } + } + return raw +} diff --git a/tasks/format_json5_test.go b/tasks/format_json5_test.go new file mode 100644 index 0000000..70aaa10 --- /dev/null +++ b/tasks/format_json5_test.go @@ -0,0 +1,168 @@ +package tasks + +import ( + "strings" + "testing" +) + +func TestFormatJSON5IdempotentOnCanonicalInput(t *testing.T) { + in := `[ + { + name: "api", + tasks: [ + { + name: "create", + dokku_app: { + app: "api", + }, + }, + ], + }, +] +` + out, err := FormatJSON5([]byte(in)) + if err != nil { + t.Fatalf("FormatJSON5: %v", err) + } + if string(out) != in { + t.Errorf("non-idempotent format:\nwant:\n%s\ngot:\n%s", in, string(out)) + } +} + +func TestFormatJSON5ReordersPlayKeys(t *testing.T) { + in := []byte(`[{ tasks: [], inputs: [], name: "api" }]`) + out, err := FormatJSON5(in) + if err != nil { + t.Fatalf("FormatJSON5: %v", err) + } + // name should come first, inputs second, tasks last among canonical keys. + idxName := strings.Index(string(out), "name:") + idxInputs := strings.Index(string(out), "inputs:") + idxTasks := strings.Index(string(out), "tasks:") + if !(idxName < idxInputs && idxInputs < idxTasks) { + t.Errorf("canonical key order not enforced:\n%s", out) + } +} + +func TestFormatJSON5ReordersTaskEnvelopeKeys(t *testing.T) { + in := []byte(`[{ tasks: [{ dokku_app: { app: "api" }, name: "create" }] }]`) + out, err := FormatJSON5(in) + if err != nil { + t.Fatalf("FormatJSON5: %v", err) + } + idxName := strings.Index(string(out), "name:") + idxDokku := strings.Index(string(out), "dokku_app:") + if idxName < 0 || idxDokku < 0 || idxName >= idxDokku { + t.Errorf("envelope key order not enforced (name should come before task-type):\n%s", out) + } +} + +func TestFormatJSON5PreservesLineComments(t *testing.T) { + in := []byte(`[ + // top of recipe + { + tasks: [ + { + name: "create", // inline comment + dokku_app: { app: "api" }, + }, + ], + }, +]`) + out, err := FormatJSON5(in) + if err != nil { + t.Fatalf("FormatJSON5: %v", err) + } + if !strings.Contains(string(out), "// top of recipe") { + t.Errorf("head comment lost:\n%s", out) + } + if !strings.Contains(string(out), "// inline comment") { + t.Errorf("trailing line comment lost:\n%s", out) + } +} + +func TestFormatJSON5PreservesBlockComments(t *testing.T) { + in := []byte(`[ + /* preface */ + { tasks: [] }, +]`) + out, err := FormatJSON5(in) + if err != nil { + t.Fatalf("FormatJSON5: %v", err) + } + if !strings.Contains(string(out), "/* preface */") { + t.Errorf("block comment lost:\n%s", out) + } +} + +func TestFormatJSON5BlankLinesBetweenPlays(t *testing.T) { + in := []byte(`[ + { name: "a", tasks: [] }, + { name: "b", tasks: [] }, +]`) + out, err := FormatJSON5(in) + if err != nil { + t.Fatalf("FormatJSON5: %v", err) + } + // Expect exactly one blank line between the two plays. + if !strings.Contains(string(out), "},\n\n {") { + t.Errorf("blank line between plays missing:\n%s", out) + } +} + +func TestFormatJSON5RejectsInvalidInput(t *testing.T) { + in := []byte(`[{ tasks: [`) + _, err := FormatJSON5(in) + if err == nil { + t.Fatal("expected error on truncated input") + } + if !strings.Contains(err.Error(), "json5 parse error") { + t.Errorf("error = %q, want json5 parse error", err.Error()) + } +} + +func TestFormatJSON5HandlesEmptyArray(t *testing.T) { + in := []byte(`[]`) + out, err := FormatJSON5(in) + if err != nil { + t.Fatalf("FormatJSON5: %v", err) + } + got := strings.TrimSpace(string(out)) + if got != "[]" { + t.Errorf("empty array not preserved: %q", got) + } +} + +func TestFormatJSON5InlinesScalarArrays(t *testing.T) { + in := []byte(`[{ tasks: [{ dokku_domains: { app: "api", domains: ["a.example.com", "b.example.com"] } }] }]`) + out, err := FormatJSON5(in) + if err != nil { + t.Fatalf("FormatJSON5: %v", err) + } + if !strings.Contains(string(out), `["a.example.com", "b.example.com"]`) { + t.Errorf("scalar array should be inlined:\n%s", out) + } +} + +func TestFormatJSON5RoundTripsTrailingCommas(t *testing.T) { + in := []byte(`[{ tasks: [{ dokku_app: { app: "api" }, }, ], }, ]`) + out, err := FormatJSON5(in) + if err != nil { + t.Fatalf("FormatJSON5: %v", err) + } + // Re-parse to confirm valid JSON5. + if _, err := parseJSON5(out); err != nil { + t.Fatalf("formatted output does not re-parse: %v\n%s", err, out) + } +} + +func TestFormatJSON5SigilTemplatesSurvive(t *testing.T) { + in := []byte(`[{ tasks: [{ dokku_app: { app: "{{ .app }}" } }] }]`) + out, err := FormatJSON5(in) + if err != nil { + t.Fatalf("FormatJSON5: %v", err) + } + if !strings.Contains(string(out), `"{{ .app }}"`) { + t.Errorf("sigil template lost:\n%s", out) + } +} diff --git a/tasks/main.go b/tasks/main.go index e8f5b0a..cc81f00 100644 --- a/tasks/main.go +++ b/tasks/main.go @@ -12,9 +12,43 @@ import ( sigil "github.com/gliderlabs/sigil" "github.com/gobuffalo/flect" defaults "github.com/mcuadros/go-defaults" + json5 "github.com/titanous/json5" yaml "gopkg.in/yaml.v3" ) +// Task file format identifiers shared with the commands package. +// +// The empty string and any unrecognised value are treated as YAML so +// existing call sites that pass no format keep their pre-#218 behaviour. +const ( + FormatYAML = "yaml" + FormatNameJSON5 = "json5" +) + +// IsJSON5Format returns true when format is one of the JSON5 aliases. +// Centralised so the json/json5 split lives in exactly one place. +func IsJSON5Format(format string) bool { + return format == FormatNameJSON5 || format == "json" +} + +// UnmarshalRecipe decodes data as a Recipe using the parser keyed by +// format. Exposed because the commands package's input-extraction path +// (parseInputDocument) needs the same dispatch and there is no benefit +// to duplicating the switch. +func UnmarshalRecipe(data []byte, format string) (Recipe, error) { + recipe := Recipe{} + if IsJSON5Format(format) { + if err := json5.Unmarshal(data, &recipe); err != nil { + return nil, fmt.Errorf("json5 unmarshal error: %v", err.Error()) + } + return recipe, nil + } + if err := yaml.Unmarshal(data, &recipe); err != nil { + return nil, fmt.Errorf("unmarshal error: %v", err.Error()) + } + return recipe, nil +} + // State represents the desired state of a task type State string @@ -49,48 +83,48 @@ type Recipe []RecipeEntry type RecipeEntry struct { // Name is the play's user-facing label. Auto-generated as // "play #N" by GetPlays when omitted. - Name string `yaml:"name,omitempty"` + Name string `yaml:"name,omitempty" json:"name,omitempty"` // Tags accepts either a YAML list (`tags: [a, b]`) or a scalar // (`tags: a`). Decoded via decodeTags into the Play.Tags slice. - Tags interface{} `yaml:"tags,omitempty"` + Tags interface{} `yaml:"tags,omitempty" json:"tags,omitempty"` // When is the raw expr source for the play-level conditional. Empty // means "always run". Compiled into the Play.whenProgram by GetPlays. - When string `yaml:"when,omitempty"` + When string `yaml:"when,omitempty" json:"when,omitempty"` // Inputs are the play-local input defaults. Layer above file-level // defaults but below --vars-file / CLI overrides (per-play merge // happens in GetPlays). - Inputs []Input `yaml:"inputs,omitempty"` + Inputs []Input `yaml:"inputs,omitempty" json:"inputs,omitempty"` // Tasks is the raw per-play task list, decoded into envelopes by // GetPlays via buildEnvelopesForEntry. - Tasks []map[string]interface{} `yaml:"tasks,omitempty"` + Tasks []map[string]interface{} `yaml:"tasks,omitempty" json:"tasks,omitempty"` } // Input represents an input for a task type Input struct { // Name is the name of the input - Name string `yaml:"name"` + Name string `yaml:"name" json:"name"` // Default is the default value of the input - Default string `yaml:"default"` + Default string `yaml:"default" json:"default,omitempty"` // Description is the description of the input - Description string `yaml:"description"` + Description string `yaml:"description" json:"description,omitempty"` // Required is a flag indicating if the input is required - Required bool `yaml:"required"` + Required bool `yaml:"required" json:"required,omitempty"` // Sensitive marks the input's resolved value as a secret. When true, // the value is masked as `***` anywhere it would otherwise appear in // user-facing output (apply --verbose echoes, plan output, error // messages, and the DOKKU_TRACE debug log). - Sensitive bool `yaml:"sensitive"` + Sensitive bool `yaml:"sensitive" json:"sensitive,omitempty"` // Type is the type of the input - Type string `yaml:"type"` + Type string `yaml:"type" json:"type,omitempty"` // value is the value of the input value string @@ -410,14 +444,22 @@ func GetTasks(data []byte, context map[string]interface{}) (OrderedStringEnvelop // the evaluation context (file-level only, per the spec - the play's own // inputs are not visible to its own when). func GetPlays(data []byte, context map[string]interface{}, userSet map[string]bool) ([]*Play, error) { + return GetPlaysWithFormat(data, FormatYAML, context, userSet) +} + +// GetPlaysWithFormat is the format-aware variant of GetPlays. format is +// one of "yaml" / "json5"; the empty string is treated as YAML. The +// per-format dispatch happens at every parse point inside the function +// so sigil templates render uniformly across both surfaces. +func GetPlaysWithFormat(data []byte, format string, context map[string]interface{}, userSet map[string]bool) ([]*Play, error) { baseRendered, err := renderRecipeBytes(data, context) if err != nil { return nil, err } - baseRecipe := Recipe{} - if err := yaml.Unmarshal(baseRendered, &baseRecipe); err != nil { - return nil, fmt.Errorf("unmarshal error: %v", err.Error()) + baseRecipe, err := UnmarshalRecipe(baseRendered, format) + if err != nil { + return nil, err } if len(baseRecipe) == 0 { @@ -461,7 +503,7 @@ func GetPlays(data []byte, context map[string]interface{}, userSet map[string]bo } playCtx := BuildPerPlayContext(context, play.Inputs, userSet) - perPlayRecipe, err := renderRecipe(data, playCtx) + perPlayRecipe, err := renderRecipeWithFormat(data, format, playCtx) if err != nil { return nil, err } @@ -510,15 +552,16 @@ func renderRecipeBytes(data []byte, context map[string]interface{}) ([]byte, err // renderRecipe is the convenience wrapper that renders data with context // and unmarshals the result into a Recipe. func renderRecipe(data []byte, context map[string]interface{}) (Recipe, error) { + return renderRecipeWithFormat(data, FormatYAML, context) +} + +// renderRecipeWithFormat is renderRecipe's format-aware variant. +func renderRecipeWithFormat(data []byte, format string, context map[string]interface{}) (Recipe, error) { rendered, err := renderRecipeBytes(data, context) if err != nil { return nil, err } - recipe := Recipe{} - if err := yaml.Unmarshal(rendered, &recipe); err != nil { - return nil, fmt.Errorf("unmarshal error: %v", err.Error()) - } - return recipe, nil + return UnmarshalRecipe(rendered, format) } // BuildPerPlayContext layers the play's `inputs:` defaults above the diff --git a/tasks/play_json5_test.go b/tasks/play_json5_test.go new file mode 100644 index 0000000..334d81a --- /dev/null +++ b/tasks/play_json5_test.go @@ -0,0 +1,180 @@ +package tasks + +import ( + "strings" + "testing" + + _ "github.com/gliderlabs/sigil/builtin" +) + +func TestGetPlaysWithFormatJSON5SinglePlay(t *testing.T) { + data := []byte(`[ + { + tasks: [ + { + name: "create app", + dokku_app: { + app: "test-app", + }, + }, + ], + }, +] +`) + plays, err := GetPlaysWithFormat(data, FormatNameJSON5, map[string]interface{}{}, nil) + if err != nil { + t.Fatalf("GetPlaysWithFormat: %v", err) + } + if len(plays) != 1 { + t.Fatalf("expected 1 play, got %d", len(plays)) + } + if plays[0].Name != "tasks" { + t.Errorf("auto-name = %q, want %q", plays[0].Name, "tasks") + } + if plays[0].Tasks.Get("create app") == nil { + t.Error("task 'create app' missing") + } +} + +func TestGetPlaysWithFormatJSON5MultiPlay(t *testing.T) { + data := []byte(`[ + { + name: "api", + tasks: [{ name: "create api", dokku_app: { app: "api" } }], + }, + { + name: "worker", + tasks: [{ name: "create worker", dokku_app: { app: "worker" } }], + }, +]`) + plays, err := GetPlaysWithFormat(data, FormatNameJSON5, map[string]interface{}{}, nil) + if err != nil { + t.Fatalf("GetPlaysWithFormat: %v", err) + } + if len(plays) != 2 { + t.Fatalf("expected 2 plays, got %d", len(plays)) + } + if plays[0].Name != "api" || plays[1].Name != "worker" { + t.Errorf("plays = [%q, %q]", plays[0].Name, plays[1].Name) + } +} + +func TestGetPlaysWithFormatJSON5HandlesComments(t *testing.T) { + data := []byte(`[ + // top-level comment + { + /* play preface */ + tasks: [ + // before the task + { name: "x", dokku_app: { app: "a" } }, // inline + ], + }, +]`) + plays, err := GetPlaysWithFormat(data, FormatNameJSON5, map[string]interface{}{}, nil) + if err != nil { + t.Fatalf("GetPlaysWithFormat: %v", err) + } + if len(plays) != 1 || plays[0].Tasks.Get("x") == nil { + t.Error("expected single play with task 'x'") + } +} + +func TestGetPlaysWithFormatJSON5SigilTemplate(t *testing.T) { + data := []byte(`[ + { + tasks: [ + { name: "render", dokku_app: { app: "{{ .name }}" } }, + ], + }, +]`) + plays, err := GetPlaysWithFormat(data, FormatNameJSON5, map[string]interface{}{"name": "myapp"}, nil) + if err != nil { + t.Fatalf("GetPlaysWithFormat: %v", err) + } + if len(plays) != 1 { + t.Fatalf("expected 1 play, got %d", len(plays)) + } + if plays[0].Tasks.Get("render") == nil { + t.Fatal("task 'render' missing") + } +} + +func TestGetPlaysWithFormatJSON5RejectsInvalid(t *testing.T) { + data := []byte(`{"not": "an array"}`) + _, err := GetPlaysWithFormat(data, FormatNameJSON5, map[string]interface{}{}, nil) + if err == nil { + t.Fatal("expected error for non-array root") + } +} + +func TestUnmarshalRecipeYAMLAndJSON5MatchStructurally(t *testing.T) { + yamlData := []byte(`--- +- name: api + inputs: + - name: app + default: api + tasks: + - name: create + dokku_app: + app: api +`) + jsonData := []byte(`[ + { + name: "api", + inputs: [{ name: "app", default: "api" }], + tasks: [ + { name: "create", dokku_app: { app: "api" } }, + ], + }, +]`) + yamlRecipe, err := UnmarshalRecipe(yamlData, FormatYAML) + if err != nil { + t.Fatalf("yaml UnmarshalRecipe: %v", err) + } + jsonRecipe, err := UnmarshalRecipe(jsonData, FormatNameJSON5) + if err != nil { + t.Fatalf("json5 UnmarshalRecipe: %v", err) + } + if len(yamlRecipe) != len(jsonRecipe) { + t.Fatalf("play counts differ: yaml=%d json5=%d", len(yamlRecipe), len(jsonRecipe)) + } + if yamlRecipe[0].Name != jsonRecipe[0].Name { + t.Errorf("play name: yaml=%q json5=%q", yamlRecipe[0].Name, jsonRecipe[0].Name) + } + if len(yamlRecipe[0].Inputs) != len(jsonRecipe[0].Inputs) { + t.Errorf("input count: yaml=%d json5=%d", len(yamlRecipe[0].Inputs), len(jsonRecipe[0].Inputs)) + } + if len(yamlRecipe[0].Tasks) != len(jsonRecipe[0].Tasks) { + t.Errorf("task count: yaml=%d json5=%d", len(yamlRecipe[0].Tasks), len(jsonRecipe[0].Tasks)) + } +} + +func TestValidateAcceptsJSON5(t *testing.T) { + data := []byte(`[ + { + inputs: [{ name: "app", default: "api" }], + tasks: [ + { name: "create", dokku_app: { app: "{{ .app }}" } }, + ], + }, +]`) + problems := Validate(data, ValidateOptions{Format: FormatNameJSON5}) + if len(problems) != 0 { + var msgs []string + for _, p := range problems { + msgs = append(msgs, p.Code+":"+p.Message) + } + t.Errorf("unexpected validation problems for JSON5 input: %s", strings.Join(msgs, " | ")) + } +} + +func TestValidateRejectsMalformedJSON5(t *testing.T) { + data := []byte(`[{ tasks: [`) + problems := Validate(data, ValidateOptions{Format: FormatNameJSON5}) + if len(problems) == 0 { + t.Fatal("expected at least one problem for malformed JSON5") + } + if problems[0].Code != "json5_parse" { + t.Errorf("first problem code = %q, want json5_parse", problems[0].Code) + } +} diff --git a/tasks/validate.go b/tasks/validate.go index c5e8903..14ec637 100644 --- a/tasks/validate.go +++ b/tasks/validate.go @@ -10,9 +10,22 @@ import ( sigil "github.com/gliderlabs/sigil" defaults "github.com/mcuadros/go-defaults" + json5 "github.com/titanous/json5" yaml "gopkg.in/yaml.v3" ) +// json5ToYAMLBytes parses data as JSON5 and re-emits it as YAML so the +// rest of the validator (which is yaml.v3 Node-based) can operate +// uniformly. Used at the top of Validate; sigil templates inside string +// values survive verbatim since they are just text to both parsers. +func json5ToYAMLBytes(data []byte) ([]byte, error) { + var raw interface{} + if err := json5.Unmarshal(data, &raw); err != nil { + return nil, fmt.Errorf("json5 parse error: %v", err) + } + return yaml.Marshal(raw) +} + // Problem is a single validation finding emitted by Validate. // // Code is a stable machine-readable identifier so JSON consumers (and the @@ -37,11 +50,19 @@ type Problem struct { // `validate --start-at-task` respectively. They power the cross-reference // audit that runs under `--strict`: each non-empty value must resolve to a // real play / task name in the recipe. +// +// Format selects the on-disk surface syntax: "yaml" (default) or "json5". +// JSON5 input is normalised to YAML bytes once at the top of Validate so +// the rest of the pipeline (sigil render, yaml.Node walk, line/column +// reporting) keeps a single implementation. Line / column numbers in +// problems for a JSON5 file therefore index into the normalised form, +// not the original JSON5 source. type ValidateOptions struct { Strict bool InputOverrides map[string]bool PlayName string StartAtTask string + Format string } // validatePlaceholder is substituted for any input that has no default during @@ -124,6 +145,21 @@ func Validate(data []byte, opts ValidateOptions) []Problem { return []Problem{*renderProblem} } + // JSON5 is normalised to YAML bytes once so the AST walk below stays + // yaml.v3-only. The JSON5 parse runs after the sigil syntax check so + // a broken template surfaces as a template_error rather than a + // confusing json5 parse error. + if IsJSON5Format(opts.Format) { + converted, err := json5ToYAMLBytes(data) + if err != nil { + return []Problem{{ + Code: "json5_parse", + Message: err.Error(), + }} + } + data = converted + } + var rawRoot yaml.Node if err := yaml.Unmarshal(data, &rawRoot); err != nil { line, col := parseYAMLErrorPosition(err.Error()) diff --git a/tests/bats/task_file_json.bats b/tests/bats/task_file_json.bats new file mode 100644 index 0000000..dcf4624 --- /dev/null +++ b/tests/bats/task_file_json.bats @@ -0,0 +1,188 @@ +#!/usr/bin/env bats + +load test_helper + +setup() { + docket_build +} + +@test "docket validate accepts a JSON5 tasks.json" { + write_tasks_file tasks.json <<'EOF' +[ + { + inputs: [ + { name: "app", default: "docket-test-json5" }, + ], + tasks: [ + { name: "create app", dokku_app: { app: "{{ .app }}" } }, + ], + }, +] +EOF + run "$(docket_bin)" validate --tasks "$TASKS_FILE" + assert_success + assert_output --partial "is valid" +} + +@test "docket validate accepts a JSON5 tasks.json with comments and trailing commas" { + write_tasks_file tasks.json <<'EOF' +[ + // recipe-level comment + { + /* play-level block comment */ + tasks: [ + { name: "create app", dokku_app: { app: "api", }, }, // inline comment + ], + }, +] +EOF + run "$(docket_bin)" validate --tasks "$TASKS_FILE" + assert_success + assert_output --partial "is valid" +} + +@test "docket validate reports json5_parse on malformed JSON5" { + write_tasks_file tasks.json <<'EOF' +[ + { tasks: [ +EOF + run "$(docket_bin)" validate --tasks "$TASKS_FILE" --json + assert_failure + assert_output --partial '"code":"json5_parse"' +} + +@test "docket apply --list-tasks works against tasks.json" { + write_tasks_file tasks.json <<'EOF' +[ + { + tasks: [ + { name: "first task", dokku_app: { app: "api" } }, + { name: "second task", dokku_config: { app: "api", config: { K: "v" } } }, + ], + }, +] +EOF + run "$(docket_bin)" apply --tasks "$TASKS_FILE" --list-tasks + assert_success + assert_output --partial "first task" + assert_output --partial "second task" +} + +@test "docket auto-detects tasks.json when no --tasks flag is given" { + cd "$BATS_TEST_TMPDIR" + cat >tasks.json <<'EOF' +[ + { + tasks: [ + { name: "auto-detected", dokku_app: { app: "api" } }, + ], + }, +] +EOF + run "$(docket_bin)" validate + assert_success + assert_output --partial "tasks.json" + assert_output --partial "is valid" +} + +@test "docket prefers tasks.yml over tasks.json when both exist" { + cd "$BATS_TEST_TMPDIR" + cat >tasks.yml <<'EOF' +--- +- tasks: + - name: yaml-task + dokku_app: + app: api +EOF + cat >tasks.json <<'EOF' +[ + { tasks: [{ name: "json-task", dokku_app: { app: "api" } }] }, +] +EOF + run "$(docket_bin)" apply --list-tasks + assert_success + assert_output --partial "yaml-task" + refute_output --partial "json-task" +} + +@test "docket fmt canonicalises a JSON5 tasks.json with comments preserved" { + write_tasks_file tasks.json <<'EOF' +[ + // top of recipe + { + tasks: [ + { + dokku_app: { app: "api" }, + name: "create app", // out of order + }, + ], + }, +] +EOF + run "$(docket_bin)" fmt "$TASKS_FILE" + assert_success + + formatted="$(cat "$TASKS_FILE")" + echo "$formatted" + echo "$formatted" | grep -F "// top of recipe" + echo "$formatted" | grep -F "// out of order" + + # Re-running fmt --check should now exit 0 (idempotent). + run "$(docket_bin)" fmt --check "$TASKS_FILE" + assert_success +} + +@test "docket fmt --diff prints unified diff for non-canonical JSON5" { + write_tasks_file tasks.json <<'EOF' +[ + { tasks: [{ dokku_app: { app: "api" }, name: "x" }] }, +] +EOF + run "$(docket_bin)" fmt --diff "$TASKS_FILE" + # --diff alone exits 0 even when changes are needed. + assert_success + assert_output --partial "--- $TASKS_FILE" + assert_output --partial "+++ $TASKS_FILE" +} + +@test "docket init --output tasks.json writes valid JSON5 that round-trips" { + cd "$BATS_TEST_TMPDIR" + run "$(docket_bin)" init --output tasks.json --name api --repo https://example.com/repo.git + assert_success + [ -f tasks.json ] + + run head -1 tasks.json + assert_success + assert_output "[" + + run "$(docket_bin)" validate --tasks tasks.json + assert_success + assert_output --partial "is valid" + + run "$(docket_bin)" fmt --check tasks.json + assert_success +} + +@test "docket apply --vars-file works with a JSON5 tasks file and JSON vars file" { + require_dokku + dokku_clean_app docket-test-json5-mix + + write_tasks_file tasks.json <<'EOF' +[ + { + inputs: [ + { name: "app", default: "docket-test-default" }, + ], + tasks: [ + { name: "ensure {{ .app }}", dokku_app: { app: "{{ .app }}" } }, + ], + }, +] +EOF + cat >"$BATS_TEST_TMPDIR/vars.json" <<'EOF' +{ "app": "docket-test-json5-mix" } +EOF + run "$(docket_bin)" plan --tasks "$TASKS_FILE" --vars-file "$BATS_TEST_TMPDIR/vars.json" + assert_success + assert_output --partial "ensure docket-test-json5-mix" +} diff --git a/tests/bats/test_helper.bash b/tests/bats/test_helper.bash index 8199a05..af29cbe 100644 --- a/tests/bats/test_helper.bash +++ b/tests/bats/test_helper.bash @@ -9,19 +9,34 @@ set -euo pipefail # Load bats-support / bats-assert from the standard package paths. +# BATS_LIB_PATH is a colon-separated list following the bats-core +# convention (https://bats-core.readthedocs.io/) so developers without +# the apt packages can install the libraries anywhere and point at +# them locally; CI keeps using the apt-installed /usr/lib paths. load_bats_libraries() { - local lib - for lib in /usr/lib/bats/bats-support/load.bash /usr/lib/bats-support/load.bash; do - if [ -f "$lib" ]; then - # shellcheck disable=SC1090 - source "$lib" - break + local -a search_paths=() + if [ -n "${BATS_LIB_PATH:-}" ]; then + local IFS=: + for entry in $BATS_LIB_PATH; do + search_paths+=("$entry") + done + fi + search_paths+=(/usr/lib/bats /usr/lib) + + local found_support=0 + local found_assert=0 + for base in "${search_paths[@]}"; do + if [ "$found_support" -eq 0 ] && [ -f "$base/bats-support/load.bash" ]; then + # shellcheck disable=SC1090,SC1091 + source "$base/bats-support/load.bash" + found_support=1 fi - done - for lib in /usr/lib/bats/bats-assert/load.bash /usr/lib/bats-assert/load.bash; do - if [ -f "$lib" ]; then - # shellcheck disable=SC1090 - source "$lib" + if [ "$found_assert" -eq 0 ] && [ -f "$base/bats-assert/load.bash" ]; then + # shellcheck disable=SC1090,SC1091 + source "$base/bats-assert/load.bash" + found_assert=1 + fi + if [ "$found_support" -eq 1 ] && [ "$found_assert" -eq 1 ]; then break fi done