Added test runner

.github/workflows/test.yml

@@ -23,4 +23,4 @@ jobs:
       - name: Run tests
         run: |
           cd tests
-          ./test-cli.sh
+          ./run-tests.sh

README.md

@@ -18,10 +18,12 @@ So this could be useful for any project which wants to ship a single entrypoint
 * Formatting variables for `RESET` `RED` `GREEN` `YELLOW` `BLUE` are provided.
 * Helper functions for printing error, warning and success messages.
 * The `cli.sh` script supports being run from anywhere in the system. For example, this means it can be symlinked to `/usr/bin` and it will work.
+* A simple test runner.
 
 ## Docs
 
 [Read the CLI docs for more information on how commands works.](docs/CLI.md)
+[Read the Tests docs for more information on how to write tests.](docs/Tests.md)
 
 ## Using this template in your project
 

docs/CLI.md

@@ -4,6 +4,8 @@
 
 Create a new `.sh` script in the `commands` directory, the name of the script is the name of the command.
 
+Note that these scripts do not need to be executable, they only need to end in `.sh`.
+
 Running `cli.sh` or `cli.sh list` will then show the newly created command.
 
 When running `cli.sh` followed by the name of the command, the shell script for the command will be sourced as is. This means that commands do not need to use any specific functions to run.

docs/Tests.md

@@ -0,0 +1,55 @@
+# Tests
+
+This project includes a simple test runner for writing and running tests in Bash.
+
+## Running tests
+
+To run all tests, run `./tests/run-tests.sh`, this is known as the test runner.
+
+The test runner finds all shell scripts in `./tests` which start with `test-` and end in `.sh`. Each script is sourced and all functions starting `test_` are run.
+
+Each test file is sourced in a sub-shell, this means that variables and functions defined in test files should not leak out of the test file.
+
+## Structure of a test file
+
+A test file may contian:
+
+1. A `setup` function which is responsible for setting up the environment, this could include creating files and folders that are needed for the tests to run. If `setup` exists, the test runner will run it before running any tests, this means test functions do not need to call `setup`.
+2. One or more test functions, each test function must start with `test_`.
+3. A `teardown` function which is responsible for cleaning up the environment, if `setup` or any test functions create files, directories or make any other changes, `teardown` must undo those and restore the environment. If `teardown` exists, the test runner will run it after all tests complete, or after any tests fail, this means test functions do not need to call `teardown`. Teardown functions should try to always true.
+
+## Writing test functions
+
+Tests functions should be written as follows:
+* Test functions must be named starting with `test_`, they will be run automatically by the test runner.
+* Use the `setup` function for setting up the environment, doing things like creating files and directories used by tests.
+* Use the `teardown` function for restoring the environment, deleting anything created in the `setup` function.
+* Test functions must never call `setup` and `teardown`, the test runner will call these.
+* Test functions should return `0` if the test is successful, or `1` if the test fails.
+* Test functions must never use `exit`, they should always use `return` instead, this ensures control is properly handed back to the test runner. Tests should `return 1` for fail, and `return 0` for success.
+* The variable `$TEST_DIR` contains the absolute path to the `tests` directory, this is defined by the test runner.
+
+## Examples
+
+```Bash
+# Example setup function
+setup() {
+  echo 'echo "Test command executed"' > "$TEST_DIR/../commands/test-cmd.sh"
+}
+
+# Example test function
+test_cli_output_for_test_cmd() {
+  echo "Testing cli.sh output for our test command"
+  cd ..
+  if [[ "$(./cli.sh)" != *"test-cmd"* ]]; then
+    echo "FAIL: Test command test-cmd not found in cli.sh output."
+    return 1
+  fi
+  cd "$TEST_DIR" || { echo "Failed to cd back to tests directory"; return 1; }
+}
+
+# Example teardown function
+teardown() {
+  rm "$TEST_DIR/../commands/test-cmd.sh" || true
+}
+```

tests/run-tests.sh

@@ -0,0 +1,65 @@
+#!/usr/bin/env bash
+set -u
+
+# Run all test_ functions in test-*.sh scripts
+
+# Cd to the directory of this script, so it can be run from anywhere
+realpath="$(realpath "$0")" || { echo "Failed to get realpath"; exit 1; }
+cd "${realpath%/*}" || { echo "Failed to cd to script directory"; exit 1; }
+
+# Save the current directory if tests need a absolute path reference
+# shellcheck disable=SC2034
+TEST_DIR="$PWD"
+
+# Run a test file.
+# Uses a subshell so that each test file is isolated.
+# Usage: run_tests "test-file.sh"
+run_tests() {(
+  local test_file="$1"
+  echo -e "Running tests in $test_file...\n"
+  # shellcheck disable=SC1090
+  source "$test_file"
+
+  # As we're running in a subshell, we can use trap to run teardown and print message on exit.
+  # shellcheck disable=SC2329
+  exit_trap() {
+    # If a teardown function is defined, set it to run on EXIT
+    if [[ $(declare -F teardown) ]]; then
+      echo "Running teardown for $test_file..."
+      teardown
+    fi
+    echo -e "Finished running tests in $test_file.\n"
+  }
+  trap exit_trap EXIT
+
+  # If a setup function is defined, run it before the tests
+  if [[ $(declare -F setup) ]]; then
+    echo -e "Running setup for $test_file...\n"
+    setup
+  fi
+
+  # Discover and run all test_ functions in one loop
+  while read -r _ _ funcname; do
+    if [[ $funcname == test_* ]]; then
+      echo "Running $funcname..."
+      if ! $funcname; then
+        echo -e "FAIL: $funcname returned a non-zero exit code.\n"
+        # We are in subshell, so can exit with 1, and trap will run teardown if defined
+        exit 1
+      fi
+      echo -e "OK\n"
+    fi
+  done < <(declare -F)
+)}
+
+TEST_FILES=(test-*.sh)
+for test_file in "${TEST_FILES[@]}"; do
+  # Confirm that test_file is a file and is not empty before trying to run it
+  if [[ -f "$test_file" && -s "$test_file" ]]; then
+    if ! run_tests "$test_file"; then
+      exit 1
+    fi
+  fi
+done
+
+echo "All tests passed."

tests/test-cli-0.sh

@@ -0,0 +1,80 @@
+#!/usr/bin/env bash
+
+teardown() {
+  rm "$TEST_DIR/../commands/test-cmd.sh" || true
+  rm "$TEST_DIR/../commands/test-cmd.help.txt" || true
+  rm "$TEST_DIR/cli-symlink.sh" || true
+}
+
+setup() {
+  echo 'echo "Test command executed"' > "$TEST_DIR/../commands/test-cmd.sh"
+  echo "this is a test command" > "$TEST_DIR/../commands/test-cmd.help.txt"
+  echo "this line should not show on the command list" >> "$TEST_DIR/../commands/test-cmd.help.txt"
+  ln -s ../cli.sh cli-symlink.sh || true
+}
+
+test_cli_output_for_test_cmd() {
+  echo "Testing cli.sh output for our test command"
+  cd ..
+  if [[ "$(./cli.sh)" != *"test-cmd"* ]]; then
+    echo "FAIL: Test command test-cmd not found in cli.sh output."
+    return 1
+  fi
+  cd "$TEST_DIR" || { echo "Failed to cd back to tests directory"; return 1; }
+}
+
+test_cli_from_tests_dir() {
+  echo "Testing running cli.sh from tests directory, to proove that it can be run from anywhere"
+  if [[ "$(./../cli.sh)" != *"test-cmd"* ]]; then
+    echo "FAIL: Test command test-cmd not found when running from tests dir."
+    return 1
+  fi
+}
+
+test_cli_symlink_from_tests_dir() {
+  echo "Testing cli.sh as a symlink from tests directory, to prove that it can be run as a symlink from anywhere"
+  if [[ "$(./cli-symlink.sh)" != *"test-cmd"* ]]; then
+    echo "FAIL: Test command test-cmd not found when running cli.sh as a symlink from tests dir"
+    return 1
+  fi
+}
+
+test_cli_list_output() {
+  echo "Testing list sub-command to check output"
+  if [[ "$(./../cli.sh list)" != *"test-cmd"* ]]; then
+    echo "FAIL: Test command test-cmd not found in cli.sh list output"
+    return 1
+  fi
+}
+
+test_list_output_for_test_cmd_help() {
+  echo "Testing help text file file for test command"
+  if [[ "$(./../cli.sh list)" != *"this is a test command"* ]]; then
+    echo "FAIL: content not found in cli.sh list output"
+    return 1
+  fi
+}
+
+test_help_output_for_test_cmd() {
+  echo "Testing help output for test command"
+  if [[ "$(./../cli.sh help test-cmd)" != *"this is a test command"* ]]; then
+    echo "FAIL: content not found in cli.sh help output"
+    return 1
+  fi
+}
+
+test_list_only_shows_first_line_of_help_file() {
+  echo "Test that a second line in the help file does not show on command list"
+  if [[ "$(./../cli.sh list)" = *"this line should not show on the command list"* ]]; then
+    echo "FAIL: content not found in cli.sh list output"
+    return 1
+  fi
+}
+
+test_help_shows_all_lines_of_help_file() {
+  echo "Test that second line of help file shows when running help test-cmd"
+  if [[ "$(./../cli.sh help test-cmd)" != *"this line should not show on the command list"* ]]; then
+    echo "FAIL: content not found in cli.sh help output"
+    return 1
+  fi
+}

tests/test-cli-helper-functions.sh

@@ -0,0 +1,67 @@
+#!/usr/bin/env bash
+
+# Tests for warning, error, and success output formatting
+
+setup() {
+  # Use Bash builtin printf to write the command script
+  printf '%s\n' '#!/usr/bin/env bash' \
+    'command_exists "formatting-test" && echo "formatting-test exists"' \
+    'warning This is a warning' \
+    'error This is an error' \
+    'success This is a success' > "$TEST_DIR/../commands/formatting-test.sh"
+
+  # Run the command and store output in a variable for tests
+  TEST_FORMATTING_OUTPUT=$("$TEST_DIR/../cli.sh" formatting-test 2>&1)
+}
+
+test_command_exists() {
+  local expected
+  expected="formatting-test exists"
+  if [[ "$TEST_FORMATTING_OUTPUT" != *"$expected"* ]]; then
+    echo "Output of formatting-test command does not contian expected output."
+    echo "Expected:    $expected"
+    echo "Full output: $TEST_FORMATTING_OUTPUT"
+    return 1
+  fi
+  return 0
+}
+
+test_warning_output() {
+  local expected
+  expected="This is a warning"
+  if [[ "$TEST_FORMATTING_OUTPUT" != *"$expected"* ]]; then
+    echo "Output of formatting-test command does not contian expected output."
+    echo "Expected:    $expected"
+    echo "Full output: $TEST_FORMATTING_OUTPUT"
+    return 1
+  fi
+  return 0
+}
+
+test_error_output() {
+  local expected
+  expected="This is an error"
+  if [[ "$TEST_FORMATTING_OUTPUT" != *"$expected"* ]]; then
+    echo "Output of formatting-test command does not contian expected output."
+    echo "Expected:    $expected"
+    echo "Full output: $TEST_FORMATTING_OUTPUT"
+    return 1
+  fi
+  return 0
+}
+
+test_success_output() {
+  local expected
+  expected="This is a success"
+  if [[ "$TEST_FORMATTING_OUTPUT" != *"$expected"* ]]; then
+    echo "Output of formatting-test command does not contian expected output."
+    echo "Expected:    $expected"
+    echo "Full output: $TEST_FORMATTING_OUTPUT"
+    return 1
+  fi
+  return 0
+}
+
+teardown() {
+  rm -f "$TEST_DIR/../commands/formatting-test.sh" || true
+}