[codex] Add Swift E2E coverage #2596
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| self-hosted-runner: | ||
| # Labels of self-hosted runner in array of strings. | ||
| labels: | ||
| - macOS | ||
| - packrat-e2e | ||
|
|
||
| # Configuration variables in array of strings defined in your repository or | ||
| # organization. `null` means disabling configuration variables check. | ||
| # Empty array means no configuration variable is allowed. | ||
| config-variables: null | ||
|
|
||
| # Configuration for file paths. The keys are glob patterns to match to file | ||
| # paths relative to the repository root. The values are the configurations for | ||
| # the file paths. Note that the path separator is always '/'. | ||
| # The following configurations are available. | ||
| # | ||
| # "ignore" is an array of regular expression patterns. Matched error messages | ||
| # are ignored. This is similar to the "-ignore" command line option. | ||
| paths: | ||
| # .github/workflows/**/*.yml: | ||
| # ignore: [] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,245 @@ | ||
| name: Swift E2E Tests | ||
|
|
||
| on: | ||
| pull_request: | ||
| branches: ["**"] | ||
| paths: | ||
| - "apps/swift/**" | ||
| - "packages/api/src/**" | ||
| - "packages/api/drizzle/**" | ||
| - "packages/api/package.json" | ||
| - "package.json" | ||
| - "bun.lock" | ||
| - ".github/workflows/swift-e2e.yml" | ||
| push: | ||
| branches: [main, development] | ||
| paths: | ||
| - "apps/swift/**" | ||
| - "packages/api/src/**" | ||
| - "packages/api/drizzle/**" | ||
| - "packages/api/package.json" | ||
| - "package.json" | ||
| - "bun.lock" | ||
| - ".github/workflows/swift-e2e.yml" | ||
| schedule: | ||
| - cron: "17 8 * * *" | ||
| workflow_dispatch: | ||
| inputs: | ||
| run_macos_ui: | ||
| description: "Run the full macOS UI suite on a self-hosted Mac runner" | ||
| required: false | ||
| type: boolean | ||
| default: true | ||
| run_ios_ui: | ||
| description: "Run the exploratory Swift iOS UI suite on a GitHub-hosted macOS runner" | ||
| required: false | ||
| type: boolean | ||
| default: false | ||
|
|
||
| concurrency: | ||
| group: ${{ github.workflow }}-${{ github.ref }} | ||
| cancel-in-progress: true | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| env: | ||
| XCODE_VERSION: "26.2" | ||
| E2E_API_BASE_URL: ${{ secrets.SWIFT_E2E_API_BASE_URL }} | ||
| E2E_EMAIL: ${{ secrets.E2E_TEST_EMAIL }} | ||
| E2E_PASSWORD: ${{ secrets.E2E_TEST_PASSWORD }} | ||
| E2E_SCREENSHOT_DIR: ${{ github.workspace }}/apps/swift/TestResults/screenshots | ||
| PACKRAT_NATIVEWIND_UI_GITHUB_TOKEN: ${{ secrets.PACKRAT_NATIVEWIND_UI_GITHUB_TOKEN }} | ||
|
|
||
| jobs: | ||
| macos-ui: | ||
| name: macOS Swift UI E2E | ||
| runs-on: [self-hosted, macOS, packrat-e2e] | ||
| timeout-minutes: 45 | ||
| if: > | ||
| github.event_name == 'schedule' || | ||
| github.event_name == 'push' || | ||
| (github.event_name == 'workflow_dispatch' && inputs.run_macos_ui) | ||
|
|
||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v6 | ||
|
Comment on lines
+65
to
+66
Contributor
There was a problem hiding this comment. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
file=".github/workflows/swift-e2e.yml"
echo "== Unpinned uses (not full SHA) =="
rg -n '^\s*uses:\s*\S+' "$file" | rg -v '@[0-9a-fA-F]{40}$' || true
echo
echo "== Checkout steps missing persist-credentials: false =="
python - <<'PY'
import re, pathlib
p = pathlib.Path(".github/workflows/swift-e2e.yml")
text = p.read_text()
blocks = re.finditer(r'(?ms)^\s*-\s+name:.*?(?=^\s*-\s+name:|\Z)', text)
for m in blocks:
block = m.group(0)
if "uses: actions/checkout@" in block and "persist-credentials: false" not in block:
line = text[:m.start()].count("\n") + 1
print(f"Checkout step near line {line} is missing persist-credentials: false")
PYRepository: PackRat-AI/PackRat Length of output: 800 Pin third-party actions to full SHAs and disable persisted checkout credentials. All action references use mutable tags (v6, v1, v2, v7). Both checkout steps keep credentials in git config. This weakens CI supply-chain and token-hardening posture. Suggested patch pattern- - name: Checkout repository
- uses: actions/checkout@v6
+ - name: Checkout repository
+ uses: actions/checkout@<FULL_40_CHAR_COMMIT_SHA>
+ with:
+ persist-credentials: false
- - name: Setup Xcode
- uses: maxim-lobanov/setup-xcode@v1
+ - name: Setup Xcode
+ uses: maxim-lobanov/setup-xcode@<FULL_40_CHAR_COMMIT_SHA>
- - name: Setup Bun
- uses: oven-sh/setup-bun@v2
+ - name: Setup Bun
+ uses: oven-sh/setup-bun@<FULL_40_CHAR_COMMIT_SHA>
- - name: Upload macOS xcresult
- uses: actions/upload-artifact@v7
+ - name: Upload macOS xcresult
+ uses: actions/upload-artifact@<FULL_40_CHAR_COMMIT_SHA>Affects lines: 66–67, 70, 75, 133, 141, 150, 166–167, 170, 175, 224, 232, 241. 🧰 Tools🪛 zizmor (1.25.2)[warning] 66-67: credential persistence through GitHub Actions artifacts (artipacked): does not set persist-credentials: false (artipacked) [error] 67-67: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy) (unpinned-uses) 🤖 Prompt for AI AgentsSources: Coding guidelines, Linters/SAST tools |
||
|
|
||
| - name: Setup Xcode | ||
| uses: maxim-lobanov/setup-xcode@v1 | ||
| with: | ||
| xcode-version: ${{ env.XCODE_VERSION }} | ||
|
|
||
| - name: Setup Bun | ||
| uses: oven-sh/setup-bun@v2 | ||
| with: | ||
| bun-version: latest | ||
| cache: true | ||
|
|
||
| - name: Install dependencies | ||
| run: bun install --frozen-lockfile | ||
|
|
||
| - name: Verify Swift E2E secrets | ||
| run: | | ||
| missing=() | ||
| [ -z "${E2E_EMAIL:-}" ] && missing+=("E2E_TEST_EMAIL") | ||
| [ -z "${E2E_PASSWORD:-}" ] && missing+=("E2E_TEST_PASSWORD") | ||
| [ -z "${E2E_API_BASE_URL:-}" ] && missing+=("SWIFT_E2E_API_BASE_URL") | ||
| [ -z "${NEON_DATABASE_URL:-}" ] && missing+=("NEON_DEV_DATABASE_URL") | ||
| if [ ${#missing[@]} -gt 0 ]; then | ||
| echo "::error::Required Swift E2E secrets missing: ${missing[*]}" | ||
| exit 1 | ||
| fi | ||
| env: | ||
| NEON_DATABASE_URL: ${{ secrets.NEON_DEV_DATABASE_URL }} | ||
|
Comment on lines
+82
to
+94
Contributor
There was a problem hiding this comment. Fail fast on all required Swift E2E secrets. The workflow validates several secrets but skips Suggested fix - name: Verify Swift E2E secrets
run: |
missing=()
[ -z "${E2E_EMAIL:-}" ] && missing+=("E2E_TEST_EMAIL")
[ -z "${E2E_PASSWORD:-}" ] && missing+=("E2E_TEST_PASSWORD")
[ -z "${E2E_API_BASE_URL:-}" ] && missing+=("SWIFT_E2E_API_BASE_URL")
[ -z "${NEON_DATABASE_URL:-}" ] && missing+=("NEON_DEV_DATABASE_URL")
+ [ -z "${PACKRAT_NATIVEWIND_UI_GITHUB_TOKEN:-}" ] && missing+=("PACKRAT_NATIVEWIND_UI_GITHUB_TOKEN")
if [ ${`#missing`[@]} -gt 0 ]; then
echo "::error::Required Swift E2E secrets missing: ${missing[*]}"
exit 1
fiAlso applies to: 183-195 🤖 Prompt for AI Agents |
||
|
|
||
| - name: Check Automation Mode status | ||
| run: | | ||
| automationmodetool status || true | ||
|
|
||
| - name: Generate Swift Xcode project | ||
| run: bun run swift | ||
|
|
||
| - name: Seed E2E test user | ||
| run: bun run --filter @packrat/api db:seed:e2e-user | ||
| env: | ||
| NEON_DATABASE_URL: ${{ secrets.NEON_DEV_DATABASE_URL }} | ||
| E2E_TEST_EMAIL: ${{ env.E2E_EMAIL }} | ||
| E2E_TEST_PASSWORD: ${{ env.E2E_PASSWORD }} | ||
|
|
||
| - name: Run macOS Swift UI E2E | ||
| run: | | ||
| if [ "${{ github.event_name }}" = "pull_request" ]; then | ||
| caffeinate -dimsu bun run e2e:swift:mac-smoke | ||
| else | ||
| caffeinate -dimsu bun run e2e:swift:mac-ui | ||
| fi | ||
|
|
||
| - name: Summarize macOS xcresult | ||
| if: always() | ||
| run: | | ||
| result="$(find apps/swift/TestResults -maxdepth 1 -name '*.xcresult' -type d | sort | tail -1)" | ||
| if [ -z "$result" ]; then | ||
| echo "No xcresult bundle found." | ||
| exit 0 | ||
| fi | ||
| echo "### macOS Swift UI E2E" >> "$GITHUB_STEP_SUMMARY" | ||
| echo "\`$result\`" >> "$GITHUB_STEP_SUMMARY" | ||
| xcrun xcresulttool get test-results summary --path "$result" | tee -a "$GITHUB_STEP_SUMMARY" | ||
|
|
||
| - name: Upload macOS xcresult | ||
| if: always() | ||
| uses: actions/upload-artifact@v7 | ||
| with: | ||
| name: swift-macos-ui-xcresult | ||
| path: apps/swift/TestResults/*.xcresult | ||
| retention-days: 14 | ||
|
|
||
| - name: Upload macOS screenshots | ||
| if: always() | ||
| uses: actions/upload-artifact@v7 | ||
| with: | ||
| name: swift-macos-ui-screenshots | ||
| path: apps/swift/TestResults/screenshots/ | ||
| if-no-files-found: ignore | ||
| retention-days: 14 | ||
|
|
||
| - name: Upload macOS failure triage bundle | ||
| if: failure() | ||
| uses: actions/upload-artifact@v7 | ||
| with: | ||
| name: swift-macos-ui-failure-triage | ||
| path: apps/swift/TestResults/ | ||
| if-no-files-found: ignore | ||
| retention-days: 14 | ||
|
|
||
| ios-ui: | ||
| name: iOS Swift UI E2E (Exploratory) | ||
| runs-on: macos-15 | ||
| timeout-minutes: 60 | ||
| if: > | ||
| github.event_name == 'schedule' || | ||
| (github.event_name == 'workflow_dispatch' && inputs.run_ios_ui) | ||
|
|
||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v6 | ||
|
|
||
| - name: Setup Xcode | ||
| uses: maxim-lobanov/setup-xcode@v1 | ||
| with: | ||
| xcode-version: ${{ env.XCODE_VERSION }} | ||
|
|
||
| - name: Setup Bun | ||
| uses: oven-sh/setup-bun@v2 | ||
| with: | ||
| bun-version: latest | ||
| cache: true | ||
|
|
||
| - name: Install dependencies | ||
| run: bun install --frozen-lockfile | ||
|
|
||
| - name: Verify Swift E2E secrets | ||
| run: | | ||
| missing=() | ||
| [ -z "${E2E_EMAIL:-}" ] && missing+=("E2E_TEST_EMAIL") | ||
| [ -z "${E2E_PASSWORD:-}" ] && missing+=("E2E_TEST_PASSWORD") | ||
| [ -z "${E2E_API_BASE_URL:-}" ] && missing+=("SWIFT_E2E_API_BASE_URL") | ||
| [ -z "${NEON_DATABASE_URL:-}" ] && missing+=("NEON_DEV_DATABASE_URL") | ||
| if [ ${#missing[@]} -gt 0 ]; then | ||
| echo "::error::Required Swift E2E secrets missing: ${missing[*]}" | ||
| exit 1 | ||
| fi | ||
| env: | ||
| NEON_DATABASE_URL: ${{ secrets.NEON_DEV_DATABASE_URL }} | ||
|
|
||
| - name: Generate Swift Xcode project | ||
| run: bun run swift | ||
|
|
||
| - name: Seed E2E test user | ||
| run: bun run --filter @packrat/api db:seed:e2e-user | ||
| env: | ||
| NEON_DATABASE_URL: ${{ secrets.NEON_DEV_DATABASE_URL }} | ||
| E2E_TEST_EMAIL: ${{ env.E2E_EMAIL }} | ||
| E2E_TEST_PASSWORD: ${{ env.E2E_PASSWORD }} | ||
|
|
||
| - name: Run iOS Swift UI E2E | ||
| run: bun run e2e:swift:ios | ||
|
|
||
| - name: Summarize iOS xcresult | ||
| if: always() | ||
| run: | | ||
| result="$(find apps/swift/TestResults -maxdepth 1 -name '*.xcresult' -type d | sort | tail -1)" | ||
| if [ -z "$result" ]; then | ||
| echo "No xcresult bundle found." | ||
| exit 0 | ||
| fi | ||
| echo "### iOS Swift UI E2E" >> "$GITHUB_STEP_SUMMARY" | ||
| echo "\`$result\`" >> "$GITHUB_STEP_SUMMARY" | ||
| xcrun xcresulttool get test-results summary --path "$result" | tee -a "$GITHUB_STEP_SUMMARY" | ||
|
|
||
| - name: Upload iOS xcresult | ||
| if: always() | ||
| uses: actions/upload-artifact@v7 | ||
| with: | ||
| name: swift-ios-ui-xcresult | ||
| path: apps/swift/TestResults/*.xcresult | ||
| retention-days: 14 | ||
|
|
||
| - name: Upload iOS screenshots | ||
| if: always() | ||
| uses: actions/upload-artifact@v7 | ||
| with: | ||
| name: swift-ios-ui-screenshots | ||
| path: apps/swift/TestResults/screenshots/ | ||
| if-no-files-found: ignore | ||
| retention-days: 14 | ||
|
|
||
| - name: Upload iOS failure triage bundle | ||
| if: failure() | ||
| uses: actions/upload-artifact@v7 | ||
| with: | ||
| name: swift-ios-ui-failure-triage | ||
| path: apps/swift/TestResults/ | ||
| if-no-files-found: ignore | ||
| retention-days: 14 | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,105 @@ | ||
| # PackRat Swift Testing | ||
|
|
||
| The generated Xcode project is not committed. Regenerate it after changing | ||
| `project.yml`: | ||
|
|
||
| ```sh | ||
| bun swift | ||
| ``` | ||
|
|
||
| If Xcode or SwiftPM reports a temporary-directory error on this machine, ensure | ||
| the configured temp directory exists: | ||
|
|
||
| ```sh | ||
| mkdir -p /Volumes/CrucialX10/tmp/andrewbierman | ||
| ``` | ||
|
Comment on lines
+10
to
+15
Contributor
There was a problem hiding this comment. Generalize the temp-directory workaround instruction. The command uses a machine/user-specific absolute path, which is not portable for other contributors or runners. Suggested wording update-If Xcode or SwiftPM reports a temporary-directory error on this machine, ensure
-the configured temp directory exists:
+If Xcode or SwiftPM reports a temporary-directory error, ensure your configured
+temporary directory exists (for example, `TMPDIR`):
```sh
-mkdir -p /Volumes/CrucialX10/tmp/andrewbierman
+mkdir -p "${TMPDIR:-/tmp}"🤖 Prompt for AI Agents |
||
|
|
||
| ## Commands | ||
|
|
||
| ```sh | ||
| bun run test:swift:runner | ||
| bun run test:swift:unit | ||
| bun run e2e:swift:ios-smoke | ||
| bun run e2e:swift:ios | ||
| bun run e2e:swift:mac | ||
| bun run e2e:swift:mac-smoke | ||
| bun run e2e:swift:mac-ui | ||
| ``` | ||
|
|
||
| `e2e:swift` defaults to iOS UI tests for compatibility with the original | ||
| runner. All Xcode result bundles are written under `apps/swift/TestResults/`. | ||
|
|
||
| Smoke modes are intentionally small PR gates: | ||
|
|
||
| - `e2e:swift:mac-smoke`: macOS login, sidebar navigation, and pack create/add-item. | ||
| - `e2e:swift:ios-smoke`: iOS login, tab navigation, and pack create. | ||
|
|
||
| Full modes are the platform confidence gates: | ||
|
|
||
| - `e2e:swift:mac-ui`: full native macOS app UI suite. | ||
| - `e2e:swift:ios`: exploratory native Swift iOS app UI suite. This is separate | ||
| from the existing Expo iOS app, which remains covered by Maestro. | ||
|
|
||
| UI modes require credentials in the process environment or `.env.local`: | ||
|
|
||
| ```sh | ||
| E2E_EMAIL=... | ||
| E2E_PASSWORD=... | ||
| ``` | ||
|
|
||
| The runner also accepts `E2E_TEST_EMAIL` and `E2E_TEST_PASSWORD`, then forwards | ||
| them to XCTest as `E2E_EMAIL` and `E2E_PASSWORD`. Credential values are not | ||
| printed by the runner. | ||
|
|
||
| Set `E2E_API_BASE_URL` to point UI tests at a specific API worker without | ||
| changing the app's saved preferences: | ||
|
|
||
| ```sh | ||
| E2E_API_BASE_URL=http://localhost:8788 | ||
| ``` | ||
|
|
||
| ## CI | ||
|
|
||
| Swift E2E CI is defined in `.github/workflows/swift-e2e.yml`. | ||
|
|
||
| - Pull requests run the macOS smoke subset on a self-hosted Mac runner. | ||
| - Pushes, scheduled runs, and manual macOS runs execute the full macOS suite. | ||
| - Swift iOS runs nightly or manually and is labeled exploratory while the Expo | ||
| app remains the production iOS app. | ||
| - Each CI run uploads `.xcresult` bundles, screenshots, failure triage artifacts, | ||
| and a GitHub step summary generated with `xcresulttool`. | ||
|
|
||
| See `docs/ci/swift-e2e-runner.md` for self-hosted Mac runner setup. | ||
|
|
||
| ## Data Isolation | ||
|
|
||
| Swift E2E tests use unique names for records they create. That keeps repeated | ||
| runs safe against shared account state, but it does not fully clean historical | ||
| test data from the backend. If the shared E2E account starts accumulating enough | ||
| data to affect performance or assertions, add API-backed cleanup helpers or a | ||
| test-only reset endpoint and call it from the runner before/after UI modes. | ||
|
|
||
| ## Signing | ||
|
|
||
| `e2e:swift:mac` passes `CODE_SIGNING_ALLOWED=NO` so the local compile gate can | ||
| run without provisioning. | ||
|
|
||
| `e2e:swift:mac-ui` must still be signed because XCTest launches a runner app, | ||
| but the runner uses Xcode's local ad-hoc identity (`Sign to Run Locally`) so | ||
| smoke tests do not block on private-key prompts. | ||
|
|
||
| Normal signed builds use automatic signing with team `666HGMV2LU`. If command- | ||
| line signing fails with `errSecInternalComponent`, the certificate is installed | ||
| but `codesign` cannot access the private key from the login keychain. Unlock the | ||
| keychain and allow Apple tooling to use the key before rerunning: | ||
|
|
||
| ```sh | ||
| security unlock-keychain ~/Library/Keychains/login.keychain-db | ||
| security set-key-partition-list -S apple-tool:,apple: -s ~/Library/Keychains/login.keychain-db | ||
| ``` | ||
|
|
||
| ## Worktree Hygiene | ||
|
|
||
| The Swift branch is active and may move while multiple agents are working. | ||
| Fetch before editing shared Swift files, then compare against | ||
| `origin/claude/swift-mac-app-effort-tTGd7` before final verification. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Include
pull_requestin the macOS job gate.Line 62 excludes
pull_request, so the PR smoke path at Lines 112-114 is unreachable and PR macOS E2E coverage is skipped.Suggested fix
if: > + github.event_name == 'pull_request' || github.event_name == 'schedule' || github.event_name == 'push' || (github.event_name == 'workflow_dispatch' && inputs.run_macos_ui)🤖 Prompt for AI Agents