Skip to content

perf(webapp): avoid unindexed fileId scan in get-background-worker-by-version#4245

Merged
d-cs merged 2 commits into
mainfrom
perf-background-worker-version-endpoint
Jul 13, 2026
Merged

perf(webapp): avoid unindexed fileId scan in get-background-worker-by-version#4245
d-cs merged 2 commits into
mainfrom
perf-background-worker-version-endpoint

Conversation

@d-cs

@d-cs d-cs commented Jul 13, 2026

Copy link
Copy Markdown
Collaborator

What

The GET /api/v1/projects/:projectRef/background-workers/:envSlug/:version endpoint loaded each file's tasks through the nested files.tasks relation. Prisma resolves that as a separate query:

SELECT id, slug, "fileId" FROM "BackgroundWorkerTask" WHERE "fileId" IN (...)

BackgroundWorkerTask.fileId is not indexed — the FK constraint exists, but Postgres does not auto-create an index for foreign keys — so on a large table this can only run as a sequential scan, which gets progressively slower as the table grows and was observed taking minutes per call in production.

The loader already loads every task for the worker via tasks: true, which uses the indexed workerId relation, and those rows already include fileId. This PR groups task slugs by fileId in memory from that already-loaded data and drops the files.tasks include entirely.

Behavior change (latent bug fix)

The response shape is unchanged, but there is a semantic correction for source files reused across worker versions (files are de-duplicated by @@unique([projectId, contentHash]), so one file row can be linked to many workers).

  • Before: file.tasks came from the BackgroundWorkerFile.tasks relation, i.e. every BackgroundWorkerTask with that fileId — across all workers sharing the file. So a worker's manifest could list tasks it doesn't actually have.
  • After: file.tasks is grouped from the queried worker's own tasks, so it reflects only that worker version's tasks.

Verified on a local DB: 460 files are referenced by tasks from more than one worker; of 6819 (worker, file) pairs, 6 differ — all one file where the old union leaked a task slug (cancellation-test) into worker versions that never had it. The new per-worker behavior is the correct one for a worker-version manifest. (Thanks to the automated review for flagging this.)

Analysis

Captured the exact SQL before/after by instrumenting Prisma against real data (a worker with 62 files):

  • Before: 5 statements, including the WHERE "fileId" IN (...) scan.
  • After: 4 statements; the fileId query is gone and the other four are identical.

EXPLAIN of the two access paths:

Before  WHERE "fileId" IN (...)
  Seq Scan on "BackgroundWorkerTask"
    Filter: ("fileId" = ANY (...))          -- reads the whole table, scales with table size

After   WHERE "workerId" IN (...)
  Index Scan using "BackgroundWorkerTask_workerId_slug_key"
    Index Cond: ("workerId" = ...)          -- bounded by matching rows, scale-independent

No new index is required: the workerId access path is already covered by the existing BackgroundWorkerTask_workerId_slug_key unique index.

Testing

  • pnpm run typecheck --filter webapp passes.
  • Query capture + EXPLAIN performed against a local database seeded with real worker/file/task data.

…-version

The endpoint loaded each file's tasks via the files.tasks relation, which
queries BackgroundWorkerTask by the unindexed fileId column and
sequential-scans a very large table for every request.

Group task slugs by fileId in memory from the tasks that are already loaded
via the indexed workerId relation, and drop the files.tasks include. The
response shape is unchanged.
@changeset-bot

changeset-bot Bot commented Jul 13, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 3a7905d

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@coderabbitai

coderabbitai Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

The background worker version endpoint now loads files directly and builds a file ID-to-task-slug mapping from already-loaded background worker tasks. Each response file uses this mapping for its tasks list, while other response data remains unchanged. A change note documents the retrieval optimization and unchanged response content.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check ⚠️ Warning The PR description is detailed, but it omits required template sections like Closes #issue, checklist, changelog, and screenshots. Add the missing template sections: Closes #issue, completed checklist, testing steps, a short changelog, and screenshots or N/A notes.
✅ Passed checks (3 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly summarizes the performance fix in the background-worker version endpoint.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch perf-background-worker-version-endpoint

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@d-cs d-cs self-assigned this Jul 13, 2026
@d-cs d-cs marked this pull request as ready for review July 13, 2026 15:02

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 potential issue.

Open in Devin Review

@pkg-pr-new

pkg-pr-new Bot commented Jul 13, 2026

Copy link
Copy Markdown

Open in StackBlitz

@trigger.dev/build

npm i https://pkg.pr.new/@trigger.dev/build@3a7905d

trigger.dev

npm i https://pkg.pr.new/trigger.dev@3a7905d

@trigger.dev/core

npm i https://pkg.pr.new/@trigger.dev/core@3a7905d

@trigger.dev/python

npm i https://pkg.pr.new/@trigger.dev/python@3a7905d

@trigger.dev/react-hooks

npm i https://pkg.pr.new/@trigger.dev/react-hooks@3a7905d

@trigger.dev/redis-worker

npm i https://pkg.pr.new/@trigger.dev/redis-worker@3a7905d

@trigger.dev/rsc

npm i https://pkg.pr.new/@trigger.dev/rsc@3a7905d

@trigger.dev/schema-to-json

npm i https://pkg.pr.new/@trigger.dev/schema-to-json@3a7905d

@trigger.dev/sdk

npm i https://pkg.pr.new/@trigger.dev/sdk@3a7905d

commit: 3a7905d

@d-cs d-cs merged commit e0b42a8 into main Jul 13, 2026
39 of 41 checks passed
@d-cs d-cs deleted the perf-background-worker-version-endpoint branch July 13, 2026 15:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants