perf(webapp): avoid unindexed fileId scan in get-background-worker-by-version#4245
Conversation
…-version The endpoint loaded each file's tasks via the files.tasks relation, which queries BackgroundWorkerTask by the unindexed fileId column and sequential-scans a very large table for every request. Group task slugs by fileId in memory from the tasks that are already loaded via the indexed workerId relation, and drop the files.tasks include. The response shape is unchanged.
|
WalkthroughThe background worker version endpoint now loads files directly and builds a file ID-to-task-slug mapping from already-loaded background worker tasks. Each response file uses this mapping for its 🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
@trigger.dev/build
trigger.dev
@trigger.dev/core
@trigger.dev/python
@trigger.dev/react-hooks
@trigger.dev/redis-worker
@trigger.dev/rsc
@trigger.dev/schema-to-json
@trigger.dev/sdk
commit: |
What
The
GET /api/v1/projects/:projectRef/background-workers/:envSlug/:versionendpoint loaded each file's tasks through the nestedfiles.tasksrelation. Prisma resolves that as a separate query:BackgroundWorkerTask.fileIdis not indexed — the FK constraint exists, but Postgres does not auto-create an index for foreign keys — so on a large table this can only run as a sequential scan, which gets progressively slower as the table grows and was observed taking minutes per call in production.The loader already loads every task for the worker via
tasks: true, which uses the indexedworkerIdrelation, and those rows already includefileId. This PR groups task slugs byfileIdin memory from that already-loaded data and drops thefiles.tasksinclude entirely.Behavior change (latent bug fix)
The response shape is unchanged, but there is a semantic correction for source files reused across worker versions (files are de-duplicated by
@@unique([projectId, contentHash]), so one file row can be linked to many workers).file.taskscame from theBackgroundWorkerFile.tasksrelation, i.e. everyBackgroundWorkerTaskwith thatfileId— across all workers sharing the file. So a worker's manifest could list tasks it doesn't actually have.file.tasksis grouped from the queried worker's own tasks, so it reflects only that worker version's tasks.Verified on a local DB: 460 files are referenced by tasks from more than one worker; of 6819 (worker, file) pairs, 6 differ — all one file where the old union leaked a task slug (
cancellation-test) into worker versions that never had it. The new per-worker behavior is the correct one for a worker-version manifest. (Thanks to the automated review for flagging this.)Analysis
Captured the exact SQL before/after by instrumenting Prisma against real data (a worker with 62 files):
WHERE "fileId" IN (...)scan.fileIdquery is gone and the other four are identical.EXPLAIN of the two access paths:
No new index is required: the
workerIdaccess path is already covered by the existingBackgroundWorkerTask_workerId_slug_keyunique index.Testing
pnpm run typecheck --filter webapppasses.