Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
102a4e5
chore: ignore .worktrees/ directory
techiejd Apr 25, 2026
45b8e26
docs: add MongoDB adapter deep-dive and unified strategy docs
techiejd Apr 25, 2026
98df538
docs(spec): add MongoDB adapter spec for adapters/mongodb
techiejd Apr 25, 2026
39e0734
docs(spec): require mongodb adapter in changeset fixed array
techiejd Apr 25, 2026
dbb8357
docs(spec): add explicit root package.json scripts for mongodb (test:…
techiejd Apr 25, 2026
f1eed62
docs(spec): move test:setup/test:teardown to adapters/mongodb/package…
techiejd Apr 25, 2026
a64bdc9
docs(plan): add MongoDB adapter implementation plan
techiejd Apr 25, 2026
5697f37
feat(mongodb): scaffold adapter package skeleton
techiejd Apr 25, 2026
642a271
feat(mongodb): add escapeRegExp utility
techiejd Apr 25, 2026
5e9043f
feat(mongodb): add public types and config helpers
techiejd Apr 25, 2026
f10e09d
feat(mongodb): add lazy singleton MongoClient with test close helper
techiejd Apr 25, 2026
201f6df
feat(mongodb): convertWhereToMongo handles pre-filter leaf operators
techiejd Apr 25, 2026
d0f7ccf
feat(mongodb): convertWhereToMongo routes like/contains/all to post-f…
techiejd Apr 25, 2026
7b7b650
feat(mongodb): convertWhereToMongo handles and/or composition with pr…
techiejd Apr 25, 2026
11de118
feat(mongodb): restore explanatory comments dropped from convertWhere
techiejd Apr 25, 2026
946b0b3
feat(mongodb): add evaluatePostFilter for runtime post-filter matching
techiejd Apr 25, 2026
83343f2
feat(mongodb): ensureSearchIndex with definition-mismatch detection a…
techiejd Apr 25, 2026
47cbbfe
docs(plan): simplify Task 10 embedding coercion (remove redundant ter…
techiejd Apr 25, 2026
ddbda8b
feat(mongodb): add storeChunk that ensures index then inserts document
techiejd Apr 25, 2026
bcef1fe
fix(adapters): spread extensionFields before reserved fields in store…
techiejd Apr 25, 2026
d58e9a3
feat(mongodb): implement search via $vectorSearch with pre/post split
techiejd Apr 25, 2026
edad792
fix(mongodb): throw on missing search score instead of silently produ…
techiejd Apr 25, 2026
9d02fcb
feat(mongodb): map Payload id → Mongo _id with ObjectId casting
techiejd Apr 25, 2026
89264b6
feat(mongodb): add createMongoVectorIntegration factory and adapter w…
techiejd Apr 25, 2026
d744092
feat(mongodb): add docker-compose for local mongodb-atlas-local stack
techiejd Apr 25, 2026
1ffdcdd
fix(mongodb): ensure collection exists before createSearchIndex; add …
techiejd Apr 25, 2026
0912d18
test(mongodb): add WHERE-clause + integration suites against live Mongo
techiejd Apr 25, 2026
cb034dd
docs(plan): record Task 16 fixes — limit/numCandidates, Atlas Local v…
techiejd Apr 25, 2026
db30db4
test(mongodb): tighten Task 16 — try/finally on ad-hoc client, add le…
techiejd Apr 25, 2026
954f227
docs(mongodb): add README walking from install through Atlas + self-h…
techiejd Apr 25, 2026
a2a3070
feat(mongodb): wire mongodb adapter into root build/test scripts and …
techiejd Apr 25, 2026
c745ba5
ci(mongodb): add test_adapters_mongodb job using mongodb-atlas-local …
techiejd Apr 25, 2026
bd4c6a0
docs(plan): record Task 19 gate wiring — test_adapters_mongodb in agg…
techiejd Apr 25, 2026
91dd183
chore(changeset): add MongoDB adapter (minor)
techiejd Apr 25, 2026
240f70a
chore: gitignore adapters/mongodb/dist for parity with pg and cf
techiejd Apr 25, 2026
db295a7
fix(mongodb): drop rejected MongoClient promises from cache so next c…
techiejd Apr 26, 2026
365a0aa
test(mongodb): assert MongoClient.connect call count to gate the reje…
techiejd Apr 26, 2026
41833f2
fix(mongodb): promise-singleton ensureSearchIndex; propagate listSear…
techiejd Apr 26, 2026
45ca329
test(mongodb): mock listSearchIndexes consistently across concurrent …
techiejd Apr 26, 2026
ac1c5cc
fix(mongodb): default numCandidates to 10× limit, drop arbitrary 100 …
techiejd Apr 26, 2026
0bc7c23
refactor(mongodb): capture URI in adapter closure; drop credential fr…
techiejd Apr 26, 2026
f37c397
test(mongodb): lock URI-not-on-config invariant in compliance spec
techiejd Apr 26, 2026
5ae8fd0
test(mongodb): boot real PayloadCMS in adapter specs; remove makeFake…
techiejd Apr 26, 2026
02c2a74
chore(mongodb-tests): tighten injectDbName + drop unused import/cast
techiejd Apr 26, 2026
72297f4
test(mongodb): assert convertWhereToMongo rejects undeclared filter f…
techiejd Apr 26, 2026
e611248
test(mongodb): add extensionFields spec covering index decl + persist…
techiejd Apr 26, 2026
7b4705a
test(mongodb): multipools spec — collection+index isolation, cross-po…
techiejd Apr 26, 2026
6b3cf9e
ci: verify @payloadcms-vectorize/mongodb dist/index.d.ts is emitted
techiejd Apr 26, 2026
19b7dc7
docs(mongodb): drop experimental framing — adapter is GA on Atlas, su…
techiejd Apr 26, 2026
f60a227
docs(mongodb): drop remaining experimental wording in README lede + n…
techiejd Apr 26, 2026
072c8d5
feat(mongodb): validate limit at search entrypoint with clear error
techiejd Apr 26, 2026
9688afa
test(mongodb): cover BUILDING -> READY polling transition
techiejd Apr 26, 2026
11f8e8d
fix(mongodb): make definitionsEqual insensitive to mongot field/key o…
techiejd Apr 26, 2026
424b505
docs(mongodb): expand README — prerequisites, verify-it-works snippet…
techiejd Apr 26, 2026
b10b7ec
docs: surface mongodb adapter in root README and adapters README
techiejd Apr 26, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .changeset/add-mongodb-adapter.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
"payloadcms-vectorize": minor
"@payloadcms-vectorize/pg": minor
"@payloadcms-vectorize/cf": minor
"@payloadcms-vectorize/mongodb": minor
---

Add `@payloadcms-vectorize/mongodb` adapter (Atlas + self-hosted Community 8.2+) backed by `$vectorSearch`, with pre/post filter splitting and full WHERE-clause parity across operators (equals, not_equals, in, notIn, like, contains, gt/gte/lt/lte, exists, and/or). Search indexes are auto-ensured on first use.
2 changes: 1 addition & 1 deletion .changeset/config.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
],
"commit": false,
"fixed": [
["payloadcms-vectorize", "@payloadcms-vectorize/pg", "@payloadcms-vectorize/cf"]
["payloadcms-vectorize", "@payloadcms-vectorize/pg", "@payloadcms-vectorize/cf", "@payloadcms-vectorize/mongodb"]
],
"access": "public",
"baseBranch": "main",
Expand Down
40 changes: 39 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,42 @@ jobs:
PAYLOAD_SECRET: test-secret-key
TEST_ENV: 1

test_adapters_mongodb:
runs-on: ubuntu-latest

services:
mongodb:
image: mongodb/mongodb-atlas-local:latest
ports:
- 27018:27017
options: >-
--health-cmd "mongosh --quiet --eval 'db.runCommand({ping:1})'"
--health-interval 5s
--health-timeout 10s
--health-retries 30

steps:
- uses: actions/checkout@v4

- name: Install pnpm
uses: pnpm/action-setup@v4

- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'pnpm'

- name: Install dependencies
run: pnpm install

- name: Run mongodb adapter tests
run: pnpm test:adapters:mongodb
env:
PAYLOAD_SECRET: test-secret-key
MONGODB_URI: mongodb://localhost:27018/?directConnection=true
TEST_ENV: 1

test_e2e:
runs-on: ubuntu-latest

Expand Down Expand Up @@ -231,10 +267,11 @@ jobs:
test -f dist/index.d.ts
test -f adapters/pg/dist/index.d.ts
test -f adapters/cf/dist/index.d.ts
test -f adapters/mongodb/dist/index.d.ts

test:
runs-on: ubuntu-latest
needs: [typecheck, build, test_int, test_adapters_pg, test_adapters_cf, test_e2e]
needs: [typecheck, build, test_int, test_adapters_pg, test_adapters_cf, test_adapters_mongodb, test_e2e]
if: always()
steps:
- name: Check required jobs
Expand All @@ -244,6 +281,7 @@ jobs:
[ "${{ needs.test_int.result }}" != "success" ] || \
[ "${{ needs.test_adapters_pg.result }}" != "success" ] || \
[ "${{ needs.test_adapters_cf.result }}" != "success" ] || \
[ "${{ needs.test_adapters_mongodb.result }}" != "success" ] || \
[ "${{ needs.test_e2e.result }}" != "success" ]; then
echo "One or more required jobs failed"
exit 1
Expand Down
6 changes: 5 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ node_modules/
/dist
/adapters/pg/dist
/adapters/cf/dist
/adapters/mongodb/dist

# misc
.DS_Store
Expand Down Expand Up @@ -55,4 +56,7 @@ yarn-error.log*
*/secret

# Cursor
.cursor/
.cursor/

# Worktrees
.worktrees/
23 changes: 14 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,10 +52,11 @@ A Payload CMS plugin that adds vector search capabilities to your collections. P

This plugin requires a database adapter for vector storage. Available adapters:

| Adapter | Package | Database | Documentation |
| -------------------- | -------------------------- | --------------------------- | --------------------------------- |
| PostgreSQL | `@payloadcms-vectorize/pg` | PostgreSQL with pgvector | [README](./adapters/pg/README.md) |
| Cloudflare Vectorize | `@payloadcms-vectorize/cf` | Cloudflare Vectorize index | [README](./adapters/cf/README.md) |
| Adapter | Package | Database | Documentation |
| -------------------- | ------------------------------- | --------------------------------------- | -------------------------------------- |
| PostgreSQL | `@payloadcms-vectorize/pg` | PostgreSQL with pgvector | [README](./adapters/pg/README.md) |
| Cloudflare Vectorize | `@payloadcms-vectorize/cf` | Cloudflare Vectorize index | [README](./adapters/cf/README.md) |
| MongoDB | `@payloadcms-vectorize/mongodb` | MongoDB Atlas + self-hosted 8.2+ | [README](./adapters/mongodb/README.md) |

See [adapters/README.md](./adapters/README.md) for information on creating custom adapters.

Expand All @@ -72,8 +73,9 @@ See [adapters/README.md](./adapters/README.md) for information on creating custo
pnpm add payloadcms-vectorize

# Install a database adapter (one of the following)
pnpm add @payloadcms-vectorize/pg # PostgreSQL + pgvector
pnpm add @payloadcms-vectorize/cf # Cloudflare Vectorize
pnpm add @payloadcms-vectorize/pg # PostgreSQL + pgvector
pnpm add @payloadcms-vectorize/cf # Cloudflare Vectorize
pnpm add @payloadcms-vectorize/mongodb # MongoDB Atlas + self-hosted 8.2+
```

## Quick Start
Expand All @@ -84,6 +86,7 @@ First, configure your database adapter. See the adapter-specific documentation:

- **PostgreSQL**: [@payloadcms-vectorize/pg README](./adapters/pg/README.md) — pgvector setup, schema initialization, and migrations.
- **Cloudflare Vectorize**: [@payloadcms-vectorize/cf README](./adapters/cf/README.md) — index creation, bindings, and known limitations.
- **MongoDB**: [@payloadcms-vectorize/mongodb README](./adapters/mongodb/README.md) — Atlas / self-hosted 8.2+, `filterableFields`, and the `$vectorSearch` index lifecycle.

### 2. Configure the Plugin

Expand Down Expand Up @@ -222,6 +225,7 @@ Migration steps depend on your database adapter:

- **PostgreSQL**: [@payloadcms-vectorize/pg README → Migrations](./adapters/pg/README.md#migrations)
- **Cloudflare Vectorize**: index creation is a one-time setup step — see [@payloadcms-vectorize/cf README](./adapters/cf/README.md#1-create-vectorize-index).
- **MongoDB**: no manual migration — the `$vectorSearch` index is auto-ensured on first write. See [@payloadcms-vectorize/mongodb README → Index lifecycle](./adapters/mongodb/README.md#index-lifecycle).

### 4. Search Your Content

Expand Down Expand Up @@ -298,6 +302,7 @@ Each adapter has its own configuration shape — this is where index parameters,

- **PostgreSQL** (`dims`, `ivfflatLists`, schema initialization): [@payloadcms-vectorize/pg → Static Configuration](./adapters/pg/README.md#static-configuration)
- **Cloudflare Vectorize** (`dims`, Vectorize binding): [@payloadcms-vectorize/cf → Configuration](./adapters/cf/README.md#configuration)
- **MongoDB** (`uri`, `dbName`, per-pool `dimensions` / `similarity` / `filterableFields` / `numCandidates` / `forceExact`): [@payloadcms-vectorize/mongodb → API Reference](./adapters/mongodb/README.md#api-reference)

The embeddings collection name in Payload will be the same as the knowledge pool name.

Expand Down Expand Up @@ -374,7 +379,7 @@ You can filter on:

References to fields that don't exist on the embeddings table are silently dropped (the rest of the clause still applies).

> **Adapter parity.** All operators are implemented in `@payloadcms-vectorize/pg`. The Cloudflare Vectorize adapter has narrower native filtering — see [@payloadcms-vectorize/cf → Known Limitations](./adapters/cf/README.md#metadata-filtering) for what is and isn't supported there.
> **Adapter parity.** All operators are implemented in `@payloadcms-vectorize/pg`. The Cloudflare Vectorize adapter has narrower native filtering — see [@payloadcms-vectorize/cf → Known Limitations](./adapters/cf/README.md#metadata-filtering) for what is and isn't supported there. The MongoDB adapter splits the clause into a native `$vectorSearch` pre-filter and a JS post-filter — `like`/`contains`/`all` and any mixed-pre/post `or` are post-filtered, so they may return fewer than `limit` rows. See [@payloadcms-vectorize/mongodb → WHERE clause behavior](./adapters/mongodb/README.md#where-clause-behavior).

## Chunkers

Expand Down Expand Up @@ -1006,15 +1011,15 @@ Common scripts:
**Already shipped:**

- **Multiple Knowledge Pools** — independent configurations and embedding functions per pool.
- **Database Adapter Architecture** — pluggable backends (PostgreSQL, Cloudflare Vectorize today).
- **Database Adapter Architecture** — pluggable backends (PostgreSQL, Cloudflare Vectorize, MongoDB today).
- **More expressive queries** — configurable limits, per-collection scoping, and full Payload-style metadata filtering (see [Metadata Filtering](#metadata-filtering-where)).
- **Bulk Embed All** — admin button, provider callbacks, and run/batch tracking.
- **Serverless-friendly job model** — bulk runs are split into small, requeueable units (`prepare-bulk-embedding` and `poll-or-complete-single-batch`) so individual jobs stay well under typical serverless time limits. The `batchLimit` option (see [CollectionVectorizeOption](#collectionvectorizeoption)) lets you cap docs-per-job to fit your platform. Tested locally and on Node-style hosts; deeper Vercel-specific integration testing is on the help-wanted list.
- **Cloudflare Vectorize adapter** — `@payloadcms-vectorize/cf`.
- **MongoDB adapter** — `@payloadcms-vectorize/mongodb` (Atlas + self-hosted Community 8.2+ via `$vectorSearch`).

**Help wanted** (priority is driven by community demand — open or 👍 an issue to push something up):

- **MongoDB adapter** — `@payloadcms-vectorize/mongodb` for MongoDB Atlas Vector Search.
- **Additional adapters** — Pinecone, Qdrant, SQLite, etc. See [adapters/README.md](./adapters/README.md) for the `DbAdapter` contract.
- **Vercel CI matrix** — exercising the serverless job model end-to-end on Vercel preview deployments.

Expand Down
9 changes: 5 additions & 4 deletions adapters/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,10 +37,11 @@

## Available Adapters

| Adapter | Package | Database | Version | Status |
| -------------------- | --------------------------------------------- | -------------------------- | -------- | ----------- |
| PostgreSQL | [`@payloadcms-vectorize/pg`](./pg/README.md) | PostgreSQL with `pgvector` | `0.7.2` | Stable |
| Cloudflare Vectorize | [`@payloadcms-vectorize/cf`](./cf/README.md) | Cloudflare Vectorize index | `0.7.2` | Beta |
| Adapter | Package | Database | Version | Status |
| -------------------- | ------------------------------------------------------------- | --------------------------------- | -------- | ----------- |
| PostgreSQL | [`@payloadcms-vectorize/pg`](./pg/README.md) | PostgreSQL with `pgvector` | `0.7.2` | Stable |
| Cloudflare Vectorize | [`@payloadcms-vectorize/cf`](./cf/README.md) | Cloudflare Vectorize index | `0.7.2` | Beta |
| MongoDB | [`@payloadcms-vectorize/mongodb`](./mongodb/README.md) | MongoDB Atlas + self-hosted 8.2+ | `0.7.2` | Beta |

## Architecture

Expand Down
Loading
Loading