-
Notifications
You must be signed in to change notification settings - Fork 0
feat: implement SQLite Adapter #5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
buibaoanh
wants to merge
62
commits into
1-no-orm-core-v1
Choose a base branch
from
no-orm-sqlite-adapter
base: 1-no-orm-core-v1
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
62 commits
Select commit
Hold shift + click to select a range
61a504f
feat: implement SQLite Adapter
buibaoanh 9dfd993
feat: support nested-path filtering for json fields
buibaoanh 7c63bc8
fix: some cleanup and bug fixes
buibaoanh 680af86
fix: add path to interface for nested-path filtering
buibaoanh b323bac
fix: address CodeRabbit comments and document limitations
buibaoanh be054ee
fix: address H comments
buibaoanh 7601d41
feat: implement postgres adapter and some refactoring
buibaoanh 5ba6c3a
fix: address upsert concerns and simplify implementation
buibaoanh 23df8f0
fix: bun typecheck
buibaoanh 2ceaaa0
refactor: update dependencies and use offical types
buibaoanh 8029d80
fix: boolean handling in postgres
buibaoanh aad2a74
docs: update README.md
buibaoanh ca8e08f
refactor: clean up some disable comments
buibaoanh 6ce209d
fix: address H comments
buibaoanh df88a92
fix: bun typecheck
buibaoanh d53e56c
fix: resolve unsafe type assertion errors
buibaoanh 11d0e02
fix: resolve unsafe type assertion errors
buibaoanh f848777
refactor: structural changes and improve test coverage
buibaoanh 713a13a
refactor: cleanup and documenting
buibaoanh 6c9328a
Merge branch 'no-orm-sqlite-adapter' of github.com:8monkey-ai/no-orm …
buibaoanh e8f05f7
refactor: No redundant abstractions and interfaces, address DRY concerns
buibaoanh f02328b
refactor: address DRY concerns
buibaoanh 88628e7
refactor: make adapter logic self-contained
buibaoanh 2cb4b23
refactor: eliminate the use of unsafe methods
buibaoanh 77b0005
refactor: strengthen type safety and reduce adapter casts
buibaoanh 8c2f53f
refactor: clean up dead code and improve type safety in adapters
buibaoanh f78a113
refactor: simplify quote logic and remove identifier caching
buibaoanh 207f170
refactor: extract shared SQL clause builders and simplify adapters
buibaoanh f9443ba
docs: clean up redundant class notes and move behavior contracts to m…
buibaoanh 4c276ab
refactor: replace manual SQL fragments with a 'sql' tagged template p…
buibaoanh e03166f
refactor: replace recursive 'where' builder with iterative stack-base…
buibaoanh 68b4731
refactor: improve naming consistency and variable clarity in adapters…
buibaoanh 756c479
fix: address H comments
buibaoanh 5d316f0
refactor: unify Where AST traversal with iterative walkWhere utility
buibaoanh 2ee8239
docs: streamline and generalize AGENTS.md architectural principles
buibaoanh 4e6a8ff
refactor: unify SQL compilation logic with Sql.compile utility
buibaoanh bac3c58
refactor: optimize SQL generation hot paths and implement strict iden…
buibaoanh c5147dd
refactor: extract build*Sql statement helpers to eliminate adapter du…
buibaoanh 2d11e5e
refactor: eliminate redundant eslint-disable comments in adapters
buibaoanh b221d4c
fix: correctness fixes for memory, postgres, sqlite adapters and SQL …
buibaoanh 36afe7c
feat: run migrate() inside a transaction for atomic DDL
buibaoanh 39bad4d
refactor: diverge sqlite executors per driver and share better-sqlite…
buibaoanh 7fbafb2
fix: restrict update() to one row, guard upsert() PK mutation, and ad…
buibaoanh f45cd07
refactor: simplify function and parameter names in statements and ada…
buibaoanh cd8f8c7
refactor: extract stringifyJsonParam helper for pg bind coercion
buibaoanh cd64d9c
refactor: consolidate statement builders into sql.ts
buibaoanh 3b6ad47
refactor: eliminate mapToRecord intermediate representation in write …
buibaoanh a63b469
refactor: extract migrateSqls helper to eliminate duplicate migrate()…
buibaoanh 21889cc
refactor: simplify adapters and memory — reuse, quality, and efficien…
buibaoanh 419c06a
fix: address CodeRabbit issues — correctness and safety fixes
buibaoanh 9518e36
fix: enforce unknown field validation in MemoryAdapter and export get…
buibaoanh f97069d
fix: address CodeRabbit issues in memory and postgres adapters
buibaoanh 8e187e7
fix: return 0 from updateMany for empty data and qualify upsert predi…
buibaoanh 86f0a7a
fix: filter undefined patch fields in MemoryAdapter and replace node:…
buibaoanh 430d990
fix: make MemoryAdapter.transaction() atomic with snapshot-based roll…
buibaoanh 4a93ec6
feat: derive row types from schema in Adapter interface
buibaoanh fa58a8a
refactor: replace Sql class with plain Fragment type in SQL builder
buibaoanh 6dba5f3
fix: resolve oxlint typecheck errors in postgres adapter and sql utils
buibaoanh 77183f3
refactor: remove transaction support from MemoryAdapter
buibaoanh 30d1b8b
fix: preserve null values for nullable boolean fields in SQLite adapter
buibaoanh 170decb
refactor: use indexed loop in assertNoUnknownFields for performance
buibaoanh c8d191a
fix: use ordered array for Cursor.after to support same-field path-di…
buibaoanh File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,62 @@ | ||
| # AGENTS.md | ||
|
|
||
| ## Purpose | ||
|
|
||
| This file defines the architectural principles and implementation standards for contributing to `@8monkey/no-orm`. | ||
|
|
||
| ## Core Principles | ||
|
|
||
| 1. **Tiny Persistence Core**: We prioritize a small footprint. Avoid adding new abstractions or runtime dependencies. | ||
| 2. **Schema-First & Static**: The schema is the source of truth. Type inference must be zero-cost. | ||
| 3. **Runtime-Agnostic**: All code must run across Bun, Node.js, Deno, and Edge environments. | ||
| 4. **Performance by Default**: We accept targeted complexity in internal hot paths (CRUD loops) to ensure the library adds minimal overhead to the underlying drivers. | ||
|
|
||
| ## Architecture | ||
|
|
||
| - **The Adapter Pattern**: Every storage engine implements the `Adapter` interface. Adapters are stateful regarding their `schema` (passed at construction). | ||
| - **Autonomous Orchestration**: Each adapter owns its specific syntax orchestration (e.g., SQL template assembly, `RETURNING`, `ON CONFLICT`). | ||
| - **Shared Atomic Logic**: Shared logic for primary keys, pagination AST, and value helpers lives in `utils/common.ts`. SQL adapters utilize `utils/sql.ts` for atomic clause generation (`where`, `set`, `sort`). | ||
| - **QueryExecutor Abstraction**: Database drivers are wrapped in a `QueryExecutor` with a uniform interface (`all`/`get`/`run`/`transaction`). This allows the same adapter logic to support multiple drivers (e.g., `pg`, `postgres.js`, `bun:sqlite`). | ||
|
|
||
| ## Implementation Standards | ||
|
|
||
| ### TypeScript & Type Safety | ||
|
|
||
| - **Use `unknown` over `any`**: All internal signatures must use `unknown` or concrete types. Use narrowing (guards/typeof) before access. | ||
| - **Justified Assertions**: Use `eslint-disable-next-line` with a short, specific reason for unavoidable type assertions (e.g., at adapter boundaries where `RowData -> T`). | ||
| - **Strict Linting**: Do not modify `.oxlintrc.json`. Fix the code to satisfy the rules. | ||
|
|
||
| ### High-Performance Internals | ||
|
|
||
| These rules apply to all internal adapter methods and shared utility loops: | ||
|
|
||
| - **No Object Spreads**: Use `Object.assign({}, ...)` instead of `{ ... }` to ensure hidden class stability and reduce transpilation overhead in hot paths. | ||
| - **Avoid `delete`**: Do not delete properties from objects (deoptimizes V8/JSC). Set to `undefined` or construct a new object. | ||
| - **Indexed Loops**: Prefer `for (let i = 0; i < arr.length; i++)` over `for...of` or `.forEach` to avoid iterator protocol overhead. | ||
| - **Sync Efficiency**: If an operation is naturally synchronous (like the `MemoryAdapter`), return `Promise.resolve(value)` instead of marking the method `async` to avoid unnecessary microtask scheduling. | ||
|
|
||
| ## Dependency Strategy | ||
|
|
||
| - **Optional Peer Dependencies**: All database drivers (e.g., `pg`, `better-sqlite3`, `lru-cache`) are optional peer dependencies. Users only install what they use. | ||
| - **Driver Portability**: Ensure that unused driver imports are never evaluated (utilize separate entrypoints or dynamic checks). | ||
| - **Type Resolution**: All peer dependencies must have corresponding types in `devDependencies` to ensure `bun run typecheck` passes. | ||
|
|
||
| ## Change Workflow | ||
|
|
||
| 1. **Minimal Edits**: Keep changes localized. Avoid broad refactors unless explicitly requested. | ||
| 2. **Order Preservation**: Do not rearrange existing classes or methods. Maintaining original order ensures clean git diffs and efficient review. | ||
| 3. **Explain the "Why"**: Retain or add comments explaining architectural choices (e.g., sequential DDL, driver detection order). | ||
| 4. **Verification**: | ||
| - Check `package.json` for current scripts. | ||
| - Run linting, type-checking, and tests (`bun test`) before finishing. | ||
| - Do not run `bun run clean` unless explicitly requested (`git clean -fdx`). | ||
|
|
||
| ## PR/Commit Checklist | ||
|
|
||
| - [ ] Change is scoped to requested behavior. | ||
| - [ ] Types compile with zero errors. | ||
| - [ ] Lint/Format passes. | ||
| - [ ] Tests pass. | ||
| - [ ] No new `any` types or unjustified `eslint-disable`. | ||
| - [ ] No object spreads or `delete` in adapter hot paths. | ||
| - [ ] README updated if public API changed. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,90 +1,239 @@ | ||
| # no-orm | ||
| # @8monkey/no-orm | ||
|
|
||
| A tiny, database-independent persistence core for TypeScript libraries. No heavy abstractions, just the primitives. | ||
| A tiny, schema-first persistence core for TypeScript libraries. | ||
|
|
||
| ## Features | ||
| `no-orm` is intentionally small: | ||
|
|
||
| - **Canonical Schema**: One portable schema representation for any database. | ||
| - **Type Inference**: Derive TypeScript types directly from your schema. | ||
| - **Adapter-Based**: Small, generic execution contract for multiple backends. | ||
| - one canonical schema shape | ||
| - inferred TypeScript model types | ||
| - adapter-based persistence | ||
| - minimal CRUD, filtering, ordering, pagination, and transactions | ||
|
|
||
| It is not a query builder, migration framework, or full ORM runtime. | ||
|
|
||
| ## Installation | ||
|
|
||
| ```bash | ||
| npm install @8monkey/no-orm | ||
| # or | ||
| bun add @8monkey/no-orm | ||
| ``` | ||
|
|
||
| ## Usage | ||
|
|
||
| ### 1. Define your Schema | ||
| ## Define a Schema | ||
|
|
||
| ```typescript | ||
| import { Schema } from "@8monkey/no-orm"; | ||
| ```ts | ||
| import type { InferModel, Schema } from "@8monkey/no-orm"; | ||
|
|
||
| export const schema = { | ||
| conversations: { | ||
| users: { | ||
| fields: { | ||
| id: { type: { type: "string", max: 255 } }, | ||
| created_at: { type: { type: "timestamp" } }, | ||
| metadata: { type: { type: "json" }, nullable: true }, | ||
| id: { type: "string" }, | ||
| name: { type: "string", max: 255 }, | ||
| age: { type: "number" }, | ||
| is_active: { type: "boolean" }, | ||
| metadata: { type: "json", nullable: true }, | ||
| tags: { type: "json[]", nullable: true }, | ||
| created_at: { type: "timestamp" }, | ||
| }, | ||
| primaryKey: { | ||
| fields: ["id"], | ||
| }, | ||
| indexes: [ | ||
| { | ||
| fields: [ | ||
| { field: "created_at", order: "desc" }, | ||
| { field: "id", order: "desc" }, | ||
| ], | ||
| }, | ||
| ], | ||
| primaryKey: "id", | ||
| indexes: [{ field: "created_at", order: "desc" }], | ||
| }, | ||
| } as const satisfies Schema; | ||
|
|
||
| type User = InferModel<typeof schema.users>; | ||
| ``` | ||
|
|
||
| ### 2. Infer Types | ||
| ## Choose an Adapter | ||
|
|
||
| ```typescript | ||
| import { InferModel } from "@8monkey/no-orm"; | ||
| ### SQLite | ||
|
|
||
| export type Conversation = InferModel<typeof schema.conversations>; | ||
| // Result: { id: string; created_at: number; metadata: Record<string, unknown> | null; } | ||
| ```ts | ||
| import { Database } from "bun:sqlite"; | ||
| import { SqliteAdapter } from "@8monkey/no-orm/adapters/sqlite"; | ||
|
|
||
| const db = new Database("data.db"); // or ":memory:" for an in-process database | ||
| const adapter = new SqliteAdapter(schema, db); | ||
|
|
||
| await adapter.migrate(); | ||
| ``` | ||
|
|
||
| ### 3. Use an Adapter | ||
| ### Postgres | ||
|
|
||
| ```ts | ||
| import postgres from "postgres"; // or import { Pool } from "pg" | ||
| import { PostgresAdapter } from "@8monkey/no-orm/adapters/postgres"; | ||
|
|
||
| ```typescript | ||
| import { Adapter } from "@8monkey/no-orm"; | ||
| // Import a concrete adapter (e.g., @8monkey/no-orm-sqlite) | ||
| const sql = postgres(process.env.POSTGRES_URL!); | ||
| const adapter = new PostgresAdapter(schema, sql); | ||
|
|
||
| const adapter: Adapter = new SqliteAdapter({ schema, db }); | ||
| await adapter.migrate(); | ||
| ``` | ||
|
|
||
| ### Memory | ||
|
|
||
| In-memory storage for testing or temporary data. | ||
|
|
||
| ```ts | ||
| import { MemoryAdapter } from "@8monkey/no-orm/adapters/memory"; | ||
|
|
||
| const adapter = new MemoryAdapter(schema, { maxItems: 100 }); | ||
| await adapter.migrate(); | ||
| ``` | ||
|
|
||
| // Minimal Schema Bootstrap | ||
| await adapter.migrate({ schema }); | ||
| ## CRUD | ||
|
|
||
| // Create a record | ||
| const conv = await adapter.create({ | ||
| model: "conversations", | ||
| ```ts | ||
| // Create | ||
| const created = await adapter.create({ | ||
| model: "users", | ||
| data: { | ||
| id: "conv_123", | ||
| created_at: Date.now(), | ||
| id: "u1", | ||
| name: "Alice", | ||
| age: 30, | ||
| is_active: true, | ||
| metadata: { theme: "dark" }, | ||
| tags: ["admin"], | ||
| created_at: Date.now(), | ||
| }, | ||
| }); | ||
|
|
||
| // Find many with filters | ||
| const results = await adapter.findMany({ | ||
| model: "conversations", | ||
| // Find one | ||
| const found = await adapter.find({ | ||
| model: "users", | ||
| where: { field: "id", op: "eq", value: "u1" }, | ||
| }); | ||
|
|
||
| // Find many | ||
| const users = await adapter.findMany({ | ||
| model: "users", | ||
| where: { field: "is_active", op: "eq", value: true }, | ||
| sortBy: [{ field: "created_at", direction: "desc" }], | ||
| limit: 20, | ||
| }); | ||
|
|
||
| // Update one | ||
| const updated = await adapter.update({ | ||
| model: "users", | ||
| where: { field: "id", op: "eq", value: "u1" }, | ||
| data: { age: 31 }, | ||
| }); | ||
|
|
||
| // Update many | ||
| const updatedCount = await adapter.updateMany({ | ||
| model: "users", | ||
| where: { field: "is_active", op: "eq", value: true }, | ||
| data: { age: 99 }, | ||
| }); | ||
|
|
||
| // Delete one | ||
| await adapter.delete({ | ||
| model: "users", | ||
| where: { field: "id", op: "eq", value: "u1" }, | ||
| }); | ||
|
|
||
| // Delete many | ||
| const deletedCount = await adapter.deleteMany({ | ||
| model: "users", | ||
| where: { field: "is_active", op: "eq", value: false }, | ||
| }); | ||
|
|
||
| // Count | ||
| const total = await adapter.count({ | ||
| model: "users", | ||
| where: { field: "is_active", op: "eq", value: true }, | ||
| }); | ||
|
|
||
| // Upsert - insert or update by primary key | ||
| const user = await adapter.upsert({ | ||
| model: "users", | ||
| create: { id: "u1", name: "Alice", age: 30, is_active: true, created_at: Date.now() }, | ||
| update: { age: 31 }, | ||
| // Optional: only update if predicate is met | ||
| where: { field: "is_active", op: "eq", value: true }, | ||
| }); | ||
| ``` | ||
|
|
||
| ## Filtering | ||
|
|
||
| All operations accept a `where` clause: | ||
|
|
||
| ```ts | ||
| // Operators | ||
| where: { field: "age", op: "eq", value: 30 } | ||
| where: { field: "age", op: "ne", value: null } | ||
| where: { field: "age", op: "gt", value: 18 } | ||
| where: { field: "age", op: "gte", value: 18 } | ||
| where: { field: "age", op: "lt", value: 65 } | ||
| where: { field: "age", op: "lte", value: 65 } | ||
| where: { field: "status", op: "in", value: ["active", "pending"] } | ||
| where: { field: "status", op: "not_in", value: ["banned"] } | ||
|
|
||
| // Combine with and/or | ||
| where: { | ||
| and: [ | ||
| { field: "age", op: "gte", value: 18 }, | ||
| { field: "is_active", op: "eq", value: true }, | ||
| ], | ||
| } | ||
| ``` | ||
|
|
||
| ## JSON Paths | ||
|
|
||
| Filter nested JSON fields using `path`: | ||
|
|
||
| ```ts | ||
| const darkUsers = await adapter.findMany({ | ||
| model: "users", | ||
| where: { | ||
| field: "created_at", | ||
| op: "gt", | ||
| value: Date.now() - 86400000, | ||
| field: "metadata", | ||
| path: ["preferences", "theme"], | ||
| op: "eq", | ||
| value: "dark", | ||
| }, | ||
| }); | ||
| ``` | ||
|
|
||
| ## Pagination | ||
|
|
||
| ```ts | ||
| // Offset pagination | ||
| const page = await adapter.findMany({ | ||
| model: "users", | ||
| sortBy: [{ field: "created_at", direction: "desc" }], | ||
| limit: 20, | ||
| offset: 40, | ||
| }); | ||
|
|
||
| // Cursor pagination (keyset) | ||
| const cursorPage = await adapter.findMany({ | ||
| model: "users", | ||
| sortBy: [{ field: "created_at", direction: "desc" }], | ||
| limit: 20, | ||
| cursor: { | ||
| after: { created_at: 1699900000000, id: "u20" }, | ||
| }, | ||
| limit: 10, | ||
| }); | ||
| ``` | ||
|
|
||
| ## Transactions | ||
|
|
||
| ```ts | ||
| await adapter.transaction(async (tx) => { | ||
| await tx.create({ | ||
| model: "users", | ||
| data: { id: "u2", name: "Bob", age: 28, is_active: true, created_at: Date.now() }, | ||
| }); | ||
|
|
||
| await tx.update({ | ||
| model: "users", | ||
| where: { field: "id", op: "eq", value: "u2" }, | ||
| data: { age: 29 }, | ||
| }); | ||
| }); | ||
| ``` | ||
|
|
||
| Nested calls to `transaction()` join the existing transaction. | ||
|
|
||
| ## License | ||
|
|
||
| MIT | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The adapter should be typed by
schema, shouldn't it? Same for the ones below.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@buibaoanh not addressed yet