feat(api): implement B-Tree keyset cursor pagination and atomic merge…#11
Merged
Conversation
… patch updates - Added `PaginationParams` and `PaginatedResponse` envelope types to `forge-types` - Replaced naïve enumeration indexing in `list_paginated` with true `std::ops::Bound::Excluded` keyset range queries for maximum MVCC read performance in `forge-storage` - Implemented `update_doc` exposing an atomic read-modify-write block - Wired `PaginationParams` into `GET /v1/:collection` endpoint handling - Added `PATCH /v1/:collection/:id` endpoint implementing RFC 7396 JSON Merge Patch - Resolved clippy type complexity warnings via `PaginatedList` type alias - Included suite of integration tests covering bounds conditions and null-value patching
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR: Cursor-Based Pagination (B-Tree Keyset)
Component:
crates/storage,crates/server,crates/typesBranch:
feat/paginationSummary
Implements performant, cursor-based pagination (keyset pagination) across the document API. This replaces simple "list all" behavior, allowing ForgeDB to handle massive collections without memory exhaustion or high latency.
Key Changes
list_paginatedto the redbxStorageEngine, leveraging B-Tree keyset traversal to start scans after a specific ID without skipping rows.PaginatedResponsecontainingdata: Vec<T>,next_cursor: Option<String>, andhas_more: bool.PaginationParams::resolved_limit).Technical Details
We use "Exclusive Cursors". The
cursorpassed in the query string is the ID of the last document seen; the storage engine initiates the scan from the next available key in the COW B-tree.