Feat/secondary indexes

[jonathanmagambo]

PR: Secondary Indexes & Secure Joins (with RLS Enforcement)

Component: crates/storage, crates/query, crates/server

Branch: feat/secondary-indexes

Summary

This is a major feature PR introducing secondary indexing capabilities for fast field-lookups and "Safe Joins" for relational traversals. Crucially, it also implements strict Row-Level Security (RLS) enforcement for all list and query operations.

Key Changes

• Secondary Indexing:
  • Implemented a dedicated _indexes table in redbx to store field-to-ID mappings.
  • Added high-performance MessagePack field extractor (extract_field_raw) with a 64-depth recursion guard to prevent DoS attacks.
• Safe Joins:
  • New POST /v1/_query endpoint allowing local BFS traversal of related collections.
  • Joins are executed directly on the memory-mapped B-Tree for sub-1ms latencies.
• RLS Enforcement (Critical):
  • Implemented a fetch-filter-loop in the API layer for list_docs and query_docs.
  • Every document is authorized against Cedar policies before being returned.
  • Secure cursor logic: next_cursor always points to the last scanned B-Tree document to prevent record leakage or skipping.
• DoS Mitigation: Hardened the MessagePack extractor with recursion limits to prevent stack overflow panics from malicious payloads.

Verification Results

• All 44 tests in the workspace (storage, query, and server) pass.
• Verified RLS filtering in integration tests.
• Verified DoS fix with deeply nested payloads.