feat-015: read-only graph query surface (ckg query --graph / ckg_query)#156
Merged
Conversation
Graduate the read-only graph query surface (FA-003) into feat-015, targeting 0.6.4. Add the feature spec and the accepted design doc; register both in the trackers. The design resolves the caller-facing surface (Cypher-subset text) and the internal trust boundary (validated AST -> per-backend compile). Hardened for extensibility per review: polymorphic Compiler classes + singledispatch (not a dialect flag), capability tiers (not intersection-forever), a shared bounded-cursor making max_expansions real+tested on every backend (not approximated), a capability-driven tool registry, and a reusable cli formatter. Scope: all three backends (Kuzu + Neo4j + SurrealDB) query-capable at ship. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
First chunk of the read-only graph query surface: the framework-free store/query/ package that turns caller text into a validated AST. No backend execution yet (chunk 2). - GRAMMAR.md: versioned EBNF (v1.0) of the accepted Cypher subset; the source of truth the parser mirrors. Write verbs / CALL / WITH have no production. - ast.py: frozen query AST (the single trust boundary). - parser.py: hand-written tokenizer + recursive-descent parser, no new dependency. Syntax only; labels/kinds kept as raw strings. - validator.py: two-phase gate — (1) vocabulary (labels in NodeKind, rel types in EdgeKind, curated properties or attrs.*, bound-variable + Cartesian-product checks) and (2) capability tiers against the target backend. - schema.py: curated node-property catalogue mapped to the shared _rowmap columns (f.path -> sym_path, span -> span_start/end, provenance columns), so properties resolve identically on every backend; describe_schema() for --schema. QUERY_LANG_VERSION reported by ckg status (chunk 7). - capability.py: capability tiers, the optional QueryCapable protocol, and QuerySettings / ResultTable used by execution (chunk 2). - Extend the ADR-0001 layering test to the new subpackage. Resolved: a comparison's RHS is a parameterized literal (no property-to-property compare), so WHERE cannot join disconnected patterns — multi-pattern MATCH must be connected by shared variables. 72 unit tests. Full gate green: 1023 passed, 94.62% coverage; ruff + mypy --strict clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The first end-to-end vertical slice of the read-only query surface: a validated
AST now compiles to openCypher and runs against the default (Kuzu) backend under
enforced bounds, returning a normalized columnar result.
- compile_base.py: Compiler ABC + CompiledQuery + ParamAllocator. A class per
dialect, not a dialect flag — a new dialect is a subclass, a new construct is
one added emit-arm.
- compile_cypher.py: CypherCompiler (+ Kuzu/Neo4j subclasses) over the single
generic-table schema — (f:Function) -> (f:CkgNode {kind:'Function'}), a logical
property -> its physical column (f.path -> f.sym_path) via the curated
catalogue, all literals parameterized. Verified against a real embedded Kuzu.
- execute.py: the shared bounded driver. pull_bounded is pure and clock-injected
(row cap + expansion backstop + soft timeout -> partial result); run_bounded
runs the DB work on one worker thread with a hard wait_for backstop. Bounds are
a real, tested guarantee on every backend, not approximated where hard.
- KuzuGraphStore.run_query + QueryCapable class attrs (dialect=kuzu,
caps=core+agg.collect, read_only_execution=False — the AST gate is the
read-only guarantee for the embedded backend).
- Store.query_graph / query_enabled / query_capabilities; CodeGraph.query_graph
/ describe_schema / query_enabled / query_capabilities.
- conformance.py: QueryConformance — a 3-part contract (result parity, bounded
execution, read-only) every query-capable backend must pass; Kuzu subclass is
the always-on CI gate.
attrs.* is gated behind an optional attrs.access capability that no v1 backend
advertises (Kuzu/Neo4j store attrs as a JSON string, which native Cypher cannot
destructure without a workaround that breaks under aggregation) — an honest
deferral through the capability seam. The curated columns cover the real
structural-query use cases.
Deviations (for the spec at merge): the compiler visitor uses match statements,
not functools.singledispatchmethod (mypy-clean, same additive property);
QueryConformance lives in store/query/, not core/ (core must not import store).
Gate: 1050 passed, 45 skipped, 94.68% coverage; ruff + mypy --strict clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The power-user surface for the read-only query engine. The natural-language
retrieval path (ckg query "<text>") is unchanged; --graph selects the structural
surface.
- cli_format.py: a reusable render_table / render_json helper (introduced here,
adoptable by other verbs later — not inline in the handler).
- query subcommand: --graph '<cypher-subset>' runs a structural query; --schema
prints the queryable vocabulary (node/edge kinds + properties); --format
{table,json}; --limit caps rows (clamped to the server max). QueryError /
QueryDisabled -> stderr + exit 2 (the structured "why", not a traceback).
Verified live: ckg index a repo, then ckg query --graph 'MATCH (c:Class) RETURN
c.name' -> aligned table; --format json -> columns/rows + truncation envelope;
--schema -> vocabulary.
Gate: full suite green (1064 passed, 94.72% cov); ruff + mypy --strict clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The agent surface for the read-only query engine, plus the honest tool-gating
seam.
- CkgQuery tool (QueryInput {query, limit}); engine.query_graph folds the
staleness envelope + tool_api_version + query_lang_version and returns a
STRUCTURED error rather than raising into the tool layer. ckg_status now
reports query.{enabled, lang_version, capabilities}.
- Capability-driven registration (not always-register-then-error): _tools_for
filters ALL_TOOLS by `tool.requires <= backend capabilities`, so ckg_query is
present exactly when the backend is query-capable and cleanly absent otherwise.
_store_capabilities checks the graph DRIVER CLASS for query_dialect (no index
opened); federation offers ckg_query only when every member is query-capable.
This is a reusable gate for future capability-scoped tools.
- test_schemas is now profile-parameterized: the enabled profile includes
ckg_query, the disabled profile excludes it — drift on either path fails CI.
Deviation (for the spec at merge): ckg_query v1 has no `params` argument (the
design's open-Q1 leaned yes). The grammar inlines and parameterizes literals
internally and exposes no $placeholder syntax, so a caller-facing params map has
no binding site in v1 — honest to omit it.
Gate: 1064 passed, 45 skipped, 94.72% coverage; ruff + mypy --strict clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The second query-capable backend, proving the Cypher compiler is portable. - execute.py: pull_bounded_async + run_bounded_async — the async twin of the bounded pull, for backends whose driver is async (Neo4j, and SurrealDB next). Same row/expansion/timeout policy; hard wait_for backstop -> GuardrailError. - Neo4jGraphStore.run_query: compiles via Neo4jCypherCompiler (the shared Cypher body — Neo4j uses the identical single-table schema, so no dialect deltas were needed) and executes inside a genuine READ transaction (execute_read), so read-only is enforced by the session (gate #2) on top of the AST gate (gate #1). QueryCapable: dialect=neo4j, caps=core+agg.collect, read_only_execution=True. - Neo4j QueryConformance subclass (env-gated on CKG_NEO4J_URI). Verified against a live Neo4j 5 container: all 11 conformance tests pass — the same query set returns the same rows as Kuzu (result parity), the runaway query trips the row/expansion caps, and writes are rejected. Two of three backends now proven result-identical. Unit gate green; ruff + mypy --strict clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The third query-capable backend — and a universal query path. SurrealDB models edges as a document table (src/dst fields) with no native graph traversal to compile to, so rather than a fragile SurrealQL translator it *interprets* the query. - interpret.py: InterpretingQueryEngine evaluates a validated QueryAst using only the locked GraphStore ABC read methods (query/adjacent/get). This makes query support universal — any GraphStore is query-capable for free — and it is inherently read-only (no write path through the ABC). Same row/expansion/ timeout bounds as the compiled path, and it passes the SAME QueryConformance suite, so results are identical to the canonical rows. "Compile where the backend has a native query language (Kuzu/Neo4j); interpret elsewhere." - SurrealGraphStore.run_query delegates to the interpreter (dialect=interpreted, read_only_execution=True — the interpreter cannot write). - Interpreter coverage without a server: test_interpret_conformance runs it over embedded Kuzu (always-on CI gate), plus test_interpret unit tests for the branches the parity suite doesn't hit (avg/min/max/collect, DISTINCT, ORDER BY a non-projected property, attrs access, timeout backstop, node_property). - Live: SurrealDB QueryConformance (env-gated) — 12 tests pass against a real SurrealDB 2 container; identical rows to Kuzu and Neo4j. Parser bugfix (shared, both paths): CONTAINS is both the string operator and an EdgeKind, so the tokenizer made it a keyword and [:CONTAINS] failed to parse. Added _name() so labels/relationship types accept reserved words; locked across all three backends with a File-[:CONTAINS]->Class conformance test. All three backends — Kuzu (compiled), Neo4j (compiled), SurrealDB (interpreted) — now return identical rows. Gate: 1084 passed, 94.33% coverage; ruff + mypy clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Final chunk — make the query surface configurable and documented. - QueryConfig(_Block, KEY="query"): enabled / max_rows / timeout_ms / max_expansions / allow_in_mcp, with to_settings(limit). Auto-discovered by block_keys(); read framework-free (ADR-0001). - Wired through the surfaces: the CLI (`_query_graph`) and engine (`query_graph`) build QuerySettings from the block and refuse when enabled=false; the serve capability gate offers ckg_query only when the backend is query-capable AND enabled AND allow_in_mcp. - Docs: guide 13 (ad-hoc structural queries) + guides index; README tool list now notes ckg_query; CHANGELOG [Unreleased] entry; feat-015 Implementation- status section recording the (deliberate, documented) deviations — chiefly the portable AST interpreter for SurrealDB in place of a native SurrealQL compiler. feat-015 is now complete across all 7 chunks. Gate: 1091 passed, 69 skipped, 94.33% coverage; ruff + mypy --strict clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…UG-CARRIED feat-015] Surfaced during feat-015 end-to-end validation: `class KuzuGraphStore(GraphStore)` produced no INHERITS edge. `GraphStore` is imported `from agentforge_graph.core import GraphStore` but *defined* in `core.contracts` and only re-exported by `core/__init__.py` — so it was absent from the package module's def-`exports`, the base stayed unresolved, and no edge was emitted. Fix: compute a per-package `reexports` namespace from each `pkg/__init__.py`'s own `from .sub import Name` imports (resolved against `exports` via a fixpoint, so a re-export chained through several `__init__` files still resolves), and consult it when binding `from pkg import Name`. So an absolute import of a re-exported symbol links to its real definition. On this project's own core+store package: INHERITS 11→17 (all GraphStore / VectorStore subclasses now link to their ABC), CALLS 373→395 (functions imported via a package re-export and called by name now resolve too). Improves INHERITS/CALLS recall for every consumer, not only the query surface. 2 new tests (absolute + relative re-export). Gate: 1093 passed, 94.34% coverage; ruff + mypy --strict clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Formatting only — the feat-015 edits to CodeGraph.query_graph and the config tests weren't run through `ruff format`. No behaviour change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
neo4j's session.execute_read returns Any; assign to a typed local so mypy 2.1.0's no-any-return is satisfied. No behaviour change. (A stale local mypy incremental cache masked this; a --no-incremental run reproduces CI.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Graduates FA-003 into feat-015 (target 0.6.4): a read-only, guard-railed structural query surface over the CKG — the escape hatch for questions no typed verb covers (e.g. "classes tagged
Repositorywith no inboundCALLS", "interfaces implemented by >5 classes"). Complementsckg_search(semantic discovery) with exact structural interrogation.The one idea
Caller text is never executed. A bounded Cypher-subset string is parsed into our own validated AST (the single trust boundary), then either compiled to native Cypher (Kuzu, Neo4j) or interpreted over the
GraphStoreABC (SurrealDB + any backend without a native query language). One AST, two execution strategies, identical results — proven by a shared conformance suite run live against all three backends.What ships
ckg query --graph '<cypher>'with--format table|json,--limit, andckg query --schema.ckg_querytool, capability-gated (registered only when the backend is query-capable andquery.allow_in_mcpis on); results carry the staleness +query_lang_versionenvelope.MATCHpatterns (directions, bounded var-length[:CALLS*1..3]),WHERE(comparisons,AND/OR/NOT,IN,STARTS/ENDS WITH,CONTAINS, pattern-existence),RETURN(projections, aliases,DISTINCT,count/min/max/avg/collect),ORDER BY/SKIP/LIMIT. Writes/DDL,CALL, unbounded paths, and un-joined Cartesian products are rejected with a clear reason.max_rows/timeout_ms/max_expansionsenforced on every backend, reported viatruncated+stopped_reason. Newquery:config block;ckg statusreports the language version + capability.Backends (all verified live)
All three return identical rows for the shared query set (parity), a runaway query trips the row/expansion caps (bounded), and writes can't reach execution (read-only).
Built in 7 chunks
AST+parser+validator → Cypher compiler + Kuzu → Neo4j → AST interpreter + SurrealDB → CLI → MCP tool + capability gate → config + docs (guide 13).
Notable design decision: SurrealDB models edges as a document table (no native graph traversal), so instead of a fragile SurrealQL translator it uses a portable AST interpreter over the
GraphStoreABC — which makes every backend query-capable for free. See the spec's Implementation-status section for all (deliberate, documented) deviations.Also included
fix(resolver)[BUG-CARRIED] — surfaced during end-to-end validation: a base class imported via an absolute path and re-exported through a package__init__(e.g.class KuzuGraphStore(GraphStore)) produced noINHERITSedge. Now resolved via a package re-export namespace (fixpoint). On the real package:INHERITS11→17,CALLS373→395 — improves recall for every consumer.Verification
ruff+mypy --strictclean.core+store(30 files, 520 nodes) and ran the headline queries via CLI and theckg_querytool — inheritance, aggregates+order+truncation, string preds, pattern-existence, var-length, JSON, rejections — all correct.🤖 Generated with Claude Code