From 7dca0768189906a205ffa23e71fb5ce590070a9b Mon Sep 17 00:00:00 2001 From: Dmitry Teryaev Date: Mon, 13 Jul 2026 23:27:18 +0300 Subject: [PATCH 1/3] fix(watch): degrade to lexical/graph-only mode when vectors unavailable (Intel Mac) `jrag watch` was the only code path that hard-required the vector stack: run_foreground eagerly called warm.model() (pulls sentence_transformers) and watcher.reindex always ran cocoindex. On Intel Mac (darwin + x86_64) PEP 508 markers exclude sentence_transformers/lancedb/cocoindex, so the daemon printed "failed to load embedding model" and exited 2 before serving; and a debounced reindex would bail on a 127 cocoindex stub and never fire indexing_done. Reuse pipeline.vector_stack_installed() (already used everywhere else) to gate: - daemon: skip the model warm-up when vectors are absent; serve warm lexical/graph-only search (the read path, mcp_v2.search_v2, already degrades to lexical on its own). State/panel/--status carry mode=lexical. - watcher: skip the cocoindex reindex step so the graph reindex completes and fires indexing_done instead of erroring forever. Consistent with the cold CLI, which already runs lexical on Intel Mac (#443). Tests: graph-only daemon startup (subprocess exercises real run_foreground; WarmResources.model() raises if called, proving the gating) + graph-only reindex (skips cocoindex, runs graph, fires indexing_done). Existing watcher tests pinned to _vector_enabled=True so they're host-independent. Full suite green (1538 passed, 38 skipped). Co-Authored-By: Claude --- docs/ARCHITECTURE.md | 4 +- docs/CONFIGURATION.md | 3 + docs/JAVA-CODEBASE-RAG-CLI.md | 2 + src/java_codebase_rag/jrag.py | 2 + src/java_codebase_rag/watch/daemon.py | 49 ++++++++++---- src/java_codebase_rag/watch/watcher.py | 36 +++++++---- tests/watch/test_daemon.py | 88 ++++++++++++++++++++++++++ tests/watch/test_watcher.py | 44 +++++++++++++ 8 files changed, 203 insertions(+), 25 deletions(-) diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 5e4aa2c..7cb4336 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -90,7 +90,7 @@ MCP tool call (server.py) ──asyncio.to_thread──▶ mcp_v2.* ### Watch path (`jrag watch`) — warm reads + freshness -When a `jrag watch` daemon is running, the **read path gains a warm hop**: the `jrag` read handlers ask the daemon over a Unix socket for the already-built payload instead of cold-loading the model and graph. The daemon reuses the MCP server's warm-cache posture — a process-singleton `_st_model` (SBERT) and a `LadybugGraph` — served to the CLI, so each query skips the per-call torch/model load. Output is byte-identical to the cold path (the same payload cores in `read_payloads.py` run either way). With no daemon running, the client transparently takes the cold path — the daemon is a pure accelerator, never a dependency. +When a `jrag watch` daemon is running, the **read path gains a warm hop**: the `jrag` read handlers ask the daemon over a Unix socket for the already-built payload instead of cold-loading the model and graph. The daemon reuses the MCP server's warm-cache posture — a process-singleton `_st_model` (SBERT) and a `LadybugGraph` — served to the CLI, so each query skips the per-call torch/model load. Output is byte-identical to the cold path (the same payload cores in `read_payloads.py` run either way). With no daemon running, the client transparently takes the cold path — the daemon is a pure accelerator, never a dependency. On a **graph-only install (macOS Intel)** the daemon probes `pipeline.vector_stack_installed()` at startup: the model warm-up is skipped and `search` degrades to lexical via `mcp_v2._ensure_vector_backend`; the cocoindex vectors reindex is skipped too, so the graph reindex still completes and fires `indexing_done`. State/`--status` carry `mode: lexical`. | Concern | Module | Notes | | --- | --- | --- | @@ -101,7 +101,7 @@ When a `jrag watch` daemon is running, the **read path gains a warm hop**: the ` | Socket server | `watch/server.py` | `WatchServer` dispatches read payloads (serialized, not rendered). | | IPC client | `watch/client.py` | `is_daemon_alive` / `get_payload`; any error → cold fallback. | | Watcher | `watch/watcher.py` | `SourceWatcher` (watchdog native + polling fallback), lossless debounce, per-type routing. | -| Daemon | `watch/daemon.py` | `WatchDaemon` lifecycle: lock → warm → server → watcher → serve loop → teardown (`os._exit(0)`). | +| Daemon | `watch/daemon.py` | `WatchDaemon` lifecycle: lock → (warm, only if vector stack installed) → server → watcher → serve loop → teardown (`os._exit(0)`). | **Reindexing is subprocessed**, never in-process: cocoindex for vectors, `build_ast_graph.py --incremental` for the graph. **Concurrency:** searches never wait and never see partial state — Lance commits are atomic per version (fresh per-query reads are consistent), and the graph (LadybugDB — no transactions, single writer) is kept readable via a **copy-on-write file snapshot** of `code_graph.lbug` taken around each graph reindex: reads continue from the snapshot while the subprocess writes the original, then `reset_for_path` repoints the live handle. diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.md index 1fa137f..ec8d323 100644 --- a/docs/CONFIGURATION.md +++ b/docs/CONFIGURATION.md @@ -300,6 +300,9 @@ generated_detection: # The `jrag watch` daemon (index freshness + warm-query). No env vars are # introduced for watch — precedence is CLI flag (--debounce-ms / --backend) # > YAML > built-in default. See JAVA-CODEBASE-RAG-CLI.md § `jrag watch`. +# On Intel Mac (graph-only) the daemon runs without the vector stack: it skips +# the model warm-up + cocoindex reindex and serves warm lexical search; the +# watch: keys below apply unchanged on every platform. watch: # watch.debounce_ms — reindex debounce window in ms. Default 1500; floor 100 # (a value at/below 100 ms falls back to 1500 with a stderr warning). diff --git a/docs/JAVA-CODEBASE-RAG-CLI.md b/docs/JAVA-CODEBASE-RAG-CLI.md index e3af342..fc5542e 100644 --- a/docs/JAVA-CODEBASE-RAG-CLI.md +++ b/docs/JAVA-CODEBASE-RAG-CLI.md @@ -516,3 +516,5 @@ jrag watch --stop **Cold-fallback guarantee.** With **no daemon running**, every read command behaves byte-identically to today — the daemon is a pure accelerator + freshness layer, never a dependency. If the daemon is down or unreachable, each `jrag` read silently takes the cold path (identical output, identical exit codes; you only pay the one-off cold-start model/graph load). See [`CONFIGURATION.md`](./CONFIGURATION.md) § 2 for the `watch:` block. **Unix-only.** `jrag watch` relies on `fcntl`, so it runs on **macOS / Linux** only. On Windows it prints `jrag watch: watch mode requires macOS/Linux` to stderr and exits **2**; the cold read path is unaffected on every platform. One daemon per index dir — a pidfile + `flock` prevents two watchers (or a concurrent manual `increment`) on the same project. + +**Graph-only (macOS Intel).** On Intel Mac the vector stack is absent (PEP 508 excludes `sentence_transformers`/`lancedb`/`cocoindex`), so the daemon runs in **lexical/graph-only mode**: it skips the embedding-model warm-up and the cocoindex vectors reindex, serves warm **lexical** `search` (BM25 over the symbol graph — same as the cold `search` path on Intel Mac) plus every structural command (`find`/`inspect`/`callers`/`callees`/`flow`), and reindexes only the graph on file change. `jrag watch --status` and the TTY panel report `mode: lexical (graph-only)`. Every `search` result carries the usual `lexical_mode=true` flag + advisory. diff --git a/src/java_codebase_rag/jrag.py b/src/java_codebase_rag/jrag.py index 8f68366..4bc5d67 100644 --- a/src/java_codebase_rag/jrag.py +++ b/src/java_codebase_rag/jrag.py @@ -1462,6 +1462,8 @@ def _cmd_watch_status(cfg) -> int: if alive: print(f"jrag watch: up (pid {pid}, socket {sock})") if state: + if state.get("mode") == "lexical": + print(" mode: lexical (graph-only) — no vector ranking") kind = state.get("last_reindex_kind") at = state.get("last_reindex_at") count = state.get("reindex_count", 0) diff --git a/src/java_codebase_rag/watch/daemon.py b/src/java_codebase_rag/watch/daemon.py index 67e6718..4a0e7d2 100644 --- a/src/java_codebase_rag/watch/daemon.py +++ b/src/java_codebase_rag/watch/daemon.py @@ -15,8 +15,12 @@ 1. Acquire the project lock. Held elsewhere -> stderr line + ``return 2``. Unsupported platform -> stderr line + ``return 2``. - 2. EAGERLY warm the embedding model so a load failure fails fast (stderr + - ``on_event("error", …)`` + lock release + ``return 2``). + 2. EAGERLY warm the embedding model — but ONLY when the vector stack is + installed — so a load failure fails fast (stderr + ``on_event("error", …)`` + + lock release + ``return 2``). On graph-only installs (macOS Intel: PEP + 508 excludes sentence_transformers/lancedb) this step is skipped and the + daemon serves warm lexical/graph-only search instead; the read path + (``mcp_v2.search_v2``) degrades to lexical on its own. 3. Install SIGINT/SIGTERM handlers that set a stop flag. 4. ``server.start()`` then ``watcher.start()``. 5. Write the state file (``paths.state_path``) so ``--status``/``--stop`` from @@ -42,6 +46,7 @@ import time from typing import TYPE_CHECKING, Any +from java_codebase_rag.pipeline import vector_stack_installed from java_codebase_rag.watch import paths from java_codebase_rag.watch.lock import ( LockHeldError, @@ -75,6 +80,10 @@ class WatchDaemon: def __init__(self, cfg: "ResolvedOperatorConfig") -> None: self.cfg = cfg + # Probed once (cheap: 3x importlib.util.find_spec). When False (graph-only + # install — macOS Intel), the daemon skips the embedding-model warm-up and + # the cocoindex vectors reindex; the read path degrades to lexical on its own. + self._vector_enabled = vector_stack_installed() self.lock = ProjectLock(cfg.index_dir) self.warm = WarmResources(cfg) self.server = WatchServer(self.warm, cfg) @@ -94,6 +103,10 @@ def __init__(self, cfg: "ResolvedOperatorConfig") -> None: "started_at": None, "pid": None, "socket": str(paths.socket_path(cfg.index_dir)), + # "lexical" when the vector stack is absent (macOS Intel) — surfaced in + # the status panel and ``jrag watch --status``; omitted from display on + # the normal (vector) path to avoid noise. + "mode": "lexical" if not self._vector_enabled else "vector", "last_reindex_at": None, "last_reindex_kind": None, "reindex_count": 0, @@ -128,16 +141,26 @@ def run_foreground(self) -> int: print("jrag watch: watch mode requires macOS/Linux", file=sys.stderr) return 2 - # 2. Eagerly warm the model so a load failure fails fast (before the - # server accepts a single query). The model is the only heavy, - # failure-prone resource that is not lazy on the read path. - try: - self.warm.model() - except Exception as exc: # noqa: BLE001 — report any load failure, then bail - print(f"jrag watch: failed to load embedding model: {exc}", file=sys.stderr) - self._record("error", {"phase": "model_load", "error": repr(exc)}) - self.lock.release() - return 2 + # 2. Eagerly warm the embedding model so a load failure fails fast (before + # the server accepts a single query) — but ONLY when the vector stack is + # installed. The model is the only heavy, failure-prone resource that is + # not lazy on the read path. On a graph-only install (macOS Intel) there + # is no vector stack to warm; the daemon serves lexical/graph-only search + # and ``mcp_v2.search_v2`` degrades on its own, so we skip straight to + # serving rather than failing on a missing ``sentence_transformers``. + if self._vector_enabled: + try: + self.warm.model() + except Exception as exc: # noqa: BLE001 — report any load failure, then bail + print(f"jrag watch: failed to load embedding model: {exc}", file=sys.stderr) + self._record("error", {"phase": "model_load", "error": repr(exc)}) + self.lock.release() + return 2 + else: + print( + "jrag watch: vector stack unavailable — serving lexical (graph-only) search", + file=sys.stderr, + ) # 3. Install stop-signal handlers (main thread only). signal.signal(signal.SIGINT, self._on_signal) @@ -279,6 +302,8 @@ def _render_panel(self): state = dict(self._state) table = Table(title=f"jrag watch (pid {os.getpid()})", show_header=False, box=None) table.add_row("socket", str(state.get("socket"))) + if state.get("mode") == "lexical": + table.add_row("mode", "lexical (graph-only)") table.add_row("reindex count", str(state.get("reindex_count", 0))) last_kind = state.get("last_reindex_kind") last_at = state.get("last_reindex_at") diff --git a/src/java_codebase_rag/watch/watcher.py b/src/java_codebase_rag/watch/watcher.py index a7e6e41..b17cac8 100644 --- a/src/java_codebase_rag/watch/watcher.py +++ b/src/java_codebase_rag/watch/watcher.py @@ -40,7 +40,7 @@ from watchdog.observers.polling import PollingObserver from java_codebase_rag.graph.path_filtering import LayeredIgnore -from java_codebase_rag.pipeline import run_cocoindex_update, run_incremental_graph +from java_codebase_rag.pipeline import run_cocoindex_update, run_incremental_graph, vector_stack_installed if TYPE_CHECKING: from java_codebase_rag.config import ResolvedOperatorConfig @@ -126,6 +126,10 @@ def __init__( ) -> None: self.cfg = cfg self.warm = warm + # Probed once: when False (graph-only install — macOS Intel) the vectors + # (cocoindex) reindex step is skipped entirely, so the graph reindex still + # completes and fires ``indexing_done`` instead of bailing on a 127 stub. + self._vector_enabled = vector_stack_installed() self._debounce_s = max(int(debounce_ms), 1) / 1000.0 self._backend = backend # watchdog's PollingObserver takes a float timeout in SECONDS (it flows @@ -281,7 +285,7 @@ def _debounce_loop(self) -> None: # -- reindex (runs on the debounce worker thread) ------------------------ def reindex(self, kinds: set[str]) -> None: - """Run one debounced reindex: vectors always; graph only when java changed. + """Run one debounced reindex: vectors always (when installed); graph only when java changed. COW lifecycle (design §4.7): the graph subprocess writes the ORIGINAL graph while reads are served from a sidecar copy. ``begin_graph_snapshot`` @@ -290,20 +294,30 @@ def reindex(self, kinds: set[str]) -> None: graph failure the existing ``.graph_increment_in_progress`` crash marker drives the next full rebuild. Lance needs no snapshot (commits are atomic per version; fresh per-query reads are fine). + + Graph-only installs (macOS Intel) have no vector stack: the cocoindex + step is skipped (``vres=None``) so the graph reindex completes and fires + ``indexing_done`` instead of bailing on a 127 cocoindex-not-found stub. """ if not kinds: return kind_list = sorted(kinds) self._emit("indexing_started", {"kinds": kind_list}) try: - # Vectors run for every indexed type (java/sql/yaml all flow through cocoindex). - self._emit("vectors", {"kinds": kind_list}) - vres = run_cocoindex_update( - self.cfg.subprocess_env(), - full_reprocess=False, - quiet=True, - verbose=False, - ) + # Vectors run for every indexed type (java/sql/yaml all flow through + # cocoindex) — but only when the vector stack is installed. On a + # graph-only install cocoindex is absent and the call would return a + # 127 stub; skipping it keeps ``indexing_done`` reachable. + if self._vector_enabled: + self._emit("vectors", {"kinds": kind_list}) + vres = run_cocoindex_update( + self.cfg.subprocess_env(), + full_reprocess=False, + quiet=True, + verbose=False, + ) + else: + vres = None graph_rc = 0 if "java" in kinds: @@ -328,7 +342,7 @@ def reindex(self, kinds: set[str]) -> None: # is never left dangling. No-ops when no snapshot is active. self.warm.commit_graph_snapshot() - if vres.returncode != 0: + if vres is not None and vres.returncode != 0: self._emit("error", {"phase": "vectors", "returncode": vres.returncode}) return if graph_rc != 0: diff --git a/tests/watch/test_daemon.py b/tests/watch/test_daemon.py index 7c60c37..473695b 100644 --- a/tests/watch/test_daemon.py +++ b/tests/watch/test_daemon.py @@ -477,6 +477,94 @@ def test_state_file_written_on_start(tmp_path, daemon_stub): _stop_proc(proc, index_dir) +# =========================================================================== +# (f) graph-only startup (macOS Intel): skip model load, serve lexical mode +# =========================================================================== + + +_GRAPH_ONLY_STUB_SCRIPT = '''\ +"""Graph-only stub watch daemon: simulates macOS Intel (no vector stack). + +Patches ``daemon.vector_stack_installed`` -> False, swaps in a WarmResources whose +``model()`` RAISES (proving ``run_foreground`` never calls it in lexical mode), +fakes SourceWatcher, then runs the REAL ``run_foreground``. If the gating +regresses (model() called), the AssertionError surfaces as "failed to load +embedding model" -> exit 2 -> the daemon never comes up -> the test fails fast. +""" +import os +import sys + +from java_codebase_rag.config import resolve_operator_config +from java_codebase_rag.watch import daemon + + +class _RaisingModelWarm: + def __init__(self, cfg): + self.cfg = cfg + + def model(self): + raise AssertionError("warm.model() must not be called in lexical mode") + + def graph(self): + return None + + def begin_graph_snapshot(self): + pass + + def commit_graph_snapshot(self): + pass + + +class _FakeWatcher: + def __init__(self, cfg, warm, *, debounce_ms, backend, poll_interval_ms, on_event=None): + pass + + def start(self): + pass + + def stop(self): + pass + + +# Simulate a graph-only install (macOS Intel): vector stack absent. +daemon.vector_stack_installed = lambda: False +daemon.WarmResources = _RaisingModelWarm +daemon.SourceWatcher = _FakeWatcher + +cfg = resolve_operator_config(source_root=None, cli_index_dir=os.environ["JRAG_WATCH_TEST_INDEX"]) +cfg.apply_to_os_environ() +daemon.WatchDaemon(cfg).run_foreground() +sys.exit(0) # pragma: no cover - run_foreground ends with os._exit(0) +''' + + +def test_foreground_graph_only_starts_without_vectors(tmp_path, monkeypatch): + """Lexical-mode startup (macOS Intel): with the vector stack absent, the daemon + must NOT call ``warm.model()`` (it would import sentence_transformers and exit 2) + and must reach serving — the state file carries ``mode='lexical'``. + + The stub's ``WarmResources.model()`` raises AssertionError if called, so a gating + regression surfaces as a non-alive daemon (exit 2) rather than a silent pass. + """ + index_dir, source_root = _index_source(tmp_path, tag="goidx") + _anchor_env(monkeypatch, index_dir, source_root) + + script = tmp_path / "graph_only_stub.py" + script.write_text(_GRAPH_ONLY_STUB_SCRIPT) + proc = _spawn_stub(script, index_dir, source_root) + try: + assert _wait_alive(index_dir, _STUB_READY_S), ( + "graph-only daemon did not come up — warm.model() was not skipped (exit 2?)" + ) + import json + + state = json.loads(paths.state_path(index_dir).read_text()) + assert state.get("mode") == "lexical", f"expected mode='lexical', got {state.get('mode')!r}" + assert state["pid"] == proc.pid + finally: + _stop_proc(proc, index_dir) + + # =========================================================================== # GOLDEN IPC TEST (heavy) — jrag search over the socket == cold path, byte for byte # =========================================================================== diff --git a/tests/watch/test_watcher.py b/tests/watch/test_watcher.py index be8d81d..f56f87a 100644 --- a/tests/watch/test_watcher.py +++ b/tests/watch/test_watcher.py @@ -123,6 +123,11 @@ def _make_watcher( "java_codebase_rag.watch.watcher.run_incremental_graph", _make_graph_fake(calls, rc=graph_rc), ) + # Pin the vector capability so the vectors-path tests below are independent + # of the HOST's vector stack (they monkeypatch the pipeline fakes, not the + # probe). The graph-only path is exercised by overriding this to False in + # its own test. + watcher._vector_enabled = True return watcher @@ -275,6 +280,45 @@ def test_reindex_java_and_sql_runs_graph_once(tmp_path, monkeypatch): assert "commit_graph_snapshot" in calls +# -- reindex: graph-only (macOS Intel) -> skip vectors, still run graph --------- + + +def test_reindex_graph_only_skips_vectors_and_completes(tmp_path, monkeypatch): + """Lexical/graph-only mode (macOS Intel: no torch/lancedb/cocoindex): a java + reindex SKIPS the cocoindex vectors step entirely (it would 127 on the missing + binary and bail before firing ``indexing_done``), still runs the graph reindex + under its COW snapshot lifecycle, and completes — ``indexing_done`` fires and + ``last_reindex`` is recorded.""" + calls: list[str] = [] + w = _make_watcher(tmp_path, calls, monkeypatch) + w._vector_enabled = False # simulate a graph-only install + w.reindex({"java"}) + assert "run_cocoindex_update" not in calls + assert "on_event:vectors" not in calls + assert "run_incremental_graph" in calls + # COW snapshot lifecycle intact around the graph subprocess. + assert "begin_graph_snapshot" in calls + assert calls.index("begin_graph_snapshot") < calls.index("run_incremental_graph") + assert calls.index("run_incremental_graph") < calls.index("commit_graph_snapshot") + assert "on_event:indexing_done" in calls + assert w.last_reindex is not None + assert w.last_reindex["kinds"] == ["java"] + + +def test_reindex_graph_only_sql_is_clean_noop(tmp_path, monkeypatch): + """A sql change in graph-only mode is a clean no-op: vectors are skipped and + the graph does not index SQL, so neither pipeline fn runs but ``indexing_done`` + still fires (no perpetual vectors error as on the pre-fix path).""" + calls: list[str] = [] + w = _make_watcher(tmp_path, calls, monkeypatch) + w._vector_enabled = False + w.reindex({"sql"}) + assert "run_cocoindex_update" not in calls + assert "run_incremental_graph" not in calls + assert "begin_graph_snapshot" not in calls + assert "on_event:indexing_done" in calls + + # -- debounce coalescing ------------------------------------------------------- From c1203049d42a52822df05cd9bb5d417e34f90715 Mon Sep 17 00:00:00 2001 From: Dmitry Teryaev Date: Mon, 13 Jul 2026 23:40:38 +0300 Subject: [PATCH 2/3] test+docs: address code-review feedback (PR #446) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three-reviewer fan-out (core logic / tests / surface+docs) returned "ready to merge" with no Critical/Important correctness issues. Applied the cheap, high-value findings: - skills/explore-codebase-cli/SKILL.md + agents/explorer-rag-cli.md: the shipped watch tip "(no per-call model load)" misrepresents the benefit on Intel Mac (no model exists there). Broadened to "(no per-call model/graph load; warm lexical + graph on Intel Mac)" — accurate for both install flavors. These are verbatim-shipped artifacts consumed by the exact audience this PR enables. - tests/watch/test_daemon.py: the graph-only startup test captured no subprocess log and wasted the full 15s timeout on a regression that exits instantly. Now passes a log_path and polls proc.poll() for an early, diagnosable failure (_fail_with_log surfaces the log tail). - tests/watch/test_watcher.py: graph-only java reindex upgraded from membership asserts to the exact calls-list sequence (matches the sibling vector-path test's style; catches a spurious extra call). - jrag.py: `watch --status` mode line now prints just "lexical (graph-only)", matching the TTY panel and the CLI doc (was "— no vector ranking"). - watch/daemon.py: one-line comment clarifying _state["mode"] derives from the install-time probe, not a live search-capability check (the read path's actual lexical/vector choice is mcp_v2._ensure_vector_backend; they agree under the PEP 508 markers). - watch/watcher.py: wrapped the pipeline import to fit the 100-col line limit. - docs/JAVA-CODEBASE-RAG-CLI.md: hedged the watch intro's "skips the per-call torch/model load" to "model/graph load" + noted graph-only on Intel Mac. Deferred (pre-existing, out of scope): no positive-direction test that the vector-ENABLED daemon warms model() — the vector path was verified byte-unchanged vs base by review; adding it would require pinning _vector_enabled in the shared stub + a cross-process sentinel. Watch tests: 36 passed, 6 skipped. No new ruff errors (the 4 pre-existing jrag.py F-codes exist on origin/master). Co-Authored-By: Claude --- agents/explorer-rag-cli.md | 2 +- docs/JAVA-CODEBASE-RAG-CLI.md | 2 +- skills/explore-codebase-cli/SKILL.md | 2 +- src/java_codebase_rag/jrag.py | 2 +- src/java_codebase_rag/watch/daemon.py | 9 +++++--- src/java_codebase_rag/watch/watcher.py | 6 ++++- tests/watch/test_daemon.py | 31 ++++++++++++++++++++++---- tests/watch/test_watcher.py | 18 ++++++++------- 8 files changed, 52 insertions(+), 20 deletions(-) diff --git a/agents/explorer-rag-cli.md b/agents/explorer-rag-cli.md index 6edf3bb..bace46e 100644 --- a/agents/explorer-rag-cli.md +++ b/agents/explorer-rag-cli.md @@ -95,7 +95,7 @@ The Decision Framework above tells you *which* command; reach for `--help` only **Prerequisite.** `jrag` needs an index — unindexed, every command exits 2 (`jrag status` checks; the file-system tools work without one). -**Tip.** Run `jrag watch` once per session for fast, fresh queries — it keeps the index fresh on file change and serves every read command warm (no per-call model load). Optional; with no daemon running, all reads take the cold path byte-identically. +**Tip.** Run `jrag watch` once per session for fast, fresh queries — it keeps the index fresh on file change and serves every read command warm (no per-call model/graph load; warm lexical + graph on Intel Mac). Optional; with no daemon running, all reads take the cold path byte-identically. **Resolve-first contract.** Every `` command resolves the identifier first, then maps `one` / `many` / `none` onto one envelope: `one` → run; `many` → return candidates and stop, **no silent guess across distinct types** (a class sharing its simple name with its own constructor still resolves to the type — narrow with `--kind` / `--role` / `--fqn-contains` / `--service`); `none` → `status: not_found` (exit 0), fall back to `search` or `Grep`. Pass names (FQN / simple name / route path / topic) or prior `sym:`/`route:`/`client:`/`producer:` ids — never raw node ids. `--kind` is a true resolve input; `--role` / `--java-kind` / `--fqn-contains` post-filter client-side. diff --git a/docs/JAVA-CODEBASE-RAG-CLI.md b/docs/JAVA-CODEBASE-RAG-CLI.md index fc5542e..bceb97c 100644 --- a/docs/JAVA-CODEBASE-RAG-CLI.md +++ b/docs/JAVA-CODEBASE-RAG-CLI.md @@ -489,7 +489,7 @@ jrag search "service" --limit 20 --offset 20 ### `jrag watch` -A single long-running daemon that does two things at once **while it runs**: (a) **keeps the index fresh** — it watches the source tree and re-runs a debounced per-type reindex (vectors via cocoindex, graph via `build_ast_graph.py --incremental`) on file change; and (b) **serves every read command warm** — `search` / `find` / `inspect` / `callers` / `callees` / `flow` are served over a Unix socket from a pre-loaded model + graph, so each query skips the per-call torch/model load and is effectively instant. The "run it while you code" workflow: start it once per coding session (foreground or detached), then keep issuing the normal `jrag` read commands — they are accelerated automatically. +A single long-running daemon that does two things at once **while it runs**: (a) **keeps the index fresh** — it watches the source tree and re-runs a debounced per-type reindex (vectors via cocoindex, graph via `build_ast_graph.py --incremental`) on file change; and (b) **serves every read command warm** — `search` / `find` / `inspect` / `callers` / `callees` / `flow` are served over a Unix socket from a pre-loaded model + graph (graph-only on Intel Mac, where there is no model), so each query skips the per-call model/graph load and is effectively instant. The "run it while you code" workflow: start it once per coding session (foreground or detached), then keep issuing the normal `jrag` read commands — they are accelerated automatically. ```bash # Foreground — Ctrl+C (or SIGTERM) stops it diff --git a/skills/explore-codebase-cli/SKILL.md b/skills/explore-codebase-cli/SKILL.md index 8419b32..2ced213 100644 --- a/skills/explore-codebase-cli/SKILL.md +++ b/skills/explore-codebase-cli/SKILL.md @@ -94,7 +94,7 @@ The Decision Framework above tells you *which* command; reach for `--help` only **Prerequisite.** `jrag` needs an index — unindexed, every command exits 2 (`jrag status` checks; the file-system tools work without one). -**Tip.** Run `jrag watch` once per session for fast, fresh queries — it keeps the index fresh on file change and serves every read command warm (no per-call model load). Optional; with no daemon running, all reads take the cold path byte-identically. +**Tip.** Run `jrag watch` once per session for fast, fresh queries — it keeps the index fresh on file change and serves every read command warm (no per-call model/graph load; warm lexical + graph on Intel Mac). Optional; with no daemon running, all reads take the cold path byte-identically. **Resolve-first contract.** Every `` command resolves the identifier first, then maps `one` / `many` / `none` onto one envelope: `one` → run; `many` → return candidates and stop, **no silent guess across distinct types** (a class sharing its simple name with its own constructor still resolves to the type — narrow with `--kind` / `--role` / `--fqn-contains` / `--service`); `none` → `status: not_found` (exit 0), fall back to `search` or `Grep`. Pass names (FQN / simple name / route path / topic) or prior `sym:`/`route:`/`client:`/`producer:` ids — never raw node ids. `--kind` is a true resolve input; `--role` / `--java-kind` / `--fqn-contains` post-filter client-side. diff --git a/src/java_codebase_rag/jrag.py b/src/java_codebase_rag/jrag.py index 4bc5d67..9651f6c 100644 --- a/src/java_codebase_rag/jrag.py +++ b/src/java_codebase_rag/jrag.py @@ -1463,7 +1463,7 @@ def _cmd_watch_status(cfg) -> int: print(f"jrag watch: up (pid {pid}, socket {sock})") if state: if state.get("mode") == "lexical": - print(" mode: lexical (graph-only) — no vector ranking") + print(" mode: lexical (graph-only)") kind = state.get("last_reindex_kind") at = state.get("last_reindex_at") count = state.get("reindex_count", 0) diff --git a/src/java_codebase_rag/watch/daemon.py b/src/java_codebase_rag/watch/daemon.py index 4a0e7d2..72a3e9a 100644 --- a/src/java_codebase_rag/watch/daemon.py +++ b/src/java_codebase_rag/watch/daemon.py @@ -103,9 +103,12 @@ def __init__(self, cfg: "ResolvedOperatorConfig") -> None: "started_at": None, "pid": None, "socket": str(paths.socket_path(cfg.index_dir)), - # "lexical" when the vector stack is absent (macOS Intel) — surfaced in - # the status panel and ``jrag watch --status``; omitted from display on - # the normal (vector) path to avoid noise. + # Display label derived from the install-time probe above, NOT a live + # search-capability check — the read path's actual lexical/vector choice + # is mcp_v2's (``_ensure_vector_backend``). The two agree under the PEP + # 508 markers (the vector trio is present or absent together). Surfaced + # in the status panel and ``jrag watch --status``; omitted from display + # on the normal (vector) path to avoid noise. "mode": "lexical" if not self._vector_enabled else "vector", "last_reindex_at": None, "last_reindex_kind": None, diff --git a/src/java_codebase_rag/watch/watcher.py b/src/java_codebase_rag/watch/watcher.py index b17cac8..826d9b8 100644 --- a/src/java_codebase_rag/watch/watcher.py +++ b/src/java_codebase_rag/watch/watcher.py @@ -40,7 +40,11 @@ from watchdog.observers.polling import PollingObserver from java_codebase_rag.graph.path_filtering import LayeredIgnore -from java_codebase_rag.pipeline import run_cocoindex_update, run_incremental_graph, vector_stack_installed +from java_codebase_rag.pipeline import ( + run_cocoindex_update, + run_incremental_graph, + vector_stack_installed, +) if TYPE_CHECKING: from java_codebase_rag.config import ResolvedOperatorConfig diff --git a/tests/watch/test_daemon.py b/tests/watch/test_daemon.py index 473695b..653114d 100644 --- a/tests/watch/test_daemon.py +++ b/tests/watch/test_daemon.py @@ -122,6 +122,12 @@ def _wait_dead(proc: subprocess.Popen, timeout: float = _SHUTDOWN_WAIT_S) -> int return proc.wait() +def _fail_with_log(log_path: Path, prefix: str) -> None: + """``pytest.fail`` with the tail of a daemon subprocess log for diagnosis.""" + tail = log_path.read_bytes()[-4000:] if log_path.exists() else b"" + pytest.fail(f"{prefix}; log tail:\n{tail!r}") + + def _stop_proc(proc: subprocess.Popen, index_dir: Path) -> None: """Best-effort: SIGTERM a daemon subprocess and reap it + its files.""" if proc.poll() is None: @@ -551,11 +557,28 @@ def test_foreground_graph_only_starts_without_vectors(tmp_path, monkeypatch): script = tmp_path / "graph_only_stub.py" script.write_text(_GRAPH_ONLY_STUB_SCRIPT) - proc = _spawn_stub(script, index_dir, source_root) + log_path = tmp_path / "graph_only.log" + proc = _spawn_stub(script, index_dir, source_root, log_path=log_path) try: - assert _wait_alive(index_dir, _STUB_READY_S), ( - "graph-only daemon did not come up — warm.model() was not skipped (exit 2?)" - ) + # A gating regression makes the stub's model() raise -> exit 2 -> the process + # dies at once, so poll for early exit (not just the full _STUB_READY_S window) + # and surface the log tail for diagnosis. + came_up = False + deadline = time.monotonic() + _STUB_READY_S + while time.monotonic() < deadline: + if is_daemon_alive(index_dir): + came_up = True + break + if proc.poll() is not None: + _fail_with_log( + log_path, + f"graph-only daemon exited early (rc={proc.returncode}) — " + "warm.model() was not skipped", + ) + time.sleep(0.1) + if not came_up: + _fail_with_log(log_path, f"graph-only daemon did not come up in {_STUB_READY_S}s") + import json state = json.loads(paths.state_path(index_dir).read_text()) diff --git a/tests/watch/test_watcher.py b/tests/watch/test_watcher.py index f56f87a..193bae9 100644 --- a/tests/watch/test_watcher.py +++ b/tests/watch/test_watcher.py @@ -293,14 +293,16 @@ def test_reindex_graph_only_skips_vectors_and_completes(tmp_path, monkeypatch): w = _make_watcher(tmp_path, calls, monkeypatch) w._vector_enabled = False # simulate a graph-only install w.reindex({"java"}) - assert "run_cocoindex_update" not in calls - assert "on_event:vectors" not in calls - assert "run_incremental_graph" in calls - # COW snapshot lifecycle intact around the graph subprocess. - assert "begin_graph_snapshot" in calls - assert calls.index("begin_graph_snapshot") < calls.index("run_incremental_graph") - assert calls.index("run_incremental_graph") < calls.index("commit_graph_snapshot") - assert "on_event:indexing_done" in calls + # Exact sequence: the vectors step + its event are skipped, the graph still + # runs under its COW snapshot lifecycle, and indexing_done fires. + assert calls == [ + "on_event:indexing_started", + "on_event:graph", + "begin_graph_snapshot", + "run_incremental_graph", + "commit_graph_snapshot", + "on_event:indexing_done", + ] assert w.last_reindex is not None assert w.last_reindex["kinds"] == ["java"] From ef683f1079af13c14f34a0515c7cf791b88ff7c9 Mon Sep 17 00:00:00 2001 From: Dmitry Teryaev Date: Mon, 13 Jul 2026 23:58:55 +0300 Subject: [PATCH 3/3] fix(install-data): sync bundled agent artifacts with dev source CI (test_install_data_artifacts_in_sync_with_dev_source) failed because the review-fix commit edited the dev-source copies in skills/ and agents/ but not the bundled copies under src/java_codebase_rag/install_data/, which the sync check requires to be byte-equal. Regenerated via `scripts/sync_agent_artifacts.py` for the two edited watch tips (explorer-rag-cli.md + explore-codebase-cli/SKILL.md). Co-Authored-By: Claude --- src/java_codebase_rag/install_data/agents/explorer-rag-cli.md | 2 +- .../install_data/skills/explore-codebase-cli/SKILL.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/src/java_codebase_rag/install_data/agents/explorer-rag-cli.md b/src/java_codebase_rag/install_data/agents/explorer-rag-cli.md index 6edf3bb..bace46e 100644 --- a/src/java_codebase_rag/install_data/agents/explorer-rag-cli.md +++ b/src/java_codebase_rag/install_data/agents/explorer-rag-cli.md @@ -95,7 +95,7 @@ The Decision Framework above tells you *which* command; reach for `--help` only **Prerequisite.** `jrag` needs an index — unindexed, every command exits 2 (`jrag status` checks; the file-system tools work without one). -**Tip.** Run `jrag watch` once per session for fast, fresh queries — it keeps the index fresh on file change and serves every read command warm (no per-call model load). Optional; with no daemon running, all reads take the cold path byte-identically. +**Tip.** Run `jrag watch` once per session for fast, fresh queries — it keeps the index fresh on file change and serves every read command warm (no per-call model/graph load; warm lexical + graph on Intel Mac). Optional; with no daemon running, all reads take the cold path byte-identically. **Resolve-first contract.** Every `` command resolves the identifier first, then maps `one` / `many` / `none` onto one envelope: `one` → run; `many` → return candidates and stop, **no silent guess across distinct types** (a class sharing its simple name with its own constructor still resolves to the type — narrow with `--kind` / `--role` / `--fqn-contains` / `--service`); `none` → `status: not_found` (exit 0), fall back to `search` or `Grep`. Pass names (FQN / simple name / route path / topic) or prior `sym:`/`route:`/`client:`/`producer:` ids — never raw node ids. `--kind` is a true resolve input; `--role` / `--java-kind` / `--fqn-contains` post-filter client-side. diff --git a/src/java_codebase_rag/install_data/skills/explore-codebase-cli/SKILL.md b/src/java_codebase_rag/install_data/skills/explore-codebase-cli/SKILL.md index 8419b32..2ced213 100644 --- a/src/java_codebase_rag/install_data/skills/explore-codebase-cli/SKILL.md +++ b/src/java_codebase_rag/install_data/skills/explore-codebase-cli/SKILL.md @@ -94,7 +94,7 @@ The Decision Framework above tells you *which* command; reach for `--help` only **Prerequisite.** `jrag` needs an index — unindexed, every command exits 2 (`jrag status` checks; the file-system tools work without one). -**Tip.** Run `jrag watch` once per session for fast, fresh queries — it keeps the index fresh on file change and serves every read command warm (no per-call model load). Optional; with no daemon running, all reads take the cold path byte-identically. +**Tip.** Run `jrag watch` once per session for fast, fresh queries — it keeps the index fresh on file change and serves every read command warm (no per-call model/graph load; warm lexical + graph on Intel Mac). Optional; with no daemon running, all reads take the cold path byte-identically. **Resolve-first contract.** Every `` command resolves the identifier first, then maps `one` / `many` / `none` onto one envelope: `one` → run; `many` → return candidates and stop, **no silent guess across distinct types** (a class sharing its simple name with its own constructor still resolves to the type — narrow with `--kind` / `--role` / `--fqn-contains` / `--service`); `none` → `status: not_found` (exit 0), fall back to `search` or `Grep`. Pass names (FQN / simple name / route path / topic) or prior `sym:`/`route:`/`client:`/`producer:` ids — never raw node ids. `--kind` is a true resolve input; `--role` / `--java-kind` / `--fqn-contains` post-filter client-side.