Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion agents/explorer-rag-cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ The Decision Framework above tells you *which* command; reach for `--help` only

**Prerequisite.** `jrag` needs an index — unindexed, every command exits 2 (`jrag status` checks; the file-system tools work without one).

**Tip.** Run `jrag watch` once per session for fast, fresh queries — it keeps the index fresh on file change and serves every read command warm (no per-call model load). Optional; with no daemon running, all reads take the cold path byte-identically.
**Tip.** Run `jrag watch` once per session for fast, fresh queries — it keeps the index fresh on file change and serves every read command warm (no per-call model/graph load; warm lexical + graph on Intel Mac). Optional; with no daemon running, all reads take the cold path byte-identically.

**Resolve-first contract.** Every `<query>` command resolves the identifier first, then maps `one` / `many` / `none` onto one envelope: `one` → run; `many` → return candidates and stop, **no silent guess across distinct types** (a class sharing its simple name with its own constructor still resolves to the type — narrow with `--kind` / `--role` / `--fqn-contains` / `--service`); `none` → `status: not_found` (exit 0), fall back to `search` or `Grep`. Pass names (FQN / simple name / route path / topic) or prior `sym:`/`route:`/`client:`/`producer:` ids — never raw node ids. `--kind` is a true resolve input; `--role` / `--java-kind` / `--fqn-contains` post-filter client-side.

Expand Down
4 changes: 2 additions & 2 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ MCP tool call (server.py) ──asyncio.to_thread──▶ mcp_v2.*

### Watch path (`jrag watch`) — warm reads + freshness

When a `jrag watch` daemon is running, the **read path gains a warm hop**: the `jrag` read handlers ask the daemon over a Unix socket for the already-built payload instead of cold-loading the model and graph. The daemon reuses the MCP server's warm-cache posture — a process-singleton `_st_model` (SBERT) and a `LadybugGraph` — served to the CLI, so each query skips the per-call torch/model load. Output is byte-identical to the cold path (the same payload cores in `read_payloads.py` run either way). With no daemon running, the client transparently takes the cold path — the daemon is a pure accelerator, never a dependency.
When a `jrag watch` daemon is running, the **read path gains a warm hop**: the `jrag` read handlers ask the daemon over a Unix socket for the already-built payload instead of cold-loading the model and graph. The daemon reuses the MCP server's warm-cache posture — a process-singleton `_st_model` (SBERT) and a `LadybugGraph` — served to the CLI, so each query skips the per-call torch/model load. Output is byte-identical to the cold path (the same payload cores in `read_payloads.py` run either way). With no daemon running, the client transparently takes the cold path — the daemon is a pure accelerator, never a dependency. On a **graph-only install (macOS Intel)** the daemon probes `pipeline.vector_stack_installed()` at startup: the model warm-up is skipped and `search` degrades to lexical via `mcp_v2._ensure_vector_backend`; the cocoindex vectors reindex is skipped too, so the graph reindex still completes and fires `indexing_done`. State/`--status` carry `mode: lexical`.

| Concern | Module | Notes |
| --- | --- | --- |
Expand All @@ -101,7 +101,7 @@ When a `jrag watch` daemon is running, the **read path gains a warm hop**: the `
| Socket server | `watch/server.py` | `WatchServer` dispatches read payloads (serialized, not rendered). |
| IPC client | `watch/client.py` | `is_daemon_alive` / `get_payload`; any error → cold fallback. |
| Watcher | `watch/watcher.py` | `SourceWatcher` (watchdog native + polling fallback), lossless debounce, per-type routing. |
| Daemon | `watch/daemon.py` | `WatchDaemon` lifecycle: lock → warm → server → watcher → serve loop → teardown (`os._exit(0)`). |
| Daemon | `watch/daemon.py` | `WatchDaemon` lifecycle: lock → (warm, only if vector stack installed) → server → watcher → serve loop → teardown (`os._exit(0)`). |

**Reindexing is subprocessed**, never in-process: cocoindex for vectors, `build_ast_graph.py --incremental` for the graph. **Concurrency:** searches never wait and never see partial state — Lance commits are atomic per version (fresh per-query reads are consistent), and the graph (LadybugDB — no transactions, single writer) is kept readable via a **copy-on-write file snapshot** of `code_graph.lbug` taken around each graph reindex: reads continue from the snapshot while the subprocess writes the original, then `reset_for_path` repoints the live handle.

Expand Down
3 changes: 3 additions & 0 deletions docs/CONFIGURATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -300,6 +300,9 @@ generated_detection:
# The `jrag watch` daemon (index freshness + warm-query). No env vars are
# introduced for watch — precedence is CLI flag (--debounce-ms / --backend)
# > YAML > built-in default. See JAVA-CODEBASE-RAG-CLI.md § `jrag watch`.
# On Intel Mac (graph-only) the daemon runs without the vector stack: it skips
# the model warm-up + cocoindex reindex and serves warm lexical search; the
# watch: keys below apply unchanged on every platform.
watch:
# watch.debounce_ms — reindex debounce window in ms. Default 1500; floor 100
# (a value at/below 100 ms falls back to 1500 with a stderr warning).
Expand Down
4 changes: 3 additions & 1 deletion docs/JAVA-CODEBASE-RAG-CLI.md
Original file line number Diff line number Diff line change
Expand Up @@ -489,7 +489,7 @@ jrag search "service" --limit 20 --offset 20

### `jrag watch`

A single long-running daemon that does two things at once **while it runs**: (a) **keeps the index fresh** — it watches the source tree and re-runs a debounced per-type reindex (vectors via cocoindex, graph via `build_ast_graph.py --incremental`) on file change; and (b) **serves every read command warm** — `search` / `find` / `inspect` / `callers` / `callees` / `flow` are served over a Unix socket from a pre-loaded model + graph, so each query skips the per-call torch/model load and is effectively instant. The "run it while you code" workflow: start it once per coding session (foreground or detached), then keep issuing the normal `jrag` read commands — they are accelerated automatically.
A single long-running daemon that does two things at once **while it runs**: (a) **keeps the index fresh** — it watches the source tree and re-runs a debounced per-type reindex (vectors via cocoindex, graph via `build_ast_graph.py --incremental`) on file change; and (b) **serves every read command warm** — `search` / `find` / `inspect` / `callers` / `callees` / `flow` are served over a Unix socket from a pre-loaded model + graph (graph-only on Intel Mac, where there is no model), so each query skips the per-call model/graph load and is effectively instant. The "run it while you code" workflow: start it once per coding session (foreground or detached), then keep issuing the normal `jrag` read commands — they are accelerated automatically.

```bash
# Foreground — Ctrl+C (or SIGTERM) stops it
Expand All @@ -516,3 +516,5 @@ jrag watch --stop
**Cold-fallback guarantee.** With **no daemon running**, every read command behaves byte-identically to today — the daemon is a pure accelerator + freshness layer, never a dependency. If the daemon is down or unreachable, each `jrag` read silently takes the cold path (identical output, identical exit codes; you only pay the one-off cold-start model/graph load). See [`CONFIGURATION.md`](./CONFIGURATION.md) § 2 for the `watch:` block.

**Unix-only.** `jrag watch` relies on `fcntl`, so it runs on **macOS / Linux** only. On Windows it prints `jrag watch: watch mode requires macOS/Linux` to stderr and exits **2**; the cold read path is unaffected on every platform. One daemon per index dir — a pidfile + `flock` prevents two watchers (or a concurrent manual `increment`) on the same project.

**Graph-only (macOS Intel).** On Intel Mac the vector stack is absent (PEP 508 excludes `sentence_transformers`/`lancedb`/`cocoindex`), so the daemon runs in **lexical/graph-only mode**: it skips the embedding-model warm-up and the cocoindex vectors reindex, serves warm **lexical** `search` (BM25 over the symbol graph — same as the cold `search` path on Intel Mac) plus every structural command (`find`/`inspect`/`callers`/`callees`/`flow`), and reindexes only the graph on file change. `jrag watch --status` and the TTY panel report `mode: lexical (graph-only)`. Every `search` result carries the usual `lexical_mode=true` flag + advisory.
2 changes: 1 addition & 1 deletion skills/explore-codebase-cli/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ The Decision Framework above tells you *which* command; reach for `--help` only

**Prerequisite.** `jrag` needs an index — unindexed, every command exits 2 (`jrag status` checks; the file-system tools work without one).

**Tip.** Run `jrag watch` once per session for fast, fresh queries — it keeps the index fresh on file change and serves every read command warm (no per-call model load). Optional; with no daemon running, all reads take the cold path byte-identically.
**Tip.** Run `jrag watch` once per session for fast, fresh queries — it keeps the index fresh on file change and serves every read command warm (no per-call model/graph load; warm lexical + graph on Intel Mac). Optional; with no daemon running, all reads take the cold path byte-identically.

**Resolve-first contract.** Every `<query>` command resolves the identifier first, then maps `one` / `many` / `none` onto one envelope: `one` → run; `many` → return candidates and stop, **no silent guess across distinct types** (a class sharing its simple name with its own constructor still resolves to the type — narrow with `--kind` / `--role` / `--fqn-contains` / `--service`); `none` → `status: not_found` (exit 0), fall back to `search` or `Grep`. Pass names (FQN / simple name / route path / topic) or prior `sym:`/`route:`/`client:`/`producer:` ids — never raw node ids. `--kind` is a true resolve input; `--role` / `--java-kind` / `--fqn-contains` post-filter client-side.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ The Decision Framework above tells you *which* command; reach for `--help` only

**Prerequisite.** `jrag` needs an index — unindexed, every command exits 2 (`jrag status` checks; the file-system tools work without one).

**Tip.** Run `jrag watch` once per session for fast, fresh queries — it keeps the index fresh on file change and serves every read command warm (no per-call model load). Optional; with no daemon running, all reads take the cold path byte-identically.
**Tip.** Run `jrag watch` once per session for fast, fresh queries — it keeps the index fresh on file change and serves every read command warm (no per-call model/graph load; warm lexical + graph on Intel Mac). Optional; with no daemon running, all reads take the cold path byte-identically.

**Resolve-first contract.** Every `<query>` command resolves the identifier first, then maps `one` / `many` / `none` onto one envelope: `one` → run; `many` → return candidates and stop, **no silent guess across distinct types** (a class sharing its simple name with its own constructor still resolves to the type — narrow with `--kind` / `--role` / `--fqn-contains` / `--service`); `none` → `status: not_found` (exit 0), fall back to `search` or `Grep`. Pass names (FQN / simple name / route path / topic) or prior `sym:`/`route:`/`client:`/`producer:` ids — never raw node ids. `--kind` is a true resolve input; `--role` / `--java-kind` / `--fqn-contains` post-filter client-side.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ The Decision Framework above tells you *which* command; reach for `--help` only

**Prerequisite.** `jrag` needs an index — unindexed, every command exits 2 (`jrag status` checks; the file-system tools work without one).

**Tip.** Run `jrag watch` once per session for fast, fresh queries — it keeps the index fresh on file change and serves every read command warm (no per-call model load). Optional; with no daemon running, all reads take the cold path byte-identically.
**Tip.** Run `jrag watch` once per session for fast, fresh queries — it keeps the index fresh on file change and serves every read command warm (no per-call model/graph load; warm lexical + graph on Intel Mac). Optional; with no daemon running, all reads take the cold path byte-identically.

**Resolve-first contract.** Every `<query>` command resolves the identifier first, then maps `one` / `many` / `none` onto one envelope: `one` → run; `many` → return candidates and stop, **no silent guess across distinct types** (a class sharing its simple name with its own constructor still resolves to the type — narrow with `--kind` / `--role` / `--fqn-contains` / `--service`); `none` → `status: not_found` (exit 0), fall back to `search` or `Grep`. Pass names (FQN / simple name / route path / topic) or prior `sym:`/`route:`/`client:`/`producer:` ids — never raw node ids. `--kind` is a true resolve input; `--role` / `--java-kind` / `--fqn-contains` post-filter client-side.

Expand Down
2 changes: 2 additions & 0 deletions src/java_codebase_rag/jrag.py
Original file line number Diff line number Diff line change
Expand Up @@ -1462,6 +1462,8 @@ def _cmd_watch_status(cfg) -> int:
if alive:
print(f"jrag watch: up (pid {pid}, socket {sock})")
if state:
if state.get("mode") == "lexical":
print(" mode: lexical (graph-only)")
kind = state.get("last_reindex_kind")
at = state.get("last_reindex_at")
count = state.get("reindex_count", 0)
Expand Down
52 changes: 40 additions & 12 deletions src/java_codebase_rag/watch/daemon.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,12 @@

1. Acquire the project lock. Held elsewhere -> stderr line + ``return 2``.
Unsupported platform -> stderr line + ``return 2``.
2. EAGERLY warm the embedding model so a load failure fails fast (stderr +
``on_event("error", …)`` + lock release + ``return 2``).
2. EAGERLY warm the embedding model — but ONLY when the vector stack is
installed — so a load failure fails fast (stderr + ``on_event("error", …)``
+ lock release + ``return 2``). On graph-only installs (macOS Intel: PEP
508 excludes sentence_transformers/lancedb) this step is skipped and the
daemon serves warm lexical/graph-only search instead; the read path
(``mcp_v2.search_v2``) degrades to lexical on its own.
3. Install SIGINT/SIGTERM handlers that set a stop flag.
4. ``server.start()`` then ``watcher.start()``.
5. Write the state file (``paths.state_path``) so ``--status``/``--stop`` from
Expand All @@ -42,6 +46,7 @@
import time
from typing import TYPE_CHECKING, Any

from java_codebase_rag.pipeline import vector_stack_installed
from java_codebase_rag.watch import paths
from java_codebase_rag.watch.lock import (
LockHeldError,
Expand Down Expand Up @@ -75,6 +80,10 @@ class WatchDaemon:

def __init__(self, cfg: "ResolvedOperatorConfig") -> None:
self.cfg = cfg
# Probed once (cheap: 3x importlib.util.find_spec). When False (graph-only
# install — macOS Intel), the daemon skips the embedding-model warm-up and
# the cocoindex vectors reindex; the read path degrades to lexical on its own.
self._vector_enabled = vector_stack_installed()
self.lock = ProjectLock(cfg.index_dir)
self.warm = WarmResources(cfg)
self.server = WatchServer(self.warm, cfg)
Expand All @@ -94,6 +103,13 @@ def __init__(self, cfg: "ResolvedOperatorConfig") -> None:
"started_at": None,
"pid": None,
"socket": str(paths.socket_path(cfg.index_dir)),
# Display label derived from the install-time probe above, NOT a live
# search-capability check — the read path's actual lexical/vector choice
# is mcp_v2's (``_ensure_vector_backend``). The two agree under the PEP
# 508 markers (the vector trio is present or absent together). Surfaced
# in the status panel and ``jrag watch --status``; omitted from display
# on the normal (vector) path to avoid noise.
"mode": "lexical" if not self._vector_enabled else "vector",
"last_reindex_at": None,
"last_reindex_kind": None,
"reindex_count": 0,
Expand Down Expand Up @@ -128,16 +144,26 @@ def run_foreground(self) -> int:
print("jrag watch: watch mode requires macOS/Linux", file=sys.stderr)
return 2

# 2. Eagerly warm the model so a load failure fails fast (before the
# server accepts a single query). The model is the only heavy,
# failure-prone resource that is not lazy on the read path.
try:
self.warm.model()
except Exception as exc: # noqa: BLE001 — report any load failure, then bail
print(f"jrag watch: failed to load embedding model: {exc}", file=sys.stderr)
self._record("error", {"phase": "model_load", "error": repr(exc)})
self.lock.release()
return 2
# 2. Eagerly warm the embedding model so a load failure fails fast (before
# the server accepts a single query) — but ONLY when the vector stack is
# installed. The model is the only heavy, failure-prone resource that is
# not lazy on the read path. On a graph-only install (macOS Intel) there
# is no vector stack to warm; the daemon serves lexical/graph-only search
# and ``mcp_v2.search_v2`` degrades on its own, so we skip straight to
# serving rather than failing on a missing ``sentence_transformers``.
if self._vector_enabled:
try:
self.warm.model()
except Exception as exc: # noqa: BLE001 — report any load failure, then bail
print(f"jrag watch: failed to load embedding model: {exc}", file=sys.stderr)
self._record("error", {"phase": "model_load", "error": repr(exc)})
self.lock.release()
return 2
else:
print(
"jrag watch: vector stack unavailable — serving lexical (graph-only) search",
file=sys.stderr,
)

# 3. Install stop-signal handlers (main thread only).
signal.signal(signal.SIGINT, self._on_signal)
Expand Down Expand Up @@ -279,6 +305,8 @@ def _render_panel(self):
state = dict(self._state)
table = Table(title=f"jrag watch (pid {os.getpid()})", show_header=False, box=None)
table.add_row("socket", str(state.get("socket")))
if state.get("mode") == "lexical":
table.add_row("mode", "lexical (graph-only)")
table.add_row("reindex count", str(state.get("reindex_count", 0)))
last_kind = state.get("last_reindex_kind")
last_at = state.get("last_reindex_at")
Expand Down
Loading
Loading