Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@
- **Pull any Ollama model from the dashboard.** "Pull a new model" is now a free-text field with per-provider suggestions — type any name from the [Ollama library](https://ollama.com/library) (e.g. `mistral:7b`) and download with live progress. FastFlowLM keeps its catalog suggestions.
- **Benchmarks work on Ollama too.** The Benchmark tab runs timed generations against the running Ollama server (three prompt sizes × two passes, ~1–3 min on CPU) using Ollama's native prefill/decode counters, and records the same TTFT / prefill / decode metrics as `flm bench`. The server keeps serving during the run, and history rows now show which provider produced them.
- **Hardware-aware model suggestions.** The dashboard detects system RAM (GlobalMemoryStatusEx) and GPU VRAM (nvidia-smi, or the display-class registry's qwMemorySize — which also finds Ryzen AI iGPU carve-outs) and computes a per-provider size budget: FastFlowLM scales with installed RAM (32 GB ≈ 4B-class on the NPU, 64 GB ≈ 9B), Ollama with VRAM (e.g. 8 GB ≈ 9B) or conservatively with RAM on CPU-only boxes. The pull card shows the detected budget, suggestions hide models that don't fit (near-misses are marked "tight fit"), and free-typing an oversized model asks for confirmation. New daemon action `model_recommendations`; new module `ffp_hardware`.
- **Chat moved into the web dashboard.** Chat is now a Chat tab in the daemon-served dashboard — `Ctrl+Shift+T` and the tray "Open Chat" open it, and `Ctrl+Shift+A` sends the current selection there (prefilled). Threads, history, and the "ground answers in my notes" toggle work as before; the standalone tkinter chat popup is retired. The dashboard also deep-links by URL hash (`/#chat`). New module `ffp_chat`; new daemon actions `chat_threads_list` / `chat_thread_get` / `chat_send` / `chat_thread_delete` / `chat_stage_selection` / `chat_take_staged`.
- **Read, re-file, and delete notes from the dashboard.** The Notes tab is now an organizer: click a note to read its full body and source link, move it to a different bucket (the LLM's pick is no longer final), or delete it. New daemon actions `note_get` / `note_move` / `note_delete` — all vault-contained and path-traversal guarded.

### Fixed

Expand All @@ -23,6 +25,7 @@

### Internal

- Chat is now daemon-backed (`ffp_chat` — thread store reusing `data/chat_threads.jsonl`, provider-resolved `/v1/chat/completions`, optional notes-vault grounding) and rendered by the web dashboard's Chat tab. The standalone `chat_popup.py` tkinter app and its `127.0.0.1:52640` ingest socket (`chat_send_selection`/`chat_reload`/`chat_restart`) are removed; `Ctrl+Shift+A` now stages the selection via `chat_stage_selection` and the Chat tab picks it up. The PyInstaller freeze is now **three** executables (`ffp-daemon` / `ffp-grammar-fix` / `ffp-first-run`) instead of four.
- New modules `ffp_provider_status` (detection/capabilities) and `ffp_provider_runtime` (model list/pull/remove routing); registered in the wheel and PyInstaller spec. Provider roadmap notes live in `docs/provider-and-sync-roadmap.md`.
- Dead-code cleanup: removed the redundant `_deep_merge` / `_version_tuple` wrappers in `grammar_fix.py` (callers now use `ffp_config.deep_merge` / `ffp_updater.version_tuple` directly, and their unit tests moved next to the real functions), deduplicated the update-feed-URL config lookup, and dropped the shadowed `chat.llm_auth_bearer` key from shipped configs — `llm.auth_bearer` is the live setting and always wins; old user configs that still carry the chat key keep working.
- CI: the AutoHotkey syntax-check job no longer fails when the Chocolatey community feed has a transient outage — the install retries Chocolatey, then falls back to the official AutoHotkey v2 release zip, and fails fast if neither yields the interpreter (so the parse-check can never silently skip and pass green).
Expand Down
22 changes: 22 additions & 0 deletions scripts/ffp_daemon.py
Original file line number Diff line number Diff line change
Expand Up @@ -531,6 +531,24 @@ def _act_save_note(args: dict) -> dict:
return notes.capture_note(text=text, source_app=source_app, url=url)


def _act_note_get(args: dict) -> dict:
"""Read one note's full content for the dashboard reader."""
import notes
return notes.get_note(str(args.get("relpath") or args.get("path") or ""))


def _act_note_move(args: dict) -> dict:
"""Re-file a note into a different bucket (updates frontmatter category)."""
import notes
return notes.move_note(str(args.get("relpath") or ""), str(args.get("category") or ""))


def _act_note_delete(args: dict) -> dict:
"""Delete a note from the vault."""
import notes
return notes.delete_note(str(args.get("relpath") or ""))


def _act_notify(args: dict) -> str:
title = str(args.get("title") or "").strip() or "Flowkey"
message = str(args.get("message") or "").strip()
Expand Down Expand Up @@ -634,6 +652,9 @@ def _act_chat_take_staged(_args: dict) -> dict:
"bench_history": _act_bench_history,
"note_search": _act_note_search,
"notes_list": _act_notes_list,
"note_get": _act_note_get,
"note_move": _act_note_move,
"note_delete": _act_note_delete,
"mode_ids": _act_mode_ids,
"pull_start": _act_pull_start,
"pull_status": _act_pull_status,
Expand Down Expand Up @@ -661,6 +682,7 @@ def _act_chat_take_staged(_args: dict) -> dict:
"pull_model", "remove_model", "apply_config_patch", "update_apply",
"set_autostart", "bench_start", "pull_start",
"chat_send", "chat_thread_delete", "chat_stage_selection", "chat_take_staged",
"note_move", "note_delete",
}

_shutdown_event = threading.Event()
Expand Down
105 changes: 105 additions & 0 deletions scripts/notes.py
Original file line number Diff line number Diff line change
Expand Up @@ -466,12 +466,15 @@ def search_notes(query: str, limit: int = 5) -> dict:
category = str(rel.parent).replace("\\", "/")
if category in (".", ""):
category = "inbox"
relpath = str(rel).replace("\\", "/")
except ValueError:
category = path.parent.name
relpath = path.name
matches.append({
"title": title or path.stem,
"category": category,
"path": str(path),
"relpath": relpath,
"score": score,
"snippet": _snippet_around(body, terms),
})
Expand Down Expand Up @@ -502,16 +505,118 @@ def list_recent_notes(limit: int = 20) -> dict:
category = str(rel.parent).replace("\\", "/")
if category in (".", ""):
category = "inbox"
relpath = str(rel).replace("\\", "/")
except ValueError:
category = path.parent.name
relpath = path.name
results.append({
"title": title or path.stem,
"category": category,
"relpath": relpath,
"modified": time.strftime("%Y-%m-%d %H:%M", time.localtime(path.stat().st_mtime)),
})
return {"results": results, "count": len(files)}


# ---------- Note read / move / delete (web dashboard organizer) --------------

def _safe_relpath(relpath: str) -> str:
"""Normalize + validate a vault-relative note path (reject traversal)."""
clean = str(relpath or "").strip().replace("\\", "/").strip("/")
if not clean:
raise ValueError("empty note path")
for part in clean.split("/"):
if not part or part in (".", ".."):
raise ValueError(f"invalid note path: {relpath!r}")
return clean


def _frontmatter_field(text: str, key: str) -> str:
"""Best-effort read of a single YAML-frontmatter scalar (e.g. 'source')."""
if not text.startswith("---"):
return ""
end = text.find("\n---", 3)
fm = text[3:end] if end != -1 else ""
m = re.search(rf"(?mi)^{re.escape(key)}:\s*(.+)$", fm)
return m.group(1).strip().strip('"') if m else ""


def _set_frontmatter_category(text: str, category: str) -> str:
"""Rewrite (or append) the frontmatter `category:` line. Best-effort: leaves
the text untouched if there's no frontmatter block."""
if not text.startswith("---"):
return text
end = text.find("\n---", 3)
if end == -1:
return text
fm = text[3:end]
rest = text[end:]
new_line = f"category: {json.dumps(category, ensure_ascii=False)}"
if re.search(r"(?mi)^category:\s*.*$", fm):
fm = re.sub(r"(?mi)^category:\s*.*$", new_line, fm, count=1)
else:
fm = fm.rstrip("\n") + "\n" + new_line + "\n"
return "---" + fm + rest


def get_note(relpath: str) -> dict:
"""Return one note's full content for the dashboard reader:
{ok, title, category, body, source, relpath}. ok=False if not found."""
safe = _safe_relpath(relpath)
target = _vault_subpath(*safe.split("/"))
if not target.exists() or target.suffix.lower() != ".md":
return {"ok": False, "error": "note not found"}
text = target.read_text(encoding="utf-8", errors="replace")
title, body = _split_frontmatter_title(text)
category = str(Path(safe).parent).replace("\\", "/")
if category in (".", ""):
category = INBOX
return {
"ok": True,
"title": title or target.stem,
"category": category,
"body": body,
"source": _frontmatter_field(text, "source"),
"relpath": safe,
}


def move_note(relpath: str, category: str) -> dict:
"""Re-file a note into a different bucket folder, updating its frontmatter
`category`. Returns {ok, relpath, category}."""
safe = _safe_relpath(relpath)
src = _vault_subpath(*safe.split("/"))
if not src.exists():
return {"ok": False, "error": "note not found"}
dest_cat = _safe_category(category)
dest_dir = _vault_subpath(dest_cat)
dest = _vault_subpath(dest_cat, src.name)
if dest.resolve() == src.resolve():
return {"ok": True, "relpath": safe, "category": dest_cat} # already there
_ensure_dir(dest_dir)
if dest.exists():
dest = _vault_subpath(dest_cat, f"{src.stem}-{uuid.uuid4().hex[:6]}{src.suffix}")
text = _set_frontmatter_category(src.read_text(encoding="utf-8", errors="replace"), dest_cat)
dest.write_text(text, encoding="utf-8")
src.unlink()
Comment on lines +600 to +601

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Make pending moves update the capture worker

When this runs on a newly captured note whose _categorize_in_background thread is still running (common for URL notes while fetch/LLM work is in progress), it copies the stub to the new bucket and unlinks the original, but the background worker still has stub_path and later writes a final note via _write_note(...) even if the stub was moved. The result is two notes: the moved (categorizing…) stub plus the final LLM-filed note, so the user's re-file is not authoritative. Please coordinate with the worker, such as by recording the moved path or marking the stub as user-managed before removing it.

Useful? React with 👍 / 👎.

vault = _vault_dir().resolve()
try:
new_rel = str(dest.relative_to(vault)).replace("\\", "/")
except ValueError:
new_rel = dest.name
return {"ok": True, "relpath": new_rel, "category": dest_cat}


def delete_note(relpath: str) -> dict:
"""Delete a note from the vault. Returns {ok, deleted}."""
safe = _safe_relpath(relpath)
target = _vault_subpath(*safe.split("/"))
if not target.exists():
return {"ok": False, "error": "note not found"}
target.unlink()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Make pending deletes cancel background categorization

When deleting a freshly captured note before its _categorize_in_background thread finishes, especially for URL captures, this only removes the inbox stub; the worker still has the note contents and later writes a final note with _write_note(...). The deleted note therefore reappears in another bucket after the user removed it. Please record a cancellation/tombstone or have the worker stop when the stub is gone.

Useful? React with 👍 / 👎.

return {"ok": True, "deleted": True}


# ---------- File writing -----------------------------------------------------

def _yaml_frontmatter(d: dict) -> str:
Expand Down
94 changes: 92 additions & 2 deletions scripts/ui/web/app.js
Original file line number Diff line number Diff line change
Expand Up @@ -297,19 +297,46 @@ async function loadHistory() {

// Browse the vault: empty query lists the newest notes (notes_list); a query
// runs the ranked search (note_search) and shows snippets instead of dates.
let notesCategories = []; // buckets from config, for the reader's Move dropdown
let currentNoteRelpath = ""; // note open in the reader pane

// Render the notes table with clickable rows that open the reader. `col3` maps
// a result row to its third-column text (snippet for search, modified for list).
function renderNotesTable(results, col3) {
const body = $("notes-body");
body.replaceChildren();
for (const r of results) {
const tr = document.createElement("tr");
for (const cell of [r.title, r.category, col3(r)]) {
const td = document.createElement("td");
td.textContent = cell || "";
tr.append(td);
}
if (r.relpath) {
tr.classList.add("note-row");
tr.tabIndex = 0;
const open = () => openNoteReader(r.relpath);
tr.addEventListener("click", open);
tr.addEventListener("keydown", (e) => { if (e.key === "Enter") open(); });
}
body.append(tr);
}
return results.length;
}

async function browseNotes() {
const query = $("note-query").value.trim();
try {
if (query) {
const res = await action("note_search", { query, limit: 20 });
$("notes-col3").textContent = "Snippet";
const n = fillTable("notes-body", (res.results || []).map((r) => [r.title, r.category, r.snippet || ""]));
const n = renderNotesTable(res.results || [], (r) => r.snippet || "");
$("notes-count").textContent = `(${res.count} match${res.count === 1 ? "" : "es"})`;
$("notes-empty").hidden = n > 0;
} else {
const res = await action("notes_list", { limit: 20 });
$("notes-col3").textContent = "Modified";
const n = fillTable("notes-body", (res.results || []).map((r) => [r.title, r.category, r.modified]));
const n = renderNotesTable(res.results || [], (r) => r.modified);
$("notes-count").textContent = `(${res.count} total — newest 20)`;
$("notes-empty").hidden = n > 0;
}
Expand All @@ -319,11 +346,71 @@ async function browseNotes() {
}
}

async function openNoteReader(relpath) {
try {
const n = await action("note_get", { relpath });
if (!n.ok) { setStatus("nr-status", n.error || "note not found", false); return; }
currentNoteRelpath = n.relpath || relpath;
$("nr-title").textContent = n.title || "(untitled)";
$("nr-body").textContent = n.body || "";
const src = $("nr-source");
if (n.source) { src.textContent = n.source; src.href = n.source; src.hidden = false; }
else { src.hidden = true; src.removeAttribute("href"); }
// Bucket dropdown: configured categories + inbox, plus the note's current
// category if it isn't in the list.
const cats = [...notesCategories];
if (!cats.includes("inbox")) cats.push("inbox");
if (n.category && !cats.includes(n.category)) cats.unshift(n.category);
const sel = $("nr-bucket");
sel.replaceChildren();
for (const c of cats) {
const o = document.createElement("option");
o.value = c; o.textContent = c;
if (c === n.category) o.selected = true;
sel.append(o);
}
setStatus("nr-status", "");
$("note-reader").hidden = false;
$("note-reader").scrollIntoView({ behavior: "smooth", block: "nearest" });
} catch (e) {
setStatus("nr-status", `Open failed: ${e.message}`, false);
}
}

async function moveNoteToBucket() {
if (!currentNoteRelpath) return;
const category = $("nr-bucket").value;
try {
const res = await action("note_move", { relpath: currentNoteRelpath, category });
if (!res.ok) { setStatus("nr-status", res.error || "move failed", false); return; }
currentNoteRelpath = res.relpath || currentNoteRelpath;
setStatus("nr-status", `Moved to ${res.category}.`);
browseNotes();
} catch (e) {
setStatus("nr-status", `Move failed: ${e.message}`, false);
}
}

async function deleteCurrentNote() {
if (!currentNoteRelpath) return;
if (!confirm("Delete this note from the vault?")) return;
try {
const res = await action("note_delete", { relpath: currentNoteRelpath });
if (!res.ok) { setStatus("nr-status", res.error || "delete failed", false); return; }
$("note-reader").hidden = true;
currentNoteRelpath = "";
browseNotes();
} catch (e) {
setStatus("nr-status", `Delete failed: ${e.message}`, false);
}
}

async function loadNotes() {
browseNotes();
try {
const cfg = await action("config_snapshot");
const notes = cfg.notes || {};
notesCategories = notes.categories || [];
$("notes-vault").value = notes.vault_dir || "";
$("notes-categories").value = (notes.categories || []).join("\n");
$("notes-fetch-timeout").value = notes.fetch_timeout_seconds ?? 8;
Expand Down Expand Up @@ -1013,6 +1100,9 @@ document.addEventListener("DOMContentLoaded", () => {
});
$("notes-save").addEventListener("click", saveNotes);
$("notes-revert").addEventListener("click", loadNotes);
$("nr-move").addEventListener("click", moveNoteToBucket);
$("nr-delete").addEventListener("click", deleteCurrentNote);
$("nr-close").addEventListener("click", () => { $("note-reader").hidden = true; });
$("config-save").addEventListener("click", saveConfig);
$("config-revert").addEventListener("click", loadConfig);
$("cm-select").addEventListener("change", fillCustomModeForm);
Expand Down
16 changes: 16 additions & 0 deletions scripts/ui/web/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,22 @@ <h2>Your notes <span class="muted small" id="notes-count"></span></h2>
<tbody id="notes-body"></tbody>
</table>
<p class="muted small" id="notes-empty" hidden>No notes found.</p>
<p class="muted small">Click a note to read it, move it to another bucket, or delete it.</p>
</div>
<div class="card card-wide note-reader" id="note-reader" hidden>
<div class="card-actions reader-head">
<h2 id="nr-title" class="nr-title">Note</h2>
<button class="btn" id="nr-close" title="Close">✕</button>
</div>
<p class="muted small"><a id="nr-source" href="#" target="_blank" rel="noopener" hidden></a></p>
<div class="form-row">
<label for="nr-bucket">Bucket</label>
<select id="nr-bucket"></select>
<button class="btn" id="nr-move">Move</button>
<button class="btn" id="nr-delete">Delete…</button>
<span class="muted small" id="nr-status"></span>
</div>
<pre class="note-body" id="nr-body"></pre>
</div>
<div class="card card-wide">
<h2>Vault</h2>
Expand Down
12 changes: 12 additions & 0 deletions scripts/ui/web/styles.css
Original file line number Diff line number Diff line change
Expand Up @@ -544,3 +544,15 @@ footer { padding: 14px 24px 22px; }
.chat-msg-assistant { align-self: flex-start; background: var(--surface); border: 1px solid var(--border); border-bottom-left-radius: 5px; }
.chat-input-row { align-items: stretch; margin-top: 8px; }
.chat-input-row textarea { flex: 1; resize: vertical; font: inherit; }

/* ---- Notes reader ---- */
.note-row { cursor: pointer; }
.note-row:hover, .note-row:focus { background: var(--accent-soft); outline: none; }
.reader-head { align-items: center; }
.nr-title { margin: 0; flex: 1; }
.note-body {
white-space: pre-wrap; word-wrap: break-word; font: inherit;
max-height: 52vh; overflow-y: auto; margin: 8px 0 0;
background: var(--surface); border: 1px solid var(--border);
border-radius: 8px; padding: 12px;
}
8 changes: 5 additions & 3 deletions tests/test_ffp_daemon.py
Original file line number Diff line number Diff line change
Expand Up @@ -68,10 +68,12 @@ def test_actions_count_and_expected_names(daemon_module):
# provider work added provider_status -> 52; model_recommendations -> 53;
# web chat backend added chat_threads_list/chat_thread_get/chat_send/
# chat_thread_delete/chat_stage_selection/chat_take_staged -> 59; retiring the
# tkinter popup removed chat_send_selection/chat_reload/chat_restart -> 56.
assert len(daemon_module.ACTIONS) == 56
# tkinter popup removed chat_send_selection/chat_reload/chat_restart -> 56;
# richer notes view added note_get/note_move/note_delete -> 59.
assert len(daemon_module.ACTIONS) == 59
for a in ("chat_threads_list", "chat_thread_get", "chat_send",
"chat_thread_delete", "chat_stage_selection", "chat_take_staged"):
"chat_thread_delete", "chat_stage_selection", "chat_take_staged",
"note_get", "note_move", "note_delete"):
assert a in daemon_module.ACTIONS
# popup-era socket actions are gone (chat is daemon-backed now)
for a in ("chat_send_selection", "chat_reload", "chat_restart"):
Expand Down
Loading