Skip to content

Fix find_files slowness with SQLite FindIndex (#354).#366

Open
false200 wants to merge 1 commit into
CraftOS-dev:V1.4.1from
false200:fix/find-files-index-354
Open

Fix find_files slowness with SQLite FindIndex (#354).#366
false200 wants to merge 1 commit into
CraftOS-dev:V1.4.1from
false200:fix/find-files-index-354

Conversation

@false200

@false200 false200 commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

Fixes #354

So basically I added a new file, app/utils/file_index.py, that builds a SQLite index of filenames (FTS5 trigram on the basename). It lives at {base_directory}/.craftbot/findindex.db.

From the outside, find_files behaves the same:

  • empty base_directory still goes to home
  • the agent can still search wherever it wants
  • we still crawl everything, including stuff like $Recycle.Bin
  • the response is still just status, matches, and message - all matches, no head_limit

The difference is recursive searches hit the index instead of running os.walk every single time. Non recursive is unchanged - still just a top level glob.

After the first build, watchdog keeps the index updated with a 5 second debounce.

Also added **/.craftbot/ to .gitignore so those runtime DB files don't get committed.

Replace per-call os.walk with a cached filename index and watchdog updates while keeping existing path semantics and output shape.
@false200

false200 commented Jul 4, 2026

Copy link
Copy Markdown
Contributor Author

@makiroll1125 I've pushed an improvement for this issue. There isn't really a single best solution since each approach has tradeoffs, but this should make search work better than the previous one. Would appreciate a review.

@makiroll1125 makiroll1125 self-requested a review July 6, 2026 00:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant