Skip to content

feat: add 'none' LLM provider for chunk-only storage mode#691

Merged
nicoloboschi merged 1 commit intomainfrom
feature/no-llm-provider
Mar 25, 2026
Merged

feat: add 'none' LLM provider for chunk-only storage mode#691
nicoloboschi merged 1 commit intomainfrom
feature/no-llm-provider

Conversation

@nicoloboschi
Copy link
Collaborator

Summary

  • Adds a none LLM provider option for running Hindsight as a chunk store with semantic search but without any LLM dependency
  • Replaces the workaround of setting HINDSIGHT_API_LLM_PROVIDER=mock for chunks-only mode
  • When provider is none: retain auto-uses chunks mode, recall works normally, reflect returns HTTP 400, consolidation/observations are disabled

Changes

  • providers/none_llm.py (new): NoneLLM provider class + LLMNotAvailableError exception
  • config.py: Forces retain_extraction_mode=chunks and enable_observations=false when provider is none
  • llm_wrapper.py: Registers none as a valid provider (no API key required)
  • memory_engine.py: Guards on reflect, consolidation, mental model refresh; forces chunks mode in per-bank config
  • http.py: Catches LLMNotAvailableError → HTTP 400 on reflect and mental model refresh
  • configuration.md: Documents the none provider with usage example
  • test_none_llm_provider.py (new): 12 tests covering unit, config, integration, and HTTP API behavior

Test plan

  • NoneLLM.call() and call_with_tools() raise LLMNotAvailableError
  • NoneLLM.verify_connection() succeeds (no-op)
  • Config validation forces chunks mode and disables observations
  • Retain works (stores chunks without LLM calls)
  • Recall works (semantic search via embeddings)
  • Reflect raises LLMNotAvailableError at engine level
  • Consolidation is skipped with {skipped: true}
  • Mental model refresh raises LLMNotAvailableError
  • HTTP reflect endpoint returns 400
  • HTTP retain/recall endpoints work normally

Adds a proper 'none' provider option so users can run Hindsight as a
chunk store with semantic search but without any LLM dependency, replacing
the hacky workaround of setting provider to 'mock'.

When HINDSIGHT_API_LLM_PROVIDER=none:
- Retain automatically uses chunks mode (no fact extraction)
- Recall works normally (semantic search, BM25, graph retrieval)
- Reflect returns HTTP 400 with clear error message
- Consolidation/observations are disabled
- Mental model refresh returns HTTP 400
- No API key required
@nicoloboschi nicoloboschi merged commit 9e5a066 into main Mar 25, 2026
41 of 43 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant