Skip to content

feat: add Tavily as configurable search and crawl provider#2

Open
tavily-integrations wants to merge 1 commit into
RedSearchAgent:zchufrom
Tavily-FDE:feat/tavily-migration/deeptracehub-serper-to-tavily
Open

feat: add Tavily as configurable search and crawl provider#2
tavily-integrations wants to merge 1 commit into
RedSearchAgent:zchufrom
Tavily-FDE:feat/tavily-migration/deeptracehub-serper-to-tavily

Conversation

@tavily-integrations
Copy link
Copy Markdown

Summary

Adds Tavily as an optional, parallel search and content extraction provider alongside the existing Serper.dev (search) and Jina (crawl) tools. No defaults are changed — operators can switch by setting search_tool: tavily and/or crawl_tool: tavily in their YAML config.

What changed

  • tools/search.py: Added TavilySearchTool (subclass of SearchToolBase, name=tavily) using AsyncTavilyClient.search(), and TavilyWebCrawlTool (subclass of WebCrawlToolBase, name=tavily) using AsyncTavilyClient.extract(). Both map results to existing OrganicResults/SerpEntry/WebContentV2 dataclasses with proper provenance metadata.
  • tools/types.py: Added TAVILY = "tavily" to both SerpProvider and WebProvider enums.
  • src/agents/base.py: Imported and registered TavilySearchTool and TavilyWebCrawlTool in TOOL_CLASS_SEARCH / TOOL_CLASS_CRAWL lists.
  • config/redsearcher.yaml and config/deepseek.yaml: Added inline comments documenting tavily as an alternative value for search_tool and crawl_tool.
  • .env: Added TAVILY_API_KEY="" placeholder.
  • requirements.txt: Added tavily-python>=0.5.0.

Files changed

  • tools/search.py
  • tools/types.py
  • src/agents/base.py
  • config/redsearcher.yaml
  • config/deepseek.yaml
  • .env
  • requirements.txt

Dependency changes

  • Added tavily-python>=0.5.0 to requirements.txt

Environment variable changes

  • Added TAVILY_API_KEY (read by TavilySearchTool and TavilyWebCrawlTool via AsyncTavilyClient)

Notes for reviewers

  • This is an additive migration — all existing Serper.dev and Jina code is untouched
  • Tavily client is lazily imported inside methods to avoid import errors when the package isn't installed
  • Both tools use the existing retry, caching, and registry infrastructure
  • To activate: set search_tool: tavily and/or crawl_tool: tavily in your config YAML and provide TAVILY_API_KEY

🤖 Generated with Claude Code

Automated Review

  • Passed after 1 attempt(s)
  • Final review: The additive Tavily migration is correct and well-scoped. Both TavilySearchTool and TavilyWebCrawlTool conform to the SearchToolBase/WebCrawlToolBase interfaces, use the correct Tavily SDK methods (AsyncTavilyClient.search() and .extract()), properly map response fields to the existing SerpEntry/WebContentV2 types, and follow existing patterns for caching and retry logic. Enum values, registrations, dependency, env placeholder, and config comments are all present. Existing Serper/Jina tooling is fully intact. Three minor issues noted below, none blocking.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant