refactor: AI SDK Tools#198
Draft
esafwan wants to merge 38 commits into
Draft
Conversation
fix: harden tool creation functions for migration safety
feat: Update Project documentation
…nd add model field for STT
- Implement _resolve_stt_config helper to robustly resolve STT models and API keys,mirroring the architecture of _resolve_tts_config. - Update handle_transcribe_audio logic to actively save the identified stt_model to the generated 'Agent Message' database record for better tracking. - Add '_get_default_stt_model' to cleanly handle provider-specific model fallbacks.
feat: refactor audio transcription to support cross-provider overrides
…sponses - Fixed early exit bug in 'litellm.run_stream' upon receiving the 'stop' finish reason, allowing the stream to fully process and capture OpenAI's delayed usage metadata chunk. - Improved 'cached_tokens' extraction in agent_integration.py to support LiteLLM's internal 'cache_hit_tokens' alias. - Fixed a scoping bug in run_agent_stream where cost calculation and conversation DB metrics updates were incorrectly nested inside a fallback block.
- Fix token extraction in agent_integration.py sync & stream to handle explicit None values. - Fix litellm.py tracking logic to prevent '+= None' operations. - Ensure 'cached_tokens' always falls back to 0 to prevent SQL cannot be null constraints and comparison errors in cost calculation.
Fix: Resolve Gemini Prompt Caching & Streaming Cost Accumulation
fix: audio transcription issue in gemini
Feat/agent voice example
Comprehensive analysis and plan to break the 2,682-line sdk_tools.py god module into focused, testable modules. Key changes: extract CRUD handlers, media handlers, and conversation data handlers into separate files; create a single source of truth for tool-type mappings; eliminate ~200 lines of duplicated media boilerplate; fix dead code, duplicate definitions, and filename typos. https://claude.ai/code/session_01HQEq8q8LJ2H6XJF9AGYGEo
The huf/ai/ directory was missing an __init__.py file, making it not a proper Python package. Sub-packages (knowledge/, providers/, orchestration/) already had __init__.py files. Adding this ensures consistent import behavior and allows tools like pytest to discover tests correctly. https://claude.ai/code/session_01HQEq8q8LJ2H6XJF9AGYGEo
…tions.py The function was defined twice (lines 359 and 461). Python silently overwrites the first definition with the second, making the first one dead code. The second definition (which accepts **kwargs for multi-file attachment) is the correct and actively used version. https://claude.ai/code/session_01HQEq8q8LJ2H6XJF9AGYGEo
The filename had a typo ("cilent" instead of "client"). Updated the
corresponding function_path reference in sdk_tools.py to match.
https://claude.ai/code/session_01HQEq8q8LJ2H6XJF9AGYGEo
Move all document CRUD handler functions (_sanitize_for_doctype, handle_create_document, handle_delete_document, handle_get_list, handle_update_document, handle_get_document, handle_*_documents, handle_submit/cancel_document, handle_get/set_value, handle_get_report_result, handle_attach_file_to_document) from sdk_tools.py into a dedicated handlers/crud.py module. Also removes unused factory functions (create_get_function, create_create_function, create_update_function, create_delete_function, create_list_function, wrap_frappe_function) that were never called — create_agent_tools() uses the handle_* pattern exclusively. Backward-compatible re-exports are kept in sdk_tools.py so existing function_path values in the database continue to resolve. https://claude.ai/code/session_01HQEq8q8LJ2H6XJF9AGYGEo
Move conversation data handlers (_now_iso_utc, _load_state, handle_get/set/load_conversation_data) from sdk_tools.py into huf/ai/handlers/conversation_data.py. Delete huf/ai/conversation_data_tools.py which was a redundant copy of the same functions already present in sdk_tools.py. Backward-compatible re-exports kept in sdk_tools.py. https://claude.ai/code/session_01HQEq8q8LJ2H6XJF9AGYGEo
…r.py Move handle_run_agent() from sdk_tools.py into a dedicated module. This handler has a unique dependency on frappe.utils.background_jobs.enqueue that no other handler needs — keeping it separate reduces coupling. Backward-compatible re-export kept in sdk_tools.py. https://claude.ai/code/session_01HQEq8q8LJ2H6XJF9AGYGEo
Create huf/ai/media_utils.py with shared helpers that eliminate ~200
lines of duplicated boilerplate across image/audio/OCR handlers:
- get_agent_provider_config(): load agent, provider, API key
- get_next_conversation_index(): sequential message ordering
- create_agent_message(): Agent Message document creation
- save_media_file(): file save with message/conversation attachment
- emit_message_event(): realtime socket event publishing
- update_conversation_total_messages(): conversation counter update
Extract all media handlers to huf/ai/handlers/media.py:
- handle_generate_image (image generation via LiteLLM)
- handle_ocr_document (OCR via LiteLLM OCR/vision endpoints)
- handle_generate_audio (TTS via LiteLLM speech)
- handle_transcribe_audio (STT via LiteLLM transcription)
- All associated helper/config functions (_resolve_tts_config, etc.)
Also fixes: replace print() with frappe.logger().debug() in image
generation (was print("Returned images: ", images)).
Backward-compatible re-exports kept in sdk_tools.py. The file is now
reduced from 2,682 lines to 465 lines.
https://claude.ai/code/session_01HQEq8q8LJ2H6XJF9AGYGEo
Add huf/ai/tool_types.py as the single source of truth for tool-type- to-handler-path mapping. This eliminates the duplicated if/elif chain that was maintained in both sdk_tools.py and flow_tool_executor.py. Refactor sdk_tools.py: - Replace 40-line if/elif chain with TOOL_TYPE_HANDLERS dict lookup - Replace inline DOCTYPE_BOUND_TYPES list with shared constant - Clean up duplicate imports (base64 x2, save_file x2) - Remove unused imports (io, requests, hashlib, enqueue, client, etc.) - Remove unused datetime/timedelta module-level imports - Remove inline 'import inspect' (use module-level import) - Extract _call_with_filtered_args() helper from on_invoke_tool closure - Standardize frappe.log_error() to use keyword arguments - Use consistent tab indentation throughout - Consolidate all re-exports at bottom of file sdk_tools.py is now ~280 lines of core logic (down from 2,682). https://claude.ai/code/session_01HQEq8q8LJ2H6XJF9AGYGEo
Replace the hardcoded type_to_handler dict and inline DOCTYPE_BOUND_TYPES list with imports from huf.ai.tool_types. This ensures both sdk_tools.py and flow_tool_executor.py share a single source of truth for tool-type mappings — adding a new tool type now requires updating only one file. Also reuse _call_with_filtered_args() from sdk_tools instead of duplicating the inspect-based parameter filtering logic. https://claude.ai/code/session_01HQEq8q8LJ2H6XJF9AGYGEo
Update function_path references in tool creation/migration functions to point to the new canonical handler locations: - huf.ai.sdk_tools.handle_generate_image → huf.ai.handlers.media - huf.ai.sdk_tools.handle_ocr_document → huf.ai.handlers.media - huf.ai.sdk_tools.handle_generate_audio → huf.ai.handlers.media - huf.ai.sdk_tools.handle_transcribe_audio → huf.ai.handlers.media New installations will store the correct paths. Existing installations continue to work via backward-compatible re-exports in sdk_tools.py. https://claude.ai/code/session_01HQEq8q8LJ2H6XJF9AGYGEo
… type -Added import for get_documents from tool_functions. -Implemented handle_get_documents(reference_doctype, document_ids, **kwargs). -Added validation for missing doctype and invalid/empty document_ids. -Standardized success/error response shape to match existing handlers. -Restored compatibility with tool_types.py mapping for 'Get Multiple Documents'.
-Updated success message text from Set '{name}' match successfully to Set '{name}' successfully.
-No behavior or schema change only response clarity/readability improvement.
-Fixed extra-arg injection for HTTP tools to use tool_doc['tool_name'] instead of document name. -Prevents mismatch in handler-side policy/logging logic that depends on the actual tool name. -Keeps deterministic flow tool execution behavior aligned with SDK tool execution path.
-Fixed indentation in on_invoke_tool block that could cause runtime failure. -Removed duplicate context injection path and kept one canonical context extraction using _frappe_run_context_dict(ctx). -Preserved propagation of conversation_id, agent_run_id, and agent_name into tool args. -Kept backward-compatible re-export list intact and added missing CRUD plural handler export.
-Use context values as fallbacks in tool invocation. This fixes self-triggering behavior where an agent incorrectly invoked itself instead of the configured downstream agent. -Merge context with setdefault in on_invoke_tool
-make submit and cancel handler resilient to missing doctype args -return clear validation errors for missing document_id/doctype
- Added link filed for selecting report
- Updated 'create_agent_tools' to load all tools assigned to the agent without filtering. - Updated 'create_function_tool' signature to accept the 'tool_doc'. - Added a strict runtime permission check inside 'on_invoke_tool' using PermissionAwareToolRegistry._can_use_tool.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR performs a comprehensive refactor of the
huf/ai/sdk_tools.pymodule, which had grown into a 2,682-line "god module" with multiple overlapping responsibilities. The refactor improves modularity, removes significant code duplication, and establishes a cleaner architecture for tool handlers.Key Changes
sdk_tools.pyinto specialized handlers inhuf/ai/handlers/:crud.py: Handles tool creation and basic management.conversation_data.py: Manages data extraction and conversation indexing.media.py: Handles Image Generation, TTS, STT, and OCR.agent_runner.py: Orchestrates agent execution logic.tool_types.pyto consolidate tool-type mappings, eliminating identical 20-entryif/elifchains insdk_tools.pyandflow_tool_executor.py.media_utils.py, removing ~200 lines of redundant code across 4 handlers.tool_functions.py.cilent_side_tool.py->client_side_tool.py.conversation_data_tools.py.sdk_tools.pyhas been slimmed down (~250 lines) but maintains re-exports for existing integrations.Impact
sdk_tools.pysize by ~90%.Refactor Phases
flow_tool_executor.py,install.py, andagent_chat.py.