feat: Implement App Agent Discovery System

[esafwan]

This PR implements the complete App Agent Discovery system that enables external Frappe apps to declare AI capabilities using plain JSON files in their repositories.

Overview

The App Agent Discovery system allows any Frappe app to define AI agents, tools, prompts, providers, models, knowledge sources, and triggers in a huf/ directory. When HUF is installed, these definitions are automatically discovered, validated, and synced into HUF DocTypes.

Key Features Implemented

1. Core Discovery System

• Automatic discovery from //huf/ directories
• Support for 7 definition types: agents, tools, prompts, providers, models, knowledge, triggers
• Proper dependency order during import (providers → models → prompts → tools → knowledge → agents → triggers)
• Integration with after_migrate and after_install hooks

2. Comprehensive Test Suite (873 lines)

• Full test coverage for all modules: loader, validator, normaliser, importer, exporter, cache
• Unit tests for all 7 definition types
• Integration tests for end-to-end discovery workflow
• Version comparison tests

3. Example Definition Files (16 files)

• Agents: CRM Lead Assistant, Simple Support Bot
• Tools: Create Lead, Get Recent Leads, Weather API
• Prompts: Lead Management, Customer Support
• Providers: OpenAI, Anthropic
• Models: GPT-4o-mini, Claude 3.5 Sonnet
• Knowledge: Sales Playbook, Product Documentation
• Triggers: Doc Event, Schedule, Webhook

4. Circular Dependency Detection

• Prevents infinite loops in agent-tool-agent chains
• DFS-based cycle detection algorithm
• Logs warnings when cycles are detected
• Automatically omits cycle-causing references

5. Orphan Detection & Cleanup

• Detects definitions whose source files no longer exist
• New API endpoints for orphan management:
  • get_orphaned_definitions() - List orphans
  • cleanup_orphaned_definitions() - Remove orphans (with dry-run)
  • discover_app_definitions_with_cleanup() - Auto-cleanup during discovery

Files Changed

Modified:

• huf/ai/app_registry/init.py - Updated exports
• huf/ai/app_registry/discovery.py - Added orphan detection/cleanup (+180 lines)
• huf/ai/app_registry/importer.py - Added version comparison (+98 lines)
• huf/ai/app_registry/validator.py - Added circular dependency detection (+90 lines)

Created:

• huf/ai/app_registry/test_app_registry.py - Comprehensive test suite
• huf/ai/app_registry/README.md - Complete documentation
• IMPLEMENTATION_SUMMARY.md - Implementation summary
• huf/ai/app_registry/examples/ - 16 example definition files

Technical Details

Architecture:

huf/ai/app_registry/
├── init.py # Public API exports
├── discovery.py # Orchestration and orphan management
├── loader.py # File scanning and JSON loading
├── validator.py # Type-specific validation + circular detection
├── normaliser.py # Payload transformation
├── importer.py # DocType upsert + version comparison
├── exporter.py # Export to JSON
├── cache.py # File hash-based caching
├── test_app_registry.py # Test suite
├── README.md # Documentation
└── examples/ # 16 example files

Key APIs:

Basic discovery

discover_app_definitions()

With orphan cleanup

discover_app_definitions_with_cleanup(cleanup_orphans=True)

Export

export_agent_bundle("Agent Name")

Orphan management

get_orphaned_definitions()
cleanup_orphaned_definitions(dry_run=False)

Testing

• All Python files compile successfully
• All example JSON files validated
• Test suite covers all modules and edge cases
• Integration tests verify end-to-end workflow

Usage Example

In your app's huf/agents/my_agent.agent.json:

{
  "type": "agent",
  "agent_name": "My Assistant",
  "provider": "openai",
  "model": "gpt-4o-mini",
  "instructions": "You are a helpful assistant...",
  "tools": ["myapp.my_tool"],
  "version": "1.0"
}

After bench migrate, the agent is automatically available in HUF

Notes

• Backward Compatible: Coexists with existing huf_tools hook
• Zero Config: No hooks.py changes required
• Secure: No secrets in definition files, function paths validated
• Resilient: Errors never break migration
• Cached: File hash-based caching for performance