Chat responses are slow and may never appear

[chubes4]

Problem

Chat agent responses are very slow (30+ seconds) and sometimes fail to appear, despite existing loading indicators.

Root Cause Analysis

• AI conversation loops take excessive time based on prompt complexity
• No timeout handling for long-running requests
• Network issues or provider delays not gracefully handled
• Race conditions between mutation response and session queries

Steps to Reproduce

1. Send complex message requiring extensive AI processing
2. Observe long wait times before response appears
3. Sometimes response never loads despite loading indicators

Expected Behavior

• Responses within reasonable time limits (<15 seconds for complex requests)
• Proper timeout handling with user feedback
• Reliable delivery of responses

Technical Details

• API: POST /datamachine/v1/chat with AIConversationLoop processing
• Frontend: Loading states exist but responses still slow/fail
• Backend: No request timeouts, potential race conditions

Proposed Solutions

• Add configurable timeout limits for AI requests
• Implement retry logic for failed requests
• Optimize AI loop performance (caching, prompt optimization)
• Add server-side request queuing to prevent overload
• Better error propagation to frontend