feat: add tool call start and completed streaming events by mycodecrafting · Pull Request #737 · Greybox-Labs/tim

mycodecrafting · 2025-11-05T16:57:25Z

Summary

Add tool execution lifecycle events (start/complete) and refactor tool execution to return structured results with summaries.

Background

Clients need responsive UI updates when tools are being executed. Previously, clients only received notification after a tool call was fully streamed, leaving a gap before execution and no notification when tools completed.

Changes

Added tool_call_start event emitted when tool name is known (before full input)
Added tool_call_complete event emitted when tool execution finishes with success/summary
Added Summary field to ToolResultContent for client notifications
Updated all 26 tool implementations with appropriate execution summaries
Proto: Added summary field to ToolResult, new event types to StreamThreadEventsResponse

linear · 2025-11-05T16:57:27Z

GRE-382 Tool call streaming adjustments for better ux

We initially implemented streaming to client with tool calls only sending complete tool calls and not streaming any deltas. However, there are some tool calls which the LLM uses which to the user are not much different from the LLM responding where we do stream deltas. It creates a bit of an odd UX where some responses from the LLM stream in deltas and some only send the full text.

We should look into streaming text deltas for certain tools, e.g, clarify, work_complete, and inform_user

We may additionally want to include a "tool call start" event so clients can be responsive and show the user something when a tool is known to be going to be used but we don't have the full content for the input yet.

And we should a "tool call complete" event for when we get results for a tool call so that clients know when a system or remote tool has finished. It should not include the full tool results but some kind of summary is probably ideal.

Copilot

Pull Request Overview

This PR adds real-time tool execution lifecycle events to enable responsive UI updates during tool calls. Previously, clients only received notifications after tool streaming completed, creating a gap during tool execution.

Key Changes:

Added tool_call_start event when tool name is known (before full input is ready)
Added tool_call_complete event when tool execution finishes with success/failure status and summary
Added Summary field to tool results for brief client-friendly descriptions

Reviewed Changes

Copilot reviewed 44 out of 44 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
Proto definitions (.proto, .pb.go, .swagger.json, openapi.yaml)	Added `summary` field to `ToolResult`, new `ToolCallStartEvent` and `ToolCallCompleteEvent` message types to streaming response
tim-api/internal/services/llm_response/handlers.go	Generate tool call ID early and emit `tool_call_start` when tool name is available
tim-api/internal/services/llm_response/system_tool_execution.go	Emit `tool_call_complete` event for system tools with execution summary
tim-api/internal/services/tool_execution/handlers.go	Emit `tool_call_complete` event when user tools finish executing
tim-api/internal/natsnotifier/thread_event_notifier.go	Added notification methods for new tool lifecycle events
tim-api/internal/events/thread_events.go	Added event types and payload structures for tool lifecycle events
shared/tools/*.go (26 files)	Updated all tool implementations to return structured `ToolExecutionResult` with summaries
shared/tools/tools.go	Changed tool interface to return `ToolExecutionResult` struct instead of multiple return values
shared/tools/executor.go	Updated executor to handle new structured result format and populate summary field
shared/llm/types.go	Added `Summary` field to `ToolResultContent` struct

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

…2-tool-call-streaming-adjustments

Copilot

Pull Request Overview

Copilot reviewed 44 out of 44 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-11-05T17:27:14Z

 	)

+	// Generate internal tool call ID for tool_use blocks
+	toolCallID := ""


[nitpick] The variable initialization pattern is unnecessarily verbose. Initialize toolCallID directly in the conditional block since it's only used when the condition is true.

Suggested change

toolCallID := ""

var toolCallID string

Copilot · 2025-11-05T17:27:15Z

+	return &ToolExecutionResult{
+		StopIteration: false,
+		Result:        string(jsonResponse),
+		Summary:       fmt.Sprintf("Ran command (exit code %d)", toolResponse["exit_code"]),


Map access toolResponse[\"exit_code\"] may panic if the key doesn't exist. Use type assertion or safer access pattern since the value needs to be converted for fmt formatting.

Suggested change

Summary: fmt.Sprintf("Ran command (exit code %d)", toolResponse["exit_code"]),

Summary: func() string {

if exitCode, ok := toolResponse["exit_code"].(int); ok {

return fmt.Sprintf("Ran command (exit code %d)", exitCode)

}

return "Ran command (exit code unknown)"

}(),

Lol it's explicitly set above. Maybe for type safety clarity makes sense to have toolResponse be a struct with json field annotations?

joroshiba · 2025-11-05T19:38:13Z

+    // A tool call is about to be executed (tool name known, input may be incomplete)
+    ToolCallStartEvent tool_call_start = 4;
+    // A tool call made by the LLM with full input (input may be null if >512KB)
+    tim.api.tool.v1alpha1.ToolCall tool_call = 5;


I think this full ToolCall is here in order for local exec runner to be able to execute actions? It feels a bit out of place where everything else in this is a Event type created specifically for this? Similar to how we don't record full ToolCall Results through the stream, could be awkward in some instances as well?

Maybe we should have some sort of user side event that would only include full parameters on things we expect user side? With the above note we could have a lifecycle:

ToolCallCreated

ToolCallStarted - when the full tool call is known, has a "summary" for user facing information, and perhaps only full parameters when a user or local exec runner would be expected to execute?

ToolCallComplete

Maybe is worth a follow up ticket? Curious your thoughts here

I think this generally make sense. There is a tool call api endpoint that a client can hit to get the full tool call. Right now we don't even pass the full params if they are over a certain size to prevent the events from being too "fat". I am wondering if it also might make sense to just say that, if you are streaming events and see that you need to run a tool, you need to request the tool parameters from the API. Basically what you outlined, except ToolCallStarted would only ever have a summary. Any client listening that believes it should run the tool would need to fetch the full parameters from the API.

ToolCallStarted having a summary would be nice in that currently a client needs to understand the input structure for every single tool just to display something about what's happening to the end user. This would greatly simplify that and a client would only need to be concerned with the input to a tool if they are executing it

joroshiba · 2025-11-05T19:43:50Z

+	return &ToolExecutionResult{
+		StopIteration: false,
+		Result:        string(jsonResponse),
+		Summary:       fmt.Sprintf("Ran command (exit code %d)", toolResponse["exit_code"]),


Lol it's explicitly set above. Maybe for type safety clarity makes sense to have toolResponse be a struct with json field annotations?

joroshiba · 2025-11-05T19:47:54Z

-    // A tool call made by the LLM (input may be null if >512KB)
-    tim.api.tool.v1alpha1.ToolCall tool_call = 4;
+    // A tool call is about to be executed (tool name known, input may be incomplete)
+    ToolCallStartEvent tool_call_start = 4;


Perhaps call this ToolCallCreated since this is not that execution is started or that the call has even be made. Could be a hair confusing.

add tool call start and completed streaming events

97b84cf

Copilot AI review requested due to automatic review settings November 5, 2025 16:57

Copilot AI reviewed Nov 5, 2025

View reviewed changes

Comment thread tim-api/internal/services/llm_response/handlers.go Outdated

Comment thread tim-api/internal/services/llm_response/handlers.go

Comment thread shared/tools/list_processes.go Outdated

mycodecrafting and others added 3 commits November 5, 2025 12:07

Update shared/tools/list_processes.go

e80efbb

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Merge remote-tracking branch 'origin/main' into mycodecrafting/gre-38…

16fd9a7

…2-tool-call-streaming-adjustments

code review suggestions

e8b85d8

mycodecrafting requested a review from Copilot November 5, 2025 17:26

Copilot AI reviewed Nov 5, 2025

View reviewed changes

joroshiba reviewed Nov 5, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add tool call start and completed streaming events#737

feat: add tool call start and completed streaming events#737
mycodecrafting wants to merge 4 commits into
mainfrom
mycodecrafting/gre-382-tool-call-streaming-adjustments

mycodecrafting commented Nov 5, 2025

Uh oh!

linear Bot commented Nov 5, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Nov 5, 2025

Uh oh!

Copilot AI Nov 5, 2025

Uh oh!

joroshiba Nov 5, 2025

Uh oh!

joroshiba Nov 5, 2025

Uh oh!

mycodecrafting Nov 5, 2025

Uh oh!

mycodecrafting Nov 5, 2025

Uh oh!

joroshiba Nov 5, 2025

Uh oh!

joroshiba Nov 5, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

-		Summary:       fmt.Sprintf("Ran command (exit code %d)", toolResponse["exit_code"]),
+		Summary:       func() string {
+			if exitCode, ok := toolResponse["exit_code"].(int); ok {
+				return fmt.Sprintf("Ran command (exit code %d)", exitCode)
+			}
+			return "Ran command (exit code unknown)"
+		}(),

Conversation

mycodecrafting commented Nov 5, 2025

Summary

Background

Changes

Uh oh!

linear Bot commented Nov 5, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Copilot AI Nov 5, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 5, 2025

Choose a reason for hiding this comment

Uh oh!

joroshiba Nov 5, 2025

Choose a reason for hiding this comment

Uh oh!

joroshiba Nov 5, 2025

Choose a reason for hiding this comment

Uh oh!

mycodecrafting Nov 5, 2025

Choose a reason for hiding this comment

Uh oh!

mycodecrafting Nov 5, 2025

Choose a reason for hiding this comment

Uh oh!

joroshiba Nov 5, 2025

Choose a reason for hiding this comment

Uh oh!

joroshiba Nov 5, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants