Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,9 @@ datasUFO
# Ignore mkdocs build output
documents/site/

# Ignore frontend environment files with auto-generated content
galaxy/webui/frontend/.env.development.local

# Ignore config files with sensitive data (API keys)
config/*/agents.yaml
config/*/agent.yaml
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
![Python Version](https://img.shields.io/badge/Python-3776AB?&logo=python&logoColor=white-blue&label=3.10%20%7C%203.11) 
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) 
[![Documentation](https://img.shields.io/badge/Documentation-%230ABAB5?style=flat&logo=readthedocs&logoColor=black)](https://microsoft.github.io/UFO/) 
[![YouTube](https://img.shields.io/badge/YouTube-white?logo=youtube&logoColor=%23FF0000)](https://www.youtube.com/watch?v=QT_OhygMVXU) 
[![YouTube](https://img.shields.io/badge/YouTube-white?logo=youtube&logoColor=%23FF0000)](https://www.youtube.com/watch?v=NGrVWGcJL8o) 


</div>
Expand Down
2 changes: 1 addition & 1 deletion README_ZH.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
![Python Version](https://img.shields.io/badge/Python-3776AB?&logo=python&logoColor=white-blue&label=3.10%20%7C%203.11)&ensp;
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)&ensp;
[![Documentation](https://img.shields.io/badge/Documentation-%230ABAB5?style=flat&logo=readthedocs&logoColor=black)](https://microsoft.github.io/UFO/)&ensp;
[![YouTube](https://img.shields.io/badge/YouTube-white?logo=youtube&logoColor=%23FF0000)](https://www.youtube.com/watch?v=QT_OhygMVXU)&ensp;
[![YouTube](https://img.shields.io/badge/YouTube-white?logo=youtube&logoColor=%23FF0000)](https://www.youtube.com/watch?v=NGrVWGcJL8o)&ensp;
</div>

<p align="center">
Expand Down
64 changes: 33 additions & 31 deletions documents/docs/aip/endpoints.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
# AIP Endpoints

!!!quote "Complete Client/Server Implementations"
Endpoints combine protocol, transport, and resilience components to provide production-ready AIP communication for servers, clients, and orchestrators.
Endpoints combine protocol, transport, and resilience components to provide production-ready AIP communication for servers, clients, and orchestrators.

## Endpoint Types at a Glance

Expand Down Expand Up @@ -38,15 +37,14 @@ graph TB

The dashed arrows indicate capabilities that the base class provides to all subclasses. This inheritance design ensures consistent behavior across all endpoint types while allowing specialization for server, client, and orchestrator roles.

!!!info "Base Endpoint Components"
All endpoints inherit from `AIPEndpoint`, which provides:

- **Protocol**: Message serialization and handling
- **Reconnection Strategy**: Automatic reconnection with backoff
- **Timeout Manager**: Operation timeout management
- **Session Handlers**: Per-session state tracking
**Base Endpoint Components:**

---
All endpoints inherit from `AIPEndpoint`, which provides:

- **Protocol**: Message serialization and handling
- **Reconnection Strategy**: Automatic reconnection with backoff
- **Timeout Manager**: Operation timeout management
- **Session Handlers**: Per-session state tracking

## Base Endpoint: AIPEndpoint

Expand Down Expand Up @@ -83,18 +81,17 @@ await endpoint.stop()

## DeviceServerEndpoint

!!!tip "Server-Side Device Management"
Wraps UFO's server-side WebSocket handler with AIP protocol support for managing multiple device connections simultaneously.
Wraps UFO's server-side WebSocket handler with AIP protocol support for managing multiple device connections simultaneously.

### Configuration

```python
from aip.endpoints import DeviceServerEndpoint

endpoint = DeviceServerEndpoint(
client_manager=client_manager, # WebSocket connection manager
session_manager=session_manager, # Session state manager
local=False # Local vs remote deployment
ws_manager=ws_manager, # WebSocket connection manager
session_manager=session_manager, # Session state manager
local=False # Local vs remote deployment
)
```

Expand All @@ -105,7 +102,7 @@ from fastapi import FastAPI, WebSocket
from aip.endpoints import DeviceServerEndpoint

app = FastAPI()
endpoint = DeviceServerEndpoint(client_manager, session_manager)
endpoint = DeviceServerEndpoint(ws_manager, session_manager)

@app.websocket("/ws")
async def websocket_route(websocket: WebSocket):
Expand All @@ -122,8 +119,9 @@ async def websocket_route(websocket: WebSocket):
| **Result Aggregation** | Collect and format execution results | Unified response handling |
| **Auto Task Cancellation** | Cancel tasks on disconnect | Prevent orphaned tasks |

!!!success "Backward Compatibility"
The Device Server Endpoint maintains full compatibility with UFO's existing WebSocket handler.
**Backward Compatibility:**

The Device Server Endpoint maintains full compatibility with UFO's existing WebSocket handler.

### Task Cancellation on Disconnection

Expand All @@ -139,8 +137,7 @@ await endpoint.cancel_device_tasks(

## DeviceClientEndpoint

!!!tip "Client-Side Device Operations"
Wraps UFO's client-side WebSocket client with AIP protocol support, automatic reconnection, and heartbeat management.
Wraps UFO's client-side WebSocket client with AIP protocol support, automatic reconnection, and heartbeat management.

### Configuration

Expand All @@ -159,7 +156,7 @@ endpoint = DeviceClientEndpoint(

| Feature | Default Behavior | Configuration |
|---------|------------------|---------------|
| **Heartbeat** | Starts on connection | 20s interval (configurable) |
| **Heartbeat** | Starts on connection | 20s interval (fixed) |
| **Reconnection** | Exponential backoff | `max_retries=3`, `initial_backoff=2.0` |
| **Message Routing** | Auto-routes to UFO client | Handled internally |
| **Connection Management** | Auto-connect on start | Transparent to user |
Expand Down Expand Up @@ -199,8 +196,7 @@ endpoint = DeviceClientEndpoint(

## ConstellationEndpoint

!!!tip "Orchestrator-Side Multi-Device Coordination"
Enables the ConstellationClient to communicate with multiple devices simultaneously, managing connections, tasks, and queries.
Enables the ConstellationClient to communicate with multiple devices simultaneously, managing connections, tasks, and queries.

### Configuration

Expand Down Expand Up @@ -234,6 +230,8 @@ connection = await endpoint.connect_to_device(
)
```

Learn more about [AgentProfile configuration](../galaxy/client/device_manager.md) in the Galaxy documentation.

### Sending Tasks

```python
Expand Down Expand Up @@ -398,8 +396,8 @@ await endpoint.stop()

## Resilience Features

!!!success "Built-In Resilience"
All endpoints include automatic reconnection, timeout management, and heartbeat monitoring.
!!!warning "Built-In Resilience"
All endpoints include automatic reconnection, timeout management, and heartbeat monitoring for production reliability.

### Resilience Configuration

Expand Down Expand Up @@ -484,12 +482,13 @@ class CustomEndpoint(DeviceClientEndpoint):

## Best Practices

!!!tip "Endpoint Selection"
| Use Case | Endpoint Type |
|----------|---------------|
| Device agent server | `DeviceServerEndpoint` |
| Device agent client | `DeviceClientEndpoint` |
| Multi-device orchestrator | `ConstellationEndpoint` |
**Endpoint Selection:**

| Use Case | Endpoint Type |
|----------|---------------|
| Device agent server | `DeviceServerEndpoint` |
| Device agent client | `DeviceClientEndpoint` |
| Multi-device orchestrator | `ConstellationEndpoint` |

!!!warning "Configuration Guidelines"
- **Set appropriate timeouts** based on deployment environment
Expand Down Expand Up @@ -539,4 +538,7 @@ from aip.endpoints import (
- [Resilience](./resilience.md) - Reconnection and heartbeat management
- [Messages](./messages.md) - Message types and validation
- [Overview](./overview.md) - System architecture and design
- [Galaxy Client](../galaxy/client/overview.md) - Multi-device orchestration with ConstellationClient
- [UFO Server](../server/websocket_handler.md) - WebSocket server implementation
- [UFO Client](../client/websocket_client.md) - WebSocket client implementation

61 changes: 30 additions & 31 deletions documents/docs/aip/messages.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
# AIP Message Reference

!!!quote "Strongly-Typed Communication"
AIP uses **Pydantic-based messages** for automatic validation, serialization, and type safety. All messages transmit as JSON over WebSocket.
AIP uses **Pydantic-based messages** for automatic validation, serialization, and type safety. All messages transmit as JSON over WebSocket.

## Message Overview

Expand Down Expand Up @@ -59,8 +58,7 @@ Unidirectional arrows indicate request-response patterns, while bidirectional ar

## Core Data Structures

!!!info "Foundation Types"
These Pydantic models form the building blocks for all AIP messages.
These Pydantic models form the building blocks for all AIP messages.

### Essential Types Summary

Expand Down Expand Up @@ -105,8 +103,7 @@ control = ControlInfo(
)
```

<details>
<summary><b>Complete Field List</b></summary>
**Complete Field List:**

| Field | Type | Description |
|-------|------|-------------|
Expand All @@ -123,8 +120,6 @@ control = ControlInfo(
| `source` | str? | Data source identifier |
| `text_content` | str? | Text content |

</details>

### WindowInfo

Window metadata (extends ControlInfo).
Expand All @@ -143,8 +138,9 @@ Window metadata (extends ControlInfo).

MCP tool capability definition.

!!!tip "Tool Advertisement"
Device agents use `MCPToolInfo` to advertise their capabilities during registration.
**Tool Advertisement:**

Device agents use `MCPToolInfo` to advertise their capabilities during registration.

```python
tool_info = MCPToolInfo(
Expand Down Expand Up @@ -174,6 +170,8 @@ tool_info = MCPToolInfo(
| `meta` | dict? | Metadata |
| `annotations` | dict? | Additional annotations |

Learn more about [MCP tools and capabilities](../mcp/overview.md).

---

## Command and Result Structures
Expand All @@ -200,8 +198,9 @@ cmd = Command(
| `tool_type` | str | ✅ | `"data_collection"` or `"action"` |
| `call_id` | str | | Unique identifier for correlation |

!!!tip "Call ID Correlation"
Use `call_id` to match commands with their results in the `Result` object.
**Call ID Correlation:**

Use `call_id` to match commands with their results in the `Result` object.

### ResultStatus

Expand Down Expand Up @@ -281,8 +280,9 @@ The `CONTINUE → CONTINUE` self-loop represents multi-turn execution where task
| `OK` | ✓ Acknowledgment | Heartbeat, health check passed |
| `ERROR` | ⚠️ Protocol error | Protocol-level error |

!!!success "Multi-Turn Execution"
`CONTINUE` enables agents to request additional commands before marking a task as complete, supporting complex multi-step workflows.
**Multi-Turn Execution:**

`CONTINUE` enables agents to request additional commands before marking a task as complete, supporting complex multi-step workflows.

---

Expand Down Expand Up @@ -320,8 +320,7 @@ constellation_msg = ClientMessage(

## ClientMessage (Client → Server)

!!!tip "Client-Initiated Messages"
Devices and constellation clients use `ClientMessage` to communicate with the server.
Devices and constellation clients use `ClientMessage` to communicate with the server.

### Message Types

Expand Down Expand Up @@ -394,8 +393,7 @@ results_msg = ClientMessage(

## ServerMessage (Server → Client)

!!!tip "Server-Initiated Messages"
Device services use `ServerMessage` to assign tasks and send commands to clients.
Device services use `ServerMessage` to assign tasks and send commands to clients.

### Message Types

Expand Down Expand Up @@ -490,8 +488,8 @@ task_end_msg = ServerMessage(

## Message Validation

!!!success "Built-In Validation"
AIP provides `MessageValidator` class for ensuring message integrity.
!!!warning "Built-In Validation"
AIP provides `MessageValidator` class for ensuring message integrity. Always validate messages before processing to prevent protocol errors.

### Validation Methods

Expand Down Expand Up @@ -520,15 +518,11 @@ if MessageValidator.validate_command_results(client_message):
await process_results(client_message)
```

!!!warning "Validation Best Practice"
Always validate messages before processing to prevent protocol errors and ensure data integrity.

---

## Message Correlation

!!!info "Request-Response Chaining"
AIP uses identifier chains to maintain conversation context across multiple message exchanges.
AIP uses identifier chains to maintain conversation context across multiple message exchanges.

### Correlation Pattern

Expand Down Expand Up @@ -561,8 +555,9 @@ Each new request includes `prev_response_id` pointing to the previous server res

### Session Tracking

!!!tip "Session-Based Grouping"
All messages within a task execution share the same `session_id` for traceability.
**Session-Based Grouping:**

All messages within a task execution share the same `session_id` for traceability.

```python
# All messages use same session_id
Expand Down Expand Up @@ -596,10 +591,11 @@ task_end_msg.session_id = SESSION_ID
- Always provide meaningful error messages
- Use `ResultStatus.FAILURE` with descriptive `error` field

!!!tip "Extensibility"
- Use `metadata` field for custom data without breaking protocol
- Leverage Pydantic's validation for type safety
- Always correlate messages with `prev_response_id`
**Extensibility:**

- Use `metadata` field for custom data without breaking protocol
- Leverage Pydantic's validation for type safety
- Always correlate messages with `prev_response_id`

---

Expand Down Expand Up @@ -627,3 +623,6 @@ from aip.messages import (
- [Protocol Guide](./protocols.md) - How protocols construct and use messages
- [Endpoints](./endpoints.md) - How endpoints handle messages
- [Overview](./overview.md) - High-level message flow in system architecture
- [Transport Layer](./transport.md) - WebSocket transport for message delivery
- [Resilience](./resilience.md) - Message retry and timeout handling
- [MCP Integration](../mcp/overview.md) - How MCP tools integrate with AIP messages
Loading