Go Worker Pools Example

A simple implementation of the worker pool pattern in Go, demonstrating concurrent task processing with a fixed number of workers.

Overview

This example shows how to:

• Create a pool of worker goroutines
• Distribute tasks among workers
• Process tasks concurrently
• Collect and handle results
• Gracefully shut down workers

Worker Pool Architecture

sequenceDiagram
    participant M as Main
    participant P as WorkerPool
    participant W1 as Worker 1
    participant W2 as Worker 2
    participant W3 as Worker 3
    participant R as Results

    %% Initialization Phase
    M->>P: Create WorkerPool(3 workers)
    activate P
    P->>W1: Start Worker 1
    P->>W2: Start Worker 2
    P->>W3: Start Worker 3
    
    %% Task Processing Phase
    rect rgb(245, 245, 245)
        Note over M: Task Submission Loop
        M->>P: Submit Task 1
        P->>W1: Assign Task 1
        M->>P: Submit Task 2
        P->>W2: Assign Task 2
        M->>P: Submit Task 3
        P->>W3: Assign Task 3
        M->>P: Submit Task 4
        P->>W1: Assign Task 4 (when free)
        M->>P: Submit Task 5
        P->>W2: Assign Task 5 (when free)
    end

    %% Result Processing
    W1->>R: Send Result 1
    W2->>R: Send Result 2
    W3->>R: Send Result 3
    W1->>R: Send Result 4
    W2->>R: Send Result 5
    R-->>M: Process Result 1
    R-->>M: Process Result 2
    R-->>M: Process Result 3
    R-->>M: Process Result 4
    R-->>M: Process Result 5

    %% Shutdown Phase
    M->>P: WaitAndClose()
    P->>W1: Stop (after current task)
    P->>W2: Stop (after current task)
    P->>W3: Stop (after current task)
    W1-->>P: Worker 1 stopped
    W2-->>P: Worker 2 stopped
    W3-->>P: Worker 3 stopped
    deactivate P
    M-->>M: All tasks completed

Loading

Features

• Configurable number of workers
• Concurrent task processing
• Thread-safe task distribution
• Graceful shutdown
• Result collection

Usage

go run main.go

Output Example

2025/10/09 10:22:29 Submitting 5 tasks to 3 workers...
2025/10/09 10:22:29 Worker 2 processing task 1: Task-1-data
2025/10/09 10:22:29 Worker 3 processing task 3: Task-3-data
2025/10/09 10:22:29 Worker 1 processing task 2: Task-2-data
2025/10/09 10:22:29 Task 1 completed. Result: Processed: Task-1-data
2025/10/09 10:22:29 Worker 2 processing task 4: Task-4-data
2025/10/09 10:22:29 Task 3 completed. Result: Processed: Task-3-data
2025/10/09 10:22:29 Worker 3 processing task 5: Task-5-data
2025/10/09 10:22:29 Task 2 completed. Result: Processed: Task-2-data
2025/10/09 10:22:29 Worker 1 shutting down
2025/10/09 10:22:30 Task 4 completed. Result: Processed: Task-4-data
2025/10/09 10:22:30 Worker 2 shutting down
2025/10/09 10:22:30 Task 5 completed. Result: Processed: Task-5-data
2025/10/09 10:22:30 Worker 3 shutting down
2025/10/09 10:22:30 All tasks completed. Worker pool shut down.

License

MIT License - See LICENSE for details.