Hermes Wiki
Developer/Languages/Go/ConcurrencyPatterns/Challenges/design-a-bounded-worker-pool-with-graceful-shutdown

Design a Bounded Worker Pool with Graceful Shutdown

Scenario prompt

You're building a Go service that ingests jobs (e.g., resizing uploaded images) from an unbounded source — a channel fed by an HTTP handler. Naively spawning a goroutine per job works at low volume but under a traffic spike spawns thousands of concurrent goroutines, each holding a large in-memory buffer, and the service OOMs.

Design a worker pool that:

  1. Runs at most N jobs concurrently, regardless of how fast jobs arrive.
  2. Applies backpressure to the submitter when all workers are busy and the queue is full, rather than buffering unboundedly.
  3. On receiving a shutdown signal (context.Context cancellation), stops accepting new jobs, lets in-flight jobs finish (up to a deadline), and returns cleanly — no goroutine leaks, no dropped results silently swallowed.

Write the pool's core type and its Submit/Run/Shutdown surface. State your concurrency primitives (channels vs. sync.WaitGroup vs. errgroup) and justify the choice.

Mihir's attempt

[!todo] Not yet attempted — drop your own implementation here before reading the model solution below.

Model solution

package pool

import (
	"context"
	"fmt"
	"sync"
)

type Job func(ctx context.Context) error

type Pool struct {
	jobs    chan Job
	wg      sync.WaitGroup
	errOnce sync.Once
	firstErr error
}

// New starts a pool of n workers reading from a queue of size queueSize.
// A full queue makes Submit block, which is the backpressure mechanism —
// no unbounded buffering, no silent job dropping.
func New(ctx context.Context, n, queueSize int) *Pool {
	p := &Pool{jobs: make(chan Job, queueSize)}
	for i := 0; i < n; i++ {
		p.wg.Add(1)
		go p.worker(ctx)
	}
	return p
}

func (p *Pool) worker(ctx context.Context) {
	defer p.wg.Done()
	for {
		select {
		case job, ok := <-p.jobs:
			if !ok {
				return // channel closed: no more work, drain complete
			}
			if err := job(ctx); err != nil {
				p.errOnce.Do(func() { p.firstErr = err })
			}
		case <-ctx.Done():
			return // shutdown: stop pulling new jobs, exit immediately
		}
	}
}

// Submit blocks if the queue is full (backpressure) or returns ctx.Err()
// if the context is cancelled while waiting — never blocks forever.
func (p *Pool) Submit(ctx context.Context, j Job) error {
	select {
	case p.jobs <- j:
		return nil
	case <-ctx.Done():
		return ctx.Err()
	}
}

// Shutdown closes the submission channel (no new jobs accepted) and
// waits for in-flight + already-queued jobs to drain, or for the
// caller's context to expire first — whichever comes first.
func (p *Pool) Shutdown(ctx context.Context) error {
	close(p.jobs)
	done := make(chan struct{})
	go func() {
		p.wg.Wait()
		close(done)
	}()
	select {
	case <-done:
		return p.firstErr
	case <-ctx.Done():
		return fmt.Errorf("shutdown deadline exceeded, workers still draining: %w", ctx.Err())
	}
}

Usage:

runCtx, cancel := context.WithCancel(context.Background())
p := pool.New(runCtx, 8, 100) // 8 workers, 100-job buffer

// submit path (e.g., HTTP handlers)
if err := p.Submit(runCtx, resizeJob); err != nil {
	// queue full and runCtx already cancelled — reject the request
}

// on SIGTERM:
cancel() // stop workers from pulling *new* work off an already-queued backlog... (see Gaps)
shutdownCtx, _ := context.WithTimeout(context.Background(), 30*time.Second)
if err := p.Shutdown(shutdownCtx); err != nil {
	log.Warn(err) // forced exit after deadline; some jobs may not have run
}

Gaps to revisit

  • cancel() before Shutdown is too blunt in the example above: cancelling runCtx makes every worker's select immediately prefer <-ctx.Done(), which can abandon jobs already pulled off the queue mid-flight if the job function itself checks ctx.Done() — for image resizing that's probably fine (abort and retry), but for a job that must complete once started (e.g., writing a partial file), you need a separate "stop accepting" signal (close the channel via Shutdown alone, without cancelling runCtx first) so workers finish their current job before observing cancellation.
  • Result delivery is unaddressed: this pool assumes fire-and-forget jobs (error return only). A version that needs per-job results back to the submitter needs each Job to carry its own result channel or use an errgroup.Group with a bounded semaphore instead of a raw worker pool — worth comparing against this channel-based design.
  • No panic recovery: a panicking job kills its worker goroutine (and, without a recover(), likely the process) — production code needs a defer recover() in worker that converts a panic into an error rather than taking down the pool.

Engineering Lens

The three real design decisions here — bounded queue size (backpressure vs. OOM), select with ctx.Done() in every blocking point (no goroutine can wait forever), and a Shutdown with its own deadline (bounded wait, not indefinite) — are the same three decisions any bounded-concurrency system needs, in any language: a thread pool with a bounded BlockingQueue in Java or a Semaphore-gated pool in Python make the identical tradeoffs with different syntax. The Go-specific value of this exercise is fluency with select as the mechanism that makes "wait for A, but give up if B happens" a first-class, race-free expression rather than a manually-coded polling loop.

Hermes Wiki