ThreadPool WorkerPool
A bounded set of reusable threads/workers that pick up tasks from a queue, avoiding the cost of spawning a new thread per task and capping concurrency.
Why we need this / what value this brings
Spawning an unbounded number of threads/processes for concurrent work can exhaust system resources; a pool caps concurrency to a sustainable level while still reusing threads instead of paying creation cost per task.
When to use this
Any workload with many short-lived concurrent tasks, especially CPU-bound or blocking work that can't just be async/await'd.
How to use or implement this
Size the pool to the actual resource constraint (CPU cores for CPU-bound work, a safe concurrency limit for I/O-bound blocking work), and monitor for queueing/starvation under load.
Research questions
- FastAPI's default thread pool for sync route handlers — what's it sized to, and could an unbounded number of blocking calls exhaust it?
Empty folder — drop notes, links, and findings here as you research.