Concurrency and Scalability Checklist for Async Python Series

Applicable to FastAPI/Uvicorn apps with chatbot + individual feature endpoints using background tasks, external APIs, and shared state

1. Server & Process Model

• Identify the ASGI Server and Worker Count - Is it Uvicorn, Gunicorn + Uvicorn, Hypercorn? How many workers per process?
• Check if workers are configurable - Is workers= hardcoded, env-driven, or defaulting to 1?
• Identify the event loop model - Single async loop? Multiple processes? Forked workers?
• Check container CPU limits - os.cpu_count() inside a container may return host CPU count, causing miscalculated thread pool defaults.
• Check if default thread pool size is explicitly set - asyncio.to_thread() uses min(32, cpu_count + 4) by default. Is this adequate?

2. Endpoint Concurrency Patterns

• Classify each endpoint - Is the handler async def (runs on event loop) or def (runs in threadpool)?
• Identify background task mechanism - FastAPI BackgroundTasks, Celery, ARQ, or manual asyncio.create_task()?
• Verify background tasks don't block the event loop - Are CPU-bound or sync blocking calls wrapped in asyncio.to_thread() or run_in_executor()?
• Check for fire-and-forget tasks - Is there any mechanism to track/cancel background tasks, or are they unmanaged?
• Identify per-endpoint timeout enforcement - Is asyncio.wait_for() used? Are timeout values configurable?

3. In-Process State (the multi-worker killer)

• Audit all module-level mutable state - Dicts, locks, events, caches stored at module scope
• Identify in-process locks - asyncio.Lock, threading.Lock at module level? These break across workers/pods
• Identify in-process events/signals - Cancel events, completion flags? These can't cross process boundaries.
• Identify in-process singletons - Agent instances, config objects, connection managers. Are they safe to duplicate across workers?
• Document which state must be externalized before adding workers (Redis locks, Redis Pub/Sub for cancel signals, etc.)

4. Shared State & Data Stores

• Identify all data stores - Redis, MongoDB, S3, Elasticsearch, etc.
• Check shared state thread safety - Does the shared use threading.Lock? Is it adequate for async contexts?
• Check Singleton patterns - which Classes are Singletons? Are their connections thread-safe and reusable across concurrent requests?
• Check connection pooling - Are DB/Redis/HTTP client using connection pools? Are pool sizes configured?
• Check for read-modify-write races - Any pattern where state is read, mutated in Python, then written back without atomicity?
• Check session/sticky affinity requirements - Does the load balancer need to route same-user requests to the same pod?

5. Thread Pool & Parallelism

• Count total thread consumers - How many asyncio.to_thread() calls can run concurrently? How many internal ThreadPoolExecutor instances exist?

• Identity nested thread pools - Does a background task (already in thread), spawn its own ThreadPoolExecutor ? These are separate pools.

• Calculate worst-case thread usage: N concurrent users x trades-per-request. Does it exceed the default pool?

• Check for GIL bound CPU work - Python threads don't parallelize CPU work. Is any CPU intensive logic (Diffing, jipping, parsing,) a bottleneck?

• Identify, blocking, I/O not offloaded - synchronous HTTP calls, file I/O or DB queries, running on the event loop without to_thread()?

6. External API & Resource Contention:

• List all external API dependencies - LLM, network devices, monitoring platforms (ExtraHop, Corvil), MCP Servers, S3, etc.
• Check for shared API clients (Singletons) - Are all requests sharing