Hermes Wiki
LocalzDocs/Marketplace-System-Design-Reference

Marketplace System Design Reference

Goal

Build a marketplace (seller/service provider/consumer roles) with clean boundaries so you can:

  • swap tech (Python ↔ Go, Postgres ↔ Mongo, REST ↔ GraphQL) without rewrites
  • scale from monolith → modular services safely
  • avoid coupling frontend to backend internals

This is accomplished through contracts + boundaries + data ownership + observability.

1) The Principles That Prevent Lock-In

1.1 Contracts over code coupling

Frontend and other services depend on contracts, not your implementation.

  • HTTP contract: OpenAPI
  • Event contract: AsyncAPI (or schema registry + versioned JSON/Avro/Protobuf)
  • Auth contract: OIDC/JWT claims + RBAC/ABAC policy definition

Rule: If a consumer needs knowledge of your internal DB or ORM model, you already have lock-in.

1.2 Data ownership (data contracts)

Each domain has a clear owner (service/module). Owners:

  • define schema
  • write data
  • publish events about changes

Non-owners:

  • read via API, events, or read-models
  • never write into someone else's tables/collections

This is the single biggest unlock for polyglot and safe scaling.

1.3 Replaceable adapters

Treat external dependencies as "adapters" behind interfaces:

  • DB, cache, queue, search, blob storage, payment gateway
  • you can swap implementations with minimal domain changes

Rule: Domain logic must not import Redis/Kafka/Postgres SDK directly.

1.4 Versioning and evolution

Contracts must evolve intentionally:

  • additive changes are preferred
  • breaking changes require new version path (v2)

2) The Minimum "Marketplace Domains" You Should Separate Early

Even if you start as a monolith, keep these as separate modules with boundaries:

  1. Identity & Access — users, roles, permissions, orgs/tenants (optional); token validation, RBAC enforcement
  2. Catalog — products/services listings, categories, pricing, availability
  3. Orders & Checkout — cart, order creation, order state machine
  4. Payments — payment intents, confirmation, refunds, ledgers
  5. Fulfillment — booking scheduling (services), shipping/tracking (products)
  6. Messaging & Notifications — email/push/in-app notifications
  7. Media — uploads (S3), image processing, CDN URLs
  8. Search & Discovery — search index, feed ranking, filtering
  9. Analytics/Audit — event logging, user activity, admin audit trails

Key: not necessarily separate microservices on day 1; separate boundaries first.

3) Contracts You Must Maintain

3.1 HTTP contract (OpenAPI)

Source of truth: backend publishes /openapi.json.

Rules:

  • explicit request/response models (no "any")
  • explicit error format (problem+json style)
  • stable endpoint naming and pagination conventions
  • stable auth headers / token expectations

Practical habit:

  • generate a TypeScript client from OpenAPI
  • React imports the generated client (reduces drift)

3.2 Event contract (AsyncAPI)

Use events for cross-domain integration:

  • orders.created
  • payments.succeeded
  • listing.published
  • notification.requested

Rules:

  • event name = routing key/topic (not flags in payload)
  • payload must be versioned (event_version)
  • include correlation ids (trace_id, request_id, event_id)
  • idempotency key for consumers

3.3 Data contracts (ownership + access)

Write down:

  • who owns data
  • who can write
  • who can read and how

Example:

  • Orders module owns orders and emits orders.created
  • Payments module owns payments and emits payments.succeeded
  • Notifications module never writes orders/payments; it reacts to events

4) Anti-Lock-In Architecture Pattern (Start Monolith, Grow to Services)

4.1 Start as a modular monolith

One deployment, but code structured like services:

  • domain/ (pure logic)
  • ports/ (interfaces)
  • adapters/ (db, kafka, redis, s3)
  • api/ (FastAPI routes)
  • workers/ (async jobs)

This gives you speed now and portability later.

4.2 Split only when you feel pain

You split a module into a service when:

  • scaling needs differ (payments vs feed)
  • deployment risk is high
  • ownership requires separate release cycles
  • performance hotspots demand another language/runtime

When you split, you keep:

  • OpenAPI and AsyncAPI contracts stable
  • events + APIs as the boundary
  • data ownership unchanged

5) Routing vs Consumer Groups (Kafka/Rabbit/Redis Streams)

Routing (which messages a service receives)

  • Kafka: topic name (e.g., orders.created)
  • Rabbit: exchange + routing key
  • Redis Streams: stream name (less rich routing)

Do not put "consumer flags" inside payload as a routing mechanism.

Consumer group (how a service scales)

  • Same service instances share a group id → messages partitioned among replicas
  • Different services use different group ids → each service gets its own copy of the stream

6) "Ports and Adapters" Template

Domain layer (no infrastructure imports)

  • entities, value objects, domain services
  • policy enforcement (roles/permissions decisions)
  • state transitions (order lifecycle)

Ports (interfaces)

Examples:

  • OrderRepository
  • EventPublisher
  • Cache
  • PaymentGateway

Adapters (implement ports)

  • PostgresRepo / MongoRepo
  • KafkaPublisher / RabbitPublisher
  • RedisCache
  • StripeGateway

Result: switching tech is "swap adapter", not "rewrite domain".

7) Data and Consistency Patterns (Marketplace-safe)

7.1 Transaction + Outbox (recommended)

When order is created:

  • write order record
  • write outbox event in same DB transaction
  • background dispatcher publishes to Kafka/Rabbit

This prevents "DB updated but event not published" problems.

7.2 Idempotency everywhere

  • HTTP POST endpoints accept Idempotency-Key
  • consumers store processed event_id (dedupe)

This prevents double charges, duplicate notifications, etc.

7.3 Read models for feeds/search

Feeds and search are usually not served from core tables.

  • build a denormalized view (Elastic/OpenSearch/Postgres materialized views)
  • update via events

8) Auth/RBAC Without Tight Coupling

Design it as a contract:

  • token claims you rely on (sub, roles, org_id)
  • resource-level authorization policy rules

Backend enforces:

  • coarse RBAC (role can do action)
  • fine ABAC (owner_id must match, org_id must match)

Frontend uses roles only for UI/UX; backend is source of truth.

9) Observability and Debuggability as a Contract

Always include:

  • request_id for HTTP
  • trace_id propagated into events
  • structured logs (JSON)
  • metrics on queue lag, error rates, p95 latency

This is not optional at scale; it prevents "distributed guessing."

10) Your Step-by-Step Build Plan (Portable and Safe)

Phase 1: Core skeleton (portable base)

  • monorepo with modular monolith layout
  • OpenAPI stable conventions
  • RBAC middleware + policies
  • DB migrations + repository ports
  • background jobs framework (Celery/RQ/Arq)

Phase 2: Marketplace fundamentals

  • user profiles + roles
  • catalog/listings CRUD
  • order create + state transitions
  • outbox + orders.created event
  • payment consumer skeleton (even if same repo)

Phase 3: Reliability & scale enablers

  • idempotency keys
  • retries + DLQ
  • read models for search/feed
  • rate limiting + caching

Phase 4: Selective split into services

  • payments service (common first)
  • notifications service
  • feed/search service (often needs different scaling)

11) The One Rule to Keep You Unlocked

Never let consumers depend on your internals.

  • consumers depend on OpenAPI / AsyncAPI
  • teams depend on data ownership rules
  • infrastructure is behind adapters

If you follow this, Python + TypeScript will not trap you; they become your fastest starting point.

Hermes Wiki