Architecture/CaseStudies/airbnb-identity-graph-knowledge-graph-platform

Airbnb: Scaling the Identity Graph off a Vendor Graph Database

Problem + constraints

Airbnb's identity graph — the data structure mapping relationships between users to detect suspicious activity and linked accounts for Trust and Safety — had grown to roughly 7 billion nodes and 11 billion edges, adding about 5 million new edges per day. It ran on a third-party graph database vendor, and at that scale and growth rate, the vendor solution was hitting scaling limits on write throughput specifically — a graph growing 5M edges/day needs sustained write capacity, not just acceptable read latency, and the vendor's write QPS ceiling was becoming the binding constraint on the whole system.

The constraint wasn't just "the current vendor is too slow" — it was that identity graph was very likely not going to be the last large graph workload Airbnb needed to run. Building a one-off replacement scoped only to identity graph's current needs would have solved this problem while guaranteeing the next graph workload hit the same vendor ceiling and needed its own bespoke fix.

Solution

Airbnb built an internal, multi-tenant knowledge-graph platform rather than a point replacement for identity graph alone:

Storage layer: an internal fork of JanusGraph (an open-source distributed graph database) running on DynamoDB for persistence and OpenSearch for indexing. JanusGraph's pluggable storage backend let Airbnb keep the graph-logic layer while swapping in a storage backend they could scale and operate themselves, rather than staying dependent on a vendor's internal architecture.
Three targeted optimizations to the JanusGraph fork: replacing the default locking mechanism with a DynamoDB-native approach (locking is usually the actual write-throughput bottleneck in a distributed graph database, not the storage engine itself), adding parallel execution to the high-fanout data-fetching interface (graph traversals fan out across many nodes; serial fetching serializes what should be parallelizable I/O), and instrumenting the fork with distributed tracing so a graph query's latency could actually be attributed to a stage in the traversal rather than treated as one opaque number.
A management layer on top, providing schema enforcement, index management, and a schematized API surface — the part that turns "a graph database we run" into "a platform other teams can safely build on," with each tenant (identity graph being the first) isolated in its own namespace rather than sharing schema and blast radius with every other tenant.

The result: 32-93% lower read latency across all hop patterns (1-hop through 8-hop) compared to the vendor solution, and roughly 10x the write QPS under load testing.

What to steal

When a vendor's scaling ceiling is the binding constraint, ask whether this is a one-off migration or the first of many. Airbnb built a platform, not a replacement, because identity graph's problem (vendor write-throughput ceiling) was almost certainly going to recur for the next large graph workload. The extra investment in a management layer and multi-tenant isolation only pays off if there really is a second tenant coming — worth being honest about before over-building.
In a distributed graph database, look at locking and fan-out before assuming the storage engine itself is the bottleneck. Airbnb's three fixes — locking mechanism, parallel fan-out fetching, and tracing — are all about the access pattern layered on top of storage, not the storage engine's raw throughput. This generalizes: profile where time is actually spent before concluding you need a fundamentally different storage layer.
A management layer (schema enforcement, tenant isolation, a schematized API) is what makes a shared platform safe to depend on, not the underlying database choice. Multiple teams building directly against a shared JanusGraph cluster with no schema enforcement or isolation would recreate the "one team's traffic pattern breaks another team's workload" problem that multi-tenancy is supposed to prevent.
Forking an open-source project internally is a real, viable option when the gap is a handful of targeted changes (locking, fan-out, tracing) rather than a fundamentally different architecture — it's a smaller lift than building a graph database from scratch, and it keeps you on the upstream project's ecosystem and correctness work for everything you didn't need to change.

Principal Engineer Lens

The real decision point in this case study is the platform-vs-point-fix fork in the road, and it's a decision that shows up constantly at Principal scope: when a scaling problem is discovered in one system, is the fix scoped to that system, or is it evidence of a shared, recurring need across the org? Airbnb's answer — build a multi-tenant platform with identity graph as the first tenant — is the more expensive path in the short term (a management layer, schema enforcement, and tenant isolation are pure overhead if there's only ever one tenant) and the far cheaper path in aggregate if a second, third, and fourth large graph workload materialize later, since each of those inherits the platform instead of hitting the same vendor ceiling and building its own bespoke fix. The architecture-review question this generalizes to: "is this problem specific to this team, or is it the first instance of a pattern the whole org will hit again?" Getting that judgment right — and being willing to argue for the more expensive platform investment when the answer is "this will recur" — is exactly the kind of call that separates a team-scoped fix from Principal-level architectural leverage, and it's a defensible, quantifiable argument in a review when you can point to concrete numbers like Airbnb's 10x write-QPS and 32-93% latency improvement as the return on the platform investment, not just the point fix.

Reel Script

Setup: Airbnb's identity graph — 7 billion nodes, 11 billion edges, growing 5 million edges a day, and core to Trust and Safety's ability to detect linked and suspicious accounts — was running on a third-party graph database vendor that was hitting write-throughput scaling limits under that growth rate.

Concept walkthrough: Walk through the fork-in-the-road decision: fix identity graph's specific problem, or build a platform assuming this recurs. Airbnb chose the platform — an internal JanusGraph fork on DynamoDB and OpenSearch, with three targeted engine changes (DynamoDB-native locking, parallel fan-out fetching, distributed tracing) plus a management layer providing schema enforcement and per-tenant namespace isolation on top.

Real example / case study tie-in: Trace identity graph as the first tenant onto the new platform, and connect the specific engine changes to the specific bottleneck each one fixes — locking to write contention, parallel fan-out to traversal latency, tracing to being able to see where time actually goes in a multi-hop query. Land on the numbers: 32-93% lower read latency across 1-hop through 8-hop patterns, and roughly 10x the write QPS versus the vendor solution.

Tradeoffs & alternatives: Contrast a scoped point-fix for identity graph alone (cheaper immediately, but repeats the same vendor-ceiling problem for the next large graph workload) against the multi-tenant platform Airbnb actually built (more upfront investment in a management layer that's pure overhead if there's only ever one tenant, but amortizes across every future graph workload if more tenants show up).