Airbnb: Scaling the Identity Graph off a Vendor Graph Database
Problem + constraints
Airbnb's identity graph — the data structure mapping relationships between users to detect suspicious activity and linked accounts for Trust and Safety — had grown to roughly 7 billion nodes and 11 billion edges, adding about 5 million new edges per day. It ran on a third-party graph database vendor, and at that scale and growth rate, the vendor solution was hitting scaling limits on write throughput specifically — a graph growing 5M edges/day needs sustained write capacity, not just acceptable read latency, and the vendor's write QPS ceiling was becoming the binding constraint on the whole system.
The constraint wasn't just "the current vendor is too slow" — it was that identity graph was very likely not going to be the last large graph workload Airbnb needed to run. Building a one-off replacement scoped only to identity graph's current needs would have solved this problem while guaranteeing the next graph workload hit the same vendor ceiling and needed its own bespoke fix.
Solution
Airbnb built an internal, multi-tenant knowledge-graph platform rather than a point replacement for identity graph alone:
- Storage layer: an internal fork of JanusGraph (an open-source distributed graph database) running on DynamoDB for persistence and OpenSearch for indexing. JanusGraph's pluggable storage backend let Airbnb keep the graph-logic layer while swapping in a storage backend they could scale and operate themselves, rather than staying dependent on a vendor's internal architecture.
- Three targeted optimizations to the JanusGraph fork: replacing the default locking mechanism with a DynamoDB-native approach (locking is usually the actual write-throughput bottleneck in a distributed graph database, not the storage engine itself), adding parallel execution to the high-fanout data-fetching interface (graph traversals fan out across many nodes; serial fetching serializes what should be parallelizable I/O), and instrumenting the fork with distributed tracing so a graph query's latency could actually be attributed to a stage in the traversal rather than treated as one opaque number.
- A management layer on top, providing schema enforcement, index management, and a schematized API surface — the part that turns "a graph database we run" into "a platform other teams can safely build on," with each tenant (identity graph being the first) isolated in its own namespace rather than sharing schema and blast radius with every other tenant.
The result: 32-93% lower read latency across all hop patterns (1-hop through 8-hop) compared to the vendor solution, and roughly 10x the write QPS under load testing.
What to steal
- When a vendor's scaling ceiling is the binding constraint, ask whether this is a one-off migration or the first of many. Airbnb built a platform, not a replacement, because identity graph's problem (vendor write-throughput ceiling) was almost certainly going to recur for the next large graph workload. The extra investment in a management layer and multi-tenant isolation only pays off if there really is a second tenant coming — worth being honest about before over-building.
- In a distributed graph database, look at locking and fan-out before assuming the storage engine itself is the bottleneck. Airbnb's three fixes — locking mechanism, parallel fan-out fetching, and tracing — are all about the access pattern layered on top of storage, not the storage engine's raw throughput. This generalizes: profile where time is actually spent before concluding you need a fundamentally different storage layer.
- A management layer (schema enforcement, tenant isolation, a schematized API) is what makes a shared platform safe to depend on, not the underlying database choice. Multiple teams building directly against a shared JanusGraph cluster with no schema enforcement or isolation would recreate the "one team's traffic pattern breaks another team's workload" problem that multi-tenancy is supposed to prevent.
- Forking an open-source project internally is a real, viable option when the gap is a handful of targeted changes (locking, fan-out, tracing) rather than a fundamentally different architecture — it's a smaller lift than building a graph database from scratch, and it keeps you on the upstream project's ecosystem and correctness work for everything you didn't need to change.
Engineering Lens
The real decision point in this case study is the platform-vs-point-fix fork in the road, and it's a decision that shows up constantly at Principal scope: when a scaling problem is discovered in one system, is the fix scoped to that system, or is it evidence of a shared, recurring need across the org? Airbnb's answer — build a multi-tenant platform with identity graph as the first tenant — is the more expensive path in the short term (a management layer, schema enforcement, and tenant isolation are pure overhead if there's only ever one tenant) and the far cheaper path in aggregate if a second, third, and fourth large graph workload materialize later, since each of those inherits the platform instead of hitting the same vendor ceiling and building its own bespoke fix. The architecture-review question this generalizes to: "is this problem specific to this team, or is it the first instance of a pattern the whole org will hit again?" Getting that judgment right — and being willing to argue for the more expensive platform investment when the answer is "this will recur" — is exactly the kind of call that separates a team-scoped fix from Principal-level architectural leverage, and it's a defensible, quantifiable argument in a review when you can point to concrete numbers like Airbnb's 10x write-QPS and 32-93% latency improvement as the return on the platform investment, not just the point fix.