Netflix TechBlog | Medium
Follow
High-Throughput Graph Abstraction at Netflix: Part I
Netflix built a Graph Abstraction to handle high-throughput, low-latency graph operations, particularly for use cases like Real-Time Distributed Graphs and Social Graphs. It operates within two categories: OLAP for in-depth analysis and OLTP for streaming user experiences. The abstraction uses a Property Graph model with strongly typed nodes and edges, organized into isolated namespaces. Each namespace has a predefined graph schema managed through a Data Gateway Control Plane. This schema enables optimizations like data quality enforcement and efficient query planning. The real-time index uses key-value storage for nodes and edges, employing separate indexes for links and properties. Edge links are indexed by source-destination relationships. To ensure access regardless of direction, this organizes identifiers lexicographically. Caching is used to minimize write and read amplification. The Abstraction architecture prioritizes high performance and incorporates strategies like write-aside caching.