From Silos to Service Topology... Note

From Silos to Service Topology: Why Netflix Built a Real-Time Service Map

Netflix developed a "living map" of its distributed infrastructure, called Service Topology, to help engineers understand service dependencies and troubleshoot issues. The map addresses the need for quicker identification of service relationships and potential impact during outages. It answers crucial questions like which services depend on each other and identifies the source of problems. They gathered data from three sources: eBPF network flows, IPC metrics, and end-to-end tracing. Each data source provides a unique perspective – network connectivity, application-level details, and actual request flows. This multi-layered architecture combines these views into a unified, real-time map. The system uses separate graph databases for each data source to maintain independence and enable parallel querying. Engineers can view each graph individually or combine them for a comprehensive understanding of service interactions. The system ingests flow logs from Kafka in multiple regions, processes them, and stores the data in graph databases. This living map helps Netflix engineers quickly diagnose and resolve problems, ensuring a smooth streaming experience.
CdXz5zHNQW_fZOTLLTM4l.png