How the controller-runtime Cac... Note

How the controller-runtime Cache Actually Works, and Why Your Controller Does Not Crash the API Server

Kubernetes controllers, commonly written in Go with kubebuilder and controller-runtime, offer efficient development for distributed workloads. However, as load increases, understanding controller-runtime's internal workings becomes crucial to avoid production issues. The core concept is that r.Get() and r.List() within a reconciler do not directly query the API server; instead, they access a local, in-memory cache. This cache is populated initially by a list operation and subsequently maintained through watches.This design makes local reads exceptionally fast, avoiding strain on the control plane. The trade-off, however, is that the local cache can consume significant memory and may return stale data. Writes, conversely, are always directed to the API server, bypassing the cache. The size of this local cache and the indexing strategies directly impact memory usage. An inefficient List() operation can inadvertently lead to slow linear scans of numerous objects.The reconciliation loop continuously compares desired states with actual states to bring them into alignment. Events are queued, leading to a reconcile function that reads the current state, determines necessary actions, and potentially generates new events. The cache's primary purpose is to provide a rapid, up-to-date view of object states for these reconciliation loops. This watch-based model, originating from client-go, prevents constant API polling by maintaining a single, long-lived connection for updates.controller-runtime abstracts the complexities of lower-level client-go components like Reflector, DeltaFIFO, and Indexer. The Manager orchestrates the shared cache, controllers, and other services. An Informer monitors a specific GVK, maintaining a local store (Indexer) and dispatching events to subscribers. ResourceEventHandlers process these events, updating the store, which is then reflected in the controller's view. A workqueue manages object keys for processing by workers, and Predicates filter events before enqueueing.The Reflector is the sole component directly interacting with the API server, performing an initial list and then establishing a watch. It leverages resourceVersion to resume watches and ensure no events are missed. If a watch connection is lost or the API server indicates an outdated resourceVersion, a relist is performed. DeltaFIFO acts as an ordered buffer for changes, grouping deltas by object key. It provides all accumulated changes for an object since the last Pop() call, ensuring ordered, batched processing. However, it does not collapse consecutive "Added" or "Updated" events, which could lead to multiple processing rounds for rapidly changing objects.