Kubernetes v1.37: etcd RangeStream Cuts Memory Use on Large List Reads
etcd RangeStream is graduating to beta in Kubernetes v1.37, working with etcd v3.7. This feature significantly reduces memory usage for the API server and etcd when reading large collections of resources. Previously, populating the API server's watch cache involved expensive full reads from etcd. While pagination existed, it was based on key count, not object size, leading to unpredictable memory spikes and potential Out-Of-Memory errors. etcd's unary Range RPC assembled entire pages before sending, exacerbating memory pressure on both etcd and the API server.The new RangeStream RPC in etcd v3.7 addresses this by streaming results in chunks. The chunk size is adaptively tuned to value sizes, bounding reads by bytes rather than key count. This allows memory to be freed incrementally as the stream progresses. When the EtcdRangeStream feature gate is enabled, the API server utilizes RangeStream for both watch cache initialization and fallback list requests. It decodes and releases chunks as they arrive, preventing the entire collection from being held in memory simultaneously. This feature requires Kubernetes v1.37+ and etcd v3.7+. The API server automatically detects etcd's support and falls back if RangeStream is not implemented. To confirm RangeStream is active, monitor the etcd_request_duration_seconds_count{operation="listStream"} metric for non-zero values.
etcd_request_duration_seconds_count{operation="listStream"}metric for non-zero values.