Ray Serve, a scalable model serving library, combined with Google Kubernetes Engine (GKE) offers a powerful platform for LLM serving. Historically, Ray Serve's flexibility came at a performance cost. However, through a partnership with Anyscale, Ray Serve now offers significantly improved performance, achieving up to 5x higher throughput and 8x lower latency. These advancements are due to three key architectural optimizations. Ray Serve now integrates HAProxy for efficient request routing and load balancing, reducing proxy overhead. A direct token streaming architecture bypasses the ingress router for token streams, cutting latency. The v2 Ray executor backend for vLLM enables asynchronous scheduling, unifying the code path and closing the performance gap. Benchmarks on GKE with next-generation AI hardware demonstrated these dramatic performance improvements. The enhanced Ray Serve scales throughput while maintaining low latency, even with increasing concurrent users. GKE provides the necessary infrastructure for these optimizations, offering automated scaling, monitoring, and fault tolerance. Developers can now achieve production-grade performance on Kubernetes without sacrificing Ray's rich features. The latest Ray release (2.56 and later) incorporates these enhancements, and resources are available for further exploration.