The Gateway API Inference Extension addresses Kubernetes routing challenges posed by long-running, resource-intensive large language model (LLM) inference sessions. Traditional load balancers are insufficient for these complex workloads. This extension enhances the existing Gateway API with inference-specific capabilities. It introduces two Custom Resources: InferencePool, managing model server pods, and InferenceModel, defining user-facing model endpoints. The request flow involves Gateway routing, an Endpoint Selection Extension for optimal pod selection, and inference-aware scheduling. This results in improved model-aware routing. Benchmarking shows comparable throughput to standard Kubernetes services but significantly lower latency, particularly at higher query rates. Future development includes features like prefix-cache aware load balancing and support for various model types and accelerators. The extension simplifies and standardizes AI/ML traffic routing within Kubernetes. It aims to improve efficiency and user experience in deploying and managing LLM services.