This guide, with contributors Hossein Sarshar, Ashish Narasimham, and Chenyang Li, aims to help users efficiently serve large language models (LLMs) using vLLM. Serving LLMs requires meeting stringent latency and throughput requirements, making the choice of accelerator and configuration crucial. The guide adopts a bottoms-up approach to selecting the best accelerator and optimizing vLLM settings for cost-effectiveness. Prerequisites include a Google Cloud Project, gcloud CLI, Linux and Docker familiarity, and a Hugging Face account with access to the Gemma 3 27B model. Key considerations for choosing an accelerator involve the model being used, its precision (e.g., bfloat16), workload characteristics like requests per second and sequence lengths, desired GPU utilization, prefix cache rate, and latency requirements. Candidate accelerators are identified from Google Cloud's accelerator-optimized instances and TPU offerings, such as g2-standard-48, a2-ultragpu-1g, a3-highgpu-1g, TPU v5e, and TPU v6e. Memory requirements are estimated using a provided tool, which calculates minimum VRAM needed based on model size, precision, sequence lengths, and batch size. Tensor Parallelism may be necessary for models that exceed a single accelerator's memory, though it can introduce communication overhead impacting latency. The guide details benchmarking and tuning by launching instances, pulling vLLM Docker images, updating an auto-tune script with specific parameters, and analyzing the results to find optimal configurations. The auto-tune script iteratively finds stable vLLM server configurations by adjusting GPU utilization to prevent Out-of-Memory errors.