What’s new in AI infrastructur... Note

What’s new in AI infrastructure and orchestration this month

Google is making comprehensive advancements in AI infrastructure to support the agentic era. They develop leading AI models like Gemini and design software frameworks and hardware. Google also integrates AI into everyday tools and builds the underlying infrastructure, including compute, accelerators, and orchestration software. This robust infrastructure is crucial for companies aiming to innovate faster and deliver better user experiences. Google provides monthly updates on its AI infrastructure news, including product, technology, and tool updates. Recent updates highlight Google Cloud Managed Lustre, C4N network-optimized VMs, and GKE Dataplane V2 scaling. New features include co-operative time-slicing for RL workloads and the open-sourced k8s-aibom security tool. Practitioner guides offer detailed instructions on deploying models like Kimi K3 and running various AI workloads on TPUs. Technical blueprints showcase performance optimizations for large models on Google's TPUs. Google has been recognized as a leader in AI infrastructure by Gartner. Reports indicate a widening gap between AI ambitions and infrastructure reality, with most organizations needing upgrades. Confidential Computing is now available on accelerator-optimized G4 machine series for secure AI data handling. The TPU Developer Hub and TPU AI Telemetry Collector Agent offer new resources for developers. Building high availability into AI inference workloads on GKE and connecting AI agents to data in Cloud Storage are detailed in new guides. Independent benchmarks show GKE Inference Gateway outperforming leading managed Kubernetes services. Customer wins demonstrate the successful application of Google's AI infrastructure in healthcare and customer review platforms.