VentureBeat
Follow
Infrastructure and compute: Enterprises are buying AI compute for speed while flying blind on what it costs
AI infrastructure has become operational in two-thirds of enterprises, with three in ten running workloads at scale. However, the ability to track the costs associated with this infrastructure has not kept pace. Performance and GPU availability now outweigh total cost of ownership in purchasing decisions, and reliability is prioritized over price for success metrics. This shift is understandable for teams facing production pressures, but it highlights a significant issue: fewer than half of companies can rigorously track their AI compute costs. Many GPUs operate at half capacity or less, and upcoming investments are directed towards specialized clouds that are currently used by a very small percentage of enterprises. The survey reveals that most enterprises use three different infrastructure platforms, with major cloud providers and AI model APIs being the most common. While integration with existing systems remains a top selection factor, performance and GPU access have risen in importance. Success is now primarily measured by uptime and reliability, followed by developer productivity, rather than cost metrics. Despite the focus on performance and availability, the economics of AI compute are not well-controlled. A large majority of companies with their own GPUs report low utilization rates, and less than half rigorously track AI compute costs and returns. Consequently, value for money is the lowest satisfaction score. Looking ahead, enterprises plan to evaluate AI-specialized clouds, despite their current low usage. Non-Nvidia accelerators are also a significant area of planned evaluation. A substantial portion of enterprises intend to switch or add providers within the next year, but their consideration set is largely dominated by existing incumbents.