A recent survey of 107 enterprises highlights a widening disconnect between aggressive investments in AI infrastructure and the limited ability to monitor associated costs. While organizations continue to allocate significant resources to specialized compute resources, visibility into unit economics remains limited, creating operational inefficiencies that could compound over time.
The data shows that only 21% of respondents currently operate AI workloads at production scale. The majority, or 76%, remain in experimental phases or manage only select production workloads. This early-stage positioning coincides with forward-looking plans that emphasize evaluation of AI-specialized clouds, cited by 45% of participants. Such intentions mark a departure from current reliance on hyperscalers and model-provider APIs, where providers like Google Cloud hold 48% usage.
Utilization rates further underscore the imbalance. Among enterprises running GPUs, 83% report utilization at 50% or below, with 49% operating at 25% or less. These figures point to substantial idle capacity that increases effective costs without corresponding output. Integration capabilities and total cost of ownership drive provider selection for 41% and 35% of respondents respectively, far outpacing headline token pricing at just 8%.
One notable context is the environmental implication of sustained low utilization. Idle accelerators continue to draw power and require cooling, contributing to higher energy consumption across data centers even as workloads fail to scale proportionally. This dynamic may accelerate scrutiny from sustainability frameworks that increasingly factor compute efficiency into corporate reporting requirements.
Another dimension involves competitive positioning. Mid-market firms, which dominate the surveyed sample, risk falling behind larger operators that have already implemented granular cost-tracking systems. Without improved measurement, these organizations may encounter higher effective expenses when shifting workloads to new providers, limiting their capacity to iterate on AI applications at the pace of industry leaders.
Switching intentions reinforce the transitional nature of current setups. A majority of 64% plan to change or add providers within twelve months, including 38% within the next quarter. Yet satisfaction metrics average only 4.0 on a five-point scale, with value-for-money perceptions trailing at 3.9. This pattern suggests that procurement decisions prioritize stack compatibility over immediate economic clarity.
The emerging shift from compute to memory bandwidth constraints in inference receives uneven attention, with 18% of enterprises either unaware or unaddressed. As inference volumes grow, this blind spot could further strain existing visibility gaps, particularly where KV-cache management becomes the dominant cost driver.
Overall, the findings illustrate how rapid capital deployment in AI infrastructure outstrips supporting analytics capabilities. Enterprises that address measurement shortfalls early stand to optimize future expenditures more effectively than those continuing to invest without full economic transparency.
