It’s unavoidable. The rise of LLMs and agent-based apps is overwhelming cloud environments that were never built for this kind of compute – and CIOs are scrambling to adapt.
In this discussion, John Pettit, CTO of Promevo, emphasized the company’s goal of simplifying Google technology for clients to enhance their success in the digital landscape. He explored the competitive landscape of technology, particularly the challenges posed by AI, including energy consumption and data center complexities. He highlighted the need for IT managers to adapt to agentic AI workflows, which require new management strategies. He pointed out that traditional cost management practices are inadequate, as they focus on past expenditures rather than the value generated from investments. He stressed the importance of developing better observability and instrumentation to monitor AI systems effectively.
John also delved into the complexities of AI inference costs and the increasing energy demands associated with AI operations. He explained that AI agents autonomously generate search requests, leading to higher electricity use, and noted the limitations companies face regarding GPU resources.
John emphasized the importance of evaluating AI compute costs based on the value generated rather than just the expenses incurred and highlighted the need for organizations to develop comprehensive AI strategies, including quantifying outcomes and conducting small-scale experiments.
