The challenge of demonstrating AI ROI in Australian organisations

Australian organisations are at a critical juncture in the rapidly evolving field of artificial intelligence.

The potential for significant financial gains from AI is clear, with some reports suggesting that adopting AI in investment portfolios could deliver more than $100 million in incremental EBITDA. But the path to achieving a return on investment is challenging.

Up to 85% of enterprise deployments of AI fail to deliver on their promise to businesses. The high failure rate for AI—even more than the well-known difficulties of past digital transformation efforts—highlights the risks.

When AI deployment fails, the impact can be catastrophic. Australia is a prime example of the risks posed by AI, as evidenced by its “robo-debt” scandal. Becoming so harmful An Australian Royal Commission has been convened to investigate the matter.

Gartner analysts give advice

Although many people are excited about this Possibilities offered by artificial intelligence, The report shows that 80% of Australians Expressing deep concern about the risks posed by artificial intelligence, the United Nations said these risks should be considered a “global priority”.

Despite the risks and societal hesitation, CIOs are investing in AI projects KPMG Research It shows that more than half of Australian companies are investing 10-20% of their budgets in artificial intelligence.

This will only increase the pressure on CIOs and IT teams to ensure AI projects deliver value. Organizations that want to consider AI as a long-term investment opportunity must overcome risk concerns. Gartner Research It shows that evaluating and proving business value is the biggest obstacle to AI projects.

Nate Suda, senior director analyst at Gartner Financial Technology, Value & Risk, told TechRepublic that challenges many organizations face in articulating the value of AI include cost management, productivity benefits and the strategic approach necessary to ensure AI investments translate into tangible business value.

Understand cost dynamics

Administrative costs are a major barrier to deploying AI. Unlike traditional search engines, which have minimal costs, generative AI incurs significant costs due to its interactive nature.

Users often engage in multiple exchanges to refine their responses, which increases costs exponentially. Each interaction (measured in tokens) adds a fee. If user behavior deviates from initial assumptions, costs can skyrocket.

As Suda puts it, “One of the biggest variables in cost is human interaction. With generative AI, you don’t just feed in a question and get a perfect answer. You may need multiple iterations, and you pay for every word in both the question and the answer. If your cost model assumes one interaction and the user ends up having multiple interactions, your costs could be exponentially higher.”

To mitigate this risk, organizations are adopting a strategy of “slowly scaling.” Rather than deploying rapidly and at scale, they start with a planned AI deployment among a limited number of users and then gradually increase the number of users.

This iterative approach enables companies to observe how ambitious AI projects perform and make adjustments based on actual usage patterns, ensuring they can more accurately model costs and avoid financial surprises.

“The best organizations scale slowly,” Suda noted. “They might start with 10 users in the first month, then go to 20 in the second month, and so on. This approach helps them understand actual usage and costs in a live environment.”

Productivity Conundrum

While AI promises to boost productivity, translating those improvements into measurable economic benefits is complicated. Simply saving time, as demonstrated by tools like Microsoft Copilot, doesn’t necessarily equate to generating revenue or reducing costs, Suda said.

“You have to be clear about what productivity means and how to translate that benefit into value, whether it’s increased revenue or reduced costs,” Suda said.

He also stressed the need to distinguish between benefits and value. Benefits such as increased speed, improved customer experience, and increased productivity are important, but they are only valuable if they contribute to the bottom line.

For example, generative AI might reduce the time required to deliver a range of professional services, but unless that efficiency translates into higher revenue or reduced costs, it becomes an example of AI failing to deliver on its promised value.

Risk of cost overruns

Another key point Suda made is the risk of cost overruns due to unexpected user behavior. If an AI system becomes extremely popular and its usage exceeds expectations, the costs incurred can be astronomical. This situation highlights the importance of careful planning and real-time monitoring of AI deployments to effectively manage and predict expenses.

“If users like AI and use it extensively, your costs will go up dramatically,” Suda said. “That’s why it’s so important to understand and model user behavior.”

Strategic deployment: defense, expansion, subversion

Gartner has developed a three-tier framework to explain how AI can deliver value while balancing the associated risks. Each “tier” of AI deployment offers different potential risks and benefits, which are referred to as “defend, extend and disrupt.”

• defend: This involves small, incremental improvements, such as using AI to enhance existing tools. These low-cost, low-risk initiatives can deliver small-scale benefits. The challenge, however, is translating those benefits into significant financial returns. According to Suda, the clear benefits of many of these projects are minimal, making it difficult for CIOs and IT teams to move forward with full organizational buy-in.
• extend: Here, AI is embedded into existing applications to provide targeted improvements. These initiatives require careful planning and execution to ensure they deliver the expected value, but are also more likely to deliver noteworthy benefits.
• subversion: The most ambitious and riskiest approach is to develop new AI-driven models or applications. While the potential rewards are huge, the investment required is huge and the success rate is low.

Artificial intelligence is inevitable, but must be managed effectively

Much like digital transformation, having overly ambitious goals for AI from the outset can lead to cost overruns and slow returns on investment, leading to frustration among boards and executives, or even the abandonment of projects.

CIOs should take a cautious, measured approach. As Suda said, companies should ensure that the solutions they deploy are scalable and deliver a return on investment that can be clearly articulated from the outset.