Measuring the return on AI is unlike measuring anything you have tracked on a spreadsheet across the last few decades of your career. We are in the first generation of this technology, and right now the returns live in the learning curve and in the epiphanies inside the minds of your employees. The numbers come later, as consequences. And is it worth the hype? The honest answer: only now is AI becoming good enough to find out.
You were measuring a high school student
For the last 18 to 24 months, the AI models were interesting, but roughly equal to a high school student. You do not need, and should not want, to measure your global corporate revenues against the intelligence of a high school student. Yet that is what most ROI demands were indirectly asking. Today the models are finally beginning to deliver on more of the promises, not all of them, and that changes what measurement means. If you judged AI on those early returns and walked away, you measured the wrong period and are about to miss the right one.
What to measure right now
Say this to your employees, close to word for word: “Eventually I am looking for a number. At this time, I am looking for the ideas. I am looking for challenges to how I lead this company, how our leadership runs your project, what we deliver and how we could deliver it better, how we talk to clients, how we market, which channels and methods we should be using instead. You are the frontline of this business. With AI, come back to me and show me where we can do better.”
The output you want is a spreadsheet of incredible, uncomfortable ideas. Many will upset current leadership, because they will show the company has been doing things the wrong way. That is the cost of innovation, and the leader’s willingness to hear it is usually the real constraint.
AI can now reveal the cracks in the paint: expenditure that has been leaking through the company for years, sometimes decades, that nobody could find. Customers who have been unhappy for reasons that were standing right in front of your face. Your frontline people already know where these cracks are. AI gives them the instrument to prove it.
The two phases of AI return
Phase one, now: count what is surfacing. Ideas submitted. Challenges raised to leadership. Leaks found. Unhappy customers identified and understood. Broken processes named. People whose way of working and thinking has visibly changed. This is the period when you invest in finding out who you are.
Phase two, next: deploy what phase one taught you, as real projects with owners and governance. This is when you invest in who you are going to be, and this is when the spreadsheet numbers arrive, not as targets you demanded from a high school student, but as consequences of what your people learned.
This measurement period is your new culture
Get this right and you are not just measuring an investment. You are creating the next culture of your company, because you will be deploying projects built on what your own people discovered, not on criticising or scaring them, but on embracing them as the next version of your leadership team. Companies that reinvent themselves this way shed their old skin from the inside. People who are treated as the next version of the company tend to stay, and it shows in everything from retention to, yes, the atmosphere at the Christmas party.
Frequently asked questions
How do you measure ROI on AI?
In two phases. First, measure learning: ideas surfaced, challenges raised to leadership, cost leaks and unhappy customers identified, and how many people visibly change the way they work. Second, deploy what was learned as governed projects, and measure those in conventional financial terms. Demanding spreadsheet returns in phase one measures the wrong thing.
Is AI worth the hype for businesses?
It is only now becoming possible to answer that. For roughly the last two years the models performed at the level of a high school student, which made revenue-level ROI demands premature. Current models are beginning to deliver on more of the promises, which is exactly why the measurement approach has to change now.
Why do traditional KPIs fail for AI projects?
Because a spreadsheet reveals the numbers, not the value. In the first generation of AI the return appears first as changed thinking, surfaced ideas and exposed problems, none of which fit existing KPI structures. The financial numbers arrive later, as consequences of acting on what was learned.
What should a CEO ask employees for instead of ROI numbers?
Ideas and challenges. Ask employees to use AI to break down the business: what should be delivered differently, which customers are unhappy and why, where money is leaking, which channels and methods are outdated. The frontline sees what leadership cannot, and AI gives them the instrument to prove it.
Thomas Anglero is a Strategic AI Advisor, keynote speaker and author of Intro to Artificial Intelligence. He has delivered over 450 keynotes across 30 countries for organisations including IBM, the WHO, the World Government Summit and the European Commission. He founded the IBM Watson AI Lab for Cancer at the Oslo Cancer Cluster and closed over $500 million in enterprise transformation deals as CTO and Chief Innovation Officer at Cognizant.
If you are leading your organisation through this, I work with a limited number of senior leaders each quarter. Get in touch at Anglero.com.