Companies around the world are racing to integrate generative AI into their workflows, but a new report finds that about 95% of AI pilot projects have little or no measurable impact on profits — despite the tens of billions of dollars firms have invested.
An AI pilot project is a small test in one part of a business to see if a tool works before it’s rolled out company-wide. The report found that only 5% of the AI tools used in such pilot projects went on to be integrated at scale.
A research group from the Massachusetts Institute of Technology (MIT), analyzed 300 AI projects, interviewed 150 business leaders and surveyed 350 employees.
Their report found that AI tools may boost the productivity of individual employees, but because most AI systems “do not retain feedback, adapt to context, or improve over time,” they cannot learn from and adapt to broader company workflows.
The report also found that when companies tried to build their own in-house AI tools, they generally underperformed compared with tools bought from external providers.
There have been some success stories, said Aditya Challapally, the lead author of the report. He told Fortune, that some startups have seen “revenues jump from zero to $20 million in a year,” thanks to their smart use of AI.
He said these companies “pick one pain point, execute well, and partner smartly with companies who use their tools.”
But generally, AI hasn’t lived up to the hype, say firms. One COO told the researchers: “The hype on LinkedIn says everything has changed, but in our operations, nothing fundamental has shifted.”
In fact, some observers are worried that with such disappointing returns on investments, AI could be in a bubble.
Writing on the US news site Axios, Madison Mills said: capital spending on big tech “has not been this high since the year 2000.”
Mills added: “We all know what happened after that,” referring to the so-called “dot-com crash,” when the tech-stock bubble burst, internet shares fell, and a large number of startups folded.
