Last year, MIT's NANDA initiative published a report called The GenAI Divide: State of AI in Business 2025. Its most quoted finding: about 95% of the company generative AI pilots in its research showed no measurable return.
That number gets used two ways. Skeptics use it to say AI is overhyped. Vendors use it to say you need their product to be in the 5%. Neither is very useful if you run a business and want to know whether to try.
Here's what I think the number actually says, based on the projects I've built and the ones I've watched stall.
Most pilots aren't designed to show a return
A typical pilot goes like this. Someone senior reads about AI. The company buys licences for a chat tool, runs a lunch-and-learn and tells staff to "find ways to use it". Six months later, a few people use it to tidy emails and nobody can say what changed.
That isn't a failed experiment. It's an experiment that never defined success. There was no specific job, no measure of before and after, and no owner responsible for the result.
What the working projects have in common
The projects I've seen succeed look different from the start. They share five things.
1. One specific deliverable
Not "use AI in finance", but "process incoming supplier invoices into the accounting system, with exceptions flagged for approval". A job you can point at, that happens often, and that someone is currently doing by hand.
2. A before number
How many invoices a week? How long does each take? How many errors get caught later? If you don't know the starting point, you can't show a return, even when there is one.
3. The business written down
Generic AI produces generic work. The projects that pay off give the model the company's rules, terms, systems and examples of good output, in writing, read on every task. I wrote about this in ChatGPT is a brand new hire every time you open it.
4. A quality bar and a check
Before any work runs, someone decides what "good enough" means, and something checks every output against it. Sometimes that's a person reviewing. Sometimes it's a second automated check, like comparing extracted totals against the original document. Usually it's both, with the person only seeing what the check flags.
5. Somebody owns the outcome
Not the IT department and not "the AI committee". One person who cares whether the invoices get processed, and who's accountable if they don't.
The other side of the divide
It's worth looking at what success can look like when a company commits to it. In February 2026, Block, the company behind Square and Cash App, reorganized itself around AI, running with a team about 40% smaller than before. In a letter to shareholders, its leadership said a smaller team using AI tools can do more, and do it better.
That's Block's own claim, and a company of that size is a long way from a 15-person office in Calgary. But the principle scales down. The gains don't come from giving everyone a chatbot. They come from redesigning specific pieces of work so AI does the repeatable part and people do the rest.
How to run a pilot that can actually succeed
If you're planning a first AI project, here's the shape I'd recommend:
- Pick one job that happens at least weekly, has clear right answers, and currently takes real hours.
- Measure it for two weeks before changing anything: volume, time, error rate.
- Write the context and the procedure before you touch any AI tool.
- Agree on the quality bar and how each output gets checked.
- Run it in parallel with the current process for a few weeks, so you can compare.
- Decide on evidence. If it meets the bar and saves time, expand. If it doesn't, you've learned something specific and cheap.
Small and boring beats big and vague. A pilot that saves one person five hours a week on invoices, with fewer errors, is worth more than a company-wide rollout nobody can measure.
The real lesson in the 95%
The MIT figure doesn't say AI doesn't work. It says most companies try it in a way that can't show whether it works. That's a fixable problem, and fixing it mostly comes down to discipline you'd apply to any other investment: a clear goal, a baseline, a check and an owner.