Most businesses that try AI start with a pilot, and most pilots quietly fail. The widely cited figure is that around 95% of AI pilots never deliver measurable business value. The uncomfortable part is that the cause is rarely the technology. It's how the pilot was set up.
A pilot that works isn't a demo that impresses the board. It's a small, real deployment that proves an outcome and gives you a clear path to production. This guide covers how to run one.
Why most AI pilots fail
The failure patterns are consistent and they show up before a single line of the pilot is built.
It starts with a tool, not a problem. Someone picks a platform, then goes looking for something to use it on. The pilot becomes a search for a use case rather than a solution to a known one.
Nobody owns it. Without a single accountable owner, a pilot drifts. Decisions stall, scope creeps and the work becomes nobody's priority.
The team isn't involved. If the people who do the work aren't part of the build, adoption is low. The pilot works in a demo and dies on the floor.
Success is measured with vanity metrics. Counting prompts run or quoting model accuracy in isolation measures activity, not value. None of it tells you whether the business is better off.
There's no path to production. The pilot proves something in a sandbox and then has nowhere to go, because the foundations, data and governance needed to scale it were never in scope.
Start with a problem, not a tool
Pick one high-value process where your team loses real time, and one metric that moves if the pilot works. Reducing cycle time, cutting manual data entry, speeding up information retrieval or improving a first-time-right rate are all concrete and measurable.
The test is simple: if you can't name the metric the pilot is supposed to move, you're not ready to start. The technology choice comes after the problem is clear, not before.
How long should an AI pilot run?
A pilot should be time-boxed, usually four to eight weeks. Long enough to run with real users and real data, short enough to force focus. Anything open-ended tends to expand until it loses the point.
Time-boxing also protects you. If the pilot hasn't proven its metric by the end of the window, that's useful information, not a reason to keep spending.
What to measure
Track hard outcomes the business already cares about:
- Hours saved per person or per week
- Cost reduced or avoided
- Revenue attributed or accelerated
- Error rates and rework reduced
Avoid the vanity metrics: number of prompts, queries run or model accuracy quoted without business context. They look like progress and tell you nothing about whether the pilot earned its place.
Who should own the pilot
Put one accountable owner in the chair, and make it an operator who understands the process, not just the technology. Bring in the people who do the work from day one, because their expertise shapes the solution and their involvement is what drives adoption later.
Decide up front who approves changes and who signs off the result. A pilot with clear ownership moves. A pilot owned by a committee stalls.
The sequence that gets to production
The order of operations is what separates pilots that scale from pilots that stall.
- Get the foundations right first. Connect the data and knowledge the pilot needs. Most pilots fail here, not at the model.
- Run one process with one metric. Resist the urge to prove five things at once.
- Prove the outcome with real users. Not a demo. The actual team, doing the actual work.
- Then expand. Once the outcome holds, widen the scope deliberately.
This is the logic behind our four-pillar framework: People, then Process, then Knowledge, then Technology. Understand who does the work and how it really runs, structure the knowledge it depends on, and only then bring in the technology. It's the same order whether the pilot is funded privately or through a scheme like New Zealand's MBIE AI Advisory Pilot or an Australian AI Adopt Centre.
From pilot to production
A pilot has only succeeded if it can become part of how the business runs. That means adoption by the team, a handover that doesn't leave the capability stranded and a plan for keeping it accurate as your operations and the underlying models change.
This is where many pilots that "worked" still fail to deliver: the proof of concept is declared a success and then nobody owns the move to production. Build the path to production into the pilot from the start, not after it.
Getting started
The fastest way to set up a pilot that works is to be honest about where you stand before you begin. Take our AI Readiness Assessment to see where your business sits across people, process, knowledge and technology. It takes about three minutes and points to where a pilot would create the most value.
If you have a specific process in mind, get in touch. We'll help you judge whether it's a good first pilot and what it would take to prove the outcome.
