Weeks 1–2: Pick the Process and Define Success
Resist the temptation to brainstorm fifty use cases. List your ten most repetitive processes, then score each against four criteria: weekly volume (higher is better), standardization (does it follow a recognizable pattern?), error tolerance (mistakes should be recoverable, not catastrophic), and measurability (can you count the hours it consumes today?). Pick the highest scorer — for most service businesses it's invoice processing, lead intake, or report assembly.
Then, before anything gets built, write down the success metric and its current baseline: 'invoice processing takes 22 staff-hours per week today; the pilot succeeds if that drops below 8 by day 90.' A pilot without a pre-committed number produces a demo and an argument. A pilot with one produces a decision.
Weeks 3–4: Map the Real Process (Not the Official One)
Sit with the people who actually do the work and watch a real batch flow through. The documented process and the real one always differ — the undocumented workaround for one supplier's weird invoice format, the mental checklist the ops manager runs before approving anything, the 'everyone knows' exceptions. These details are where automation projects live or die, because the AI must handle exactly the cases your people handle.
Capture: every input format and where it arrives, every decision point and its actual rules, every exception and its frequency, and every system touched. Two outputs from this phase: a process map that engineering builds against, and an exception inventory that defines what the AI should attempt versus route to a human. Skipping these two weeks is the single most common cause of pilot failure.
Weeks 5–8: Build the Pipeline with Human Review Built In
Now build — but build the review workflow first, not last. The launch architecture should be: AI does the work, a human approves every output in a simple review queue before it takes effect. This changes the risk calculus entirely — the worst case is no longer 'the AI corrupted our accounting data' but 'a human clicked reject.' Use existing platforms where they fit (orchestration tools, commercial AI APIs) and custom code only where your process is genuinely unique.
Keep the engineering honest with a golden set: thirty to fifty real historical examples with known correct outputs, run through the pipeline weekly. Accuracy on the golden set — not vibes — tells you when you're ready for live data. By the end of week eight you should be processing real work in the review queue.
Weeks 9–12: Measure, Tune, and Earn Autonomy Gradually
Run the automation on live volume with humans reviewing everything, and track three numbers weekly: accuracy (what percentage of AI outputs get approved unmodified?), coverage (what percentage of cases does the AI attempt, versus routing to a human immediately?), and time saved (hours per week against your baseline). Expect a tuning loop — the first two weeks of live data always surface cases the golden set missed; fix them and watch accuracy climb. As specific categories prove reliable (say, invoices from your top ten suppliers hit 99% approval), remove the review step for those categories only, keeping it for the long tail.
This graduated autonomy is how you get the efficiency without betting the business on day one. By day 90 you have three trustworthy numbers and a system running partly unsupervised.
The Decision Gate: Scale, Fix, or Kill
Day 90 ends with one of three honest outcomes. Scale: the metric cleared the bar — write up the results, quantify the annual savings, and pick phase two (usually the adjacent process: if invoices worked, approvals and reconciliation are next, reusing much of the plumbing). Fix: the approach works but accuracy or coverage fell short — diagnose whether the gap is engineering (fixable, fund six more weeks) or data quality (fix the upstream mess first, then resume).
Kill: the process turned out less standardized or lower-volume than believed — write down why, because that lesson redirects your next pilot to firmer ground. All three are wins compared to the alternative: the vague, metric-free AI initiative that limps along for a year consuming budget because nobody defined what failure looks like.
The Five Mistakes That Sink First Pilots
One: picking a showcase process instead of a painful one — automating something impressive-sounding but low-volume produces applause and no ROI. Two: skipping the baseline measurement — if you don't know it took 22 hours before, you can't prove you saved 14. Three: full autonomy on day one — one early unsupervised error destroys organizational trust that takes months to rebuild.
Four: excluding the people who do the work — they know where the bodies are buried, and their buy-in decides whether the tool gets used or quietly bypassed. Five: letting scope creep in week six — 'while we're at it, could it also…' is how 90-day pilots become 9-month projects. The discipline is the framework.
Companies rarely fail at AI because the technology fell short; they fail because the project around it was soft.
Want a pilot designed around your numbers?
In a free 30-minute assessment we'll score your candidate processes, set the baseline metric, and scope a 90-day pilot with an honest cost and an honest projected ROI.
Scope My 90-Day PilotFrequently Asked Questions
How long should an AI automation pilot take?
Ninety days is the proven window: two weeks selecting the process and baseline metric, two weeks mapping the real workflow, four weeks building with human review built in, and four weeks measuring on live volume — ending with a scale, fix, or kill decision backed by numbers.
What process should I pick for a first AI pilot?
Score candidates on volume, standardization, error tolerance, and measurability. The winner is usually invoice processing, lead intake, or report assembly — high-volume document flows where 60–80% effort reduction is typical and easily measured.
How do I measure if an AI pilot succeeded?
Commit to one metric with a baseline before building — e.g., 'invoice processing takes 22 staff-hours/week today; success is under 8 by day 90.' Track accuracy (outputs approved unmodified), coverage (cases attempted), and hours saved weekly during the live phase.
Should AI automation run without human review?
Not at launch. Start with humans approving every output, then remove review category by category as accuracy proves itself (e.g., 99% approval for a specific supplier's invoices). Graduated autonomy captures the efficiency without day-one risk.
What if the pilot fails?
A disciplined failure is cheap and informative: you learn whether the gap was engineering (fund a fix), data quality (fix upstream first), or process selection (redirect the next pilot). The expensive failure is the metric-free initiative that consumes budget for a year with no decision gate.