StackCraft.— Chris Jack
Two slots · Q3 2026
§ Practice · 25 August 2025 · 6 min read

I don't write AI roadmaps. I run a four-week pilot.

A roadmap pretends you know what's worth building before you've built anything. With AI, you don't. Plan a pilot, not a roadmap.

CTOs and ops leads ask me for AI roadmaps regularly. I don't write them, and I'd push back gently on anyone who does. The companies I've watched succeed with AI didn't follow roadmaps; they ran small, scoped pilots, then did the next one.

A roadmap pretends you know what's worth building before you've built anything. With AI specifically, you don't. The model behaves differently than a deterministic system, your data has gaps you'll only find by hitting them, and your team has skill gaps you'll only spot once they're working with the tool. Plan a pilot, not a roadmap.

The pilot I run for new engagements has the same shape every time: pick one workflow, build a working prototype in two weeks, ship it to a real user in week three, measure in week four, decide. The decision in week four is one of three things — invest more, change direction, or stop. All three are valid.

Picking the workflow is the most important call. The right pilot has high volume (so you'll see it work), low judgement (so the model can plausibly help), and a real owner (so someone cares whether it works). Avoid pilots where the user is "everyone" or the metric is "engagement." Both are signs you don't yet have a real pilot.

Two weeks of build is not very long. The trick is to scope hard. One model call. One interface. One user type. Resist the urge to add a second model, a second user, or a second feature. The pilot's job is to prove the model can be useful here; everything else is downstream.

Week three matters more than the build. A pilot that nobody actually uses is a demo. Real users find the failure modes you didn't model and the success modes you didn't expect. Both are gold. Most of what I learn in a build comes from the first week of real usage, not the build itself.

The metric in week four should be honest. Not "did people enjoy it?" — they always say yes when the founder is in the room. Measure time saved, errors avoided, or tasks completed without human intervention. Pick the one that maps to the cost of the workflow today, and set the threshold before the pilot starts.

What to do if the pilot worked: do another. Pick a different workflow, run the same shape, build the second one cheaper because the team now knows the rhythm. Most clients I work with are on their third or fourth pilot before they consider building anything bigger.

What to do if the pilot didn't work: write down why. Was the model wrong? The data thin? The workflow not as routine as you thought? The user not actually motivated? Each of those tells you something different about where AI does and doesn't fit your business — and that's the real roadmap. The one written from evidence.

The deepest mistake I've seen is treating AI implementation like an enterprise software rollout. You don't need a steering committee, a maturity model, or a 47-slide strategy deck. You need a small, scoped, four-week pilot, and the honesty to keep going or stop based on what happened. The rest is theatre.

← All articlesStart a scoping call
§ Start a project · Two slots · Q3 2026

Have a feature that needs shipping?

Start a project How I work