A pilot is the right tool when you already have reasonable confidence that something works. The question is operational: can we deliver this at scale, in this context, with these resources? A pilot isn't supposed to fail. That's what makes it politically difficult to stop one that isn't working.
An experiment is different. You're testing an assumption or an idea, not a delivery model. The hypothesis is explicit before you start, the success criteria are agreed in advance, and a negative result is just as valuable as a positive one. Nobody promised it would work. That's the point.
The simplest way to keep them straight: an experiment tests a hypothesis, a pilot tests a model.
When they fail, they fail differently. A pilot that goes wrong becomes a U-turn. By the time it's running, budgets, expectations and reputations have moved in around it, and stopping starts to look like failure rather than learning.
That's why pilots need their exits designed in from the start. Sunset criteria and stop conditions have to be agreed before the work begins, not when stopping has become the harder choice.
There's a related trap. If you already know you're going to roll something out regardless of what you find, it isn't a pilot. It's a phased implementation. Calling it a pilot just makes the politics easier and the learning harder.
Before you start anything, the question is simple: do you know this solution works, or do you think it will work? If you know, pilot it. If you don't, experiment first.
At Lighthouse we run experiments and support pilots. We're deliberate about which one we're doing and why. Small, explicit, hypothesis-led experiments to find something out. Pilots when the evidence is there and the question is whether it scales.
We have recently worked with HMPPS staff to co-design new digital systems and tools. After a few iterations and rounds of experiments, the concepts have been rolled out to pilot across the wider prison estate.
Experiment before you pilot. Pilot before you scale. Call an experiment a pilot and you're claiming readiness you haven't tested. Pilot without experimenting and you're just hoping...with a bigger budget.