In brief
How do you design a proof of concept that leads to a decision?
Before building, write a one-page charter: the single hypothesis being tested, the metric and threshold for success, the data that represents real conditions, what production would require if it succeeds, and the rule for deciding go, adjust or stop. Then build only what is needed to test that hypothesis.
A familiar story: a team builds an AI proof of concept, shows an impressive demo, stakeholders are pleased, and then nothing happens. Months later someone asks whether the idea works and nobody can say with confidence. The PoC did not fail; it simply was not designed to decide anything.
The one-page PoC charter
Writing a short charter before any code is the most effective fix we know. It has six parts.
1. The decision
State the decision the PoC will inform, for example whether to fund a production document automation project for accounts payable. If no decision depends on the outcome, the PoC is exploration, which is fine, but should be labelled as such.
2. The riskiest assumption
Identify the assumption that, if false, would make the project unviable. Usually it is technical or data-related: that a model can read the documents, that the data contains a predictive signal, that retrieval finds the right policy. Write it as a hypothesis that could be proven false.
3. Success criteria
Define the metric and the threshold before seeing results. For extraction, that might be field-level accuracy on key fields. For forecasting, improvement over the current planning method on held-out periods. Include a threshold for cost per transaction or latency if they could block production.
4. Realistic data
Specify the data sample and how it represents reality. Include the awkward cases: poor scans, rare categories, recent changes, multiple languages. Set aside a test portion that is not touched during development.
5. What production would require
List the conditions that would need to be met to go live: integrations, security review, review workflow, monitoring. The PoC does not build these, but estimating them prevents a technically successful PoC from leading to an unrealistic plan.
6. The decision rule
Agree in advance what results lead to go, adjust or stop. For example: go if accuracy exceeds the threshold on key fields and cost is within limits; adjust if accuracy is close and errors are concentrated in a fixable area; stop otherwise.
What to leave out
- User management, polished interfaces and full integrations, unless the hypothesis is about them.
- Additional use cases that emerge during the work; record them for later.
- Model or vendor comparisons beyond the few needed to test the hypothesis fairly.
Reporting results honestly
A good PoC report is short and structured: the hypothesis, the criteria, the results against them, an analysis of errors, the estimated production effort and the recommendation. Include examples of failures, not only successes. If results are mixed, say what would need to change for a different conclusion.
| Result | Recommendation | Typical next step |
|---|---|---|
| Criteria clearly met | Go | Plan an engineering phase or pilot with real users |
| Close, with identifiable causes | Adjust | Narrow scope, add data or change approach, then a short retest |
| Well below criteria | Stop | Document learning; revisit if conditions change |
A stop recommendation backed by evidence is a good outcome. It saves the far larger cost of building something that would not have worked.
Sources and further reading
About the author: Dr. AarzooCo-Founder and Chief Operating Officer, AARHIT Systems. Writes on delivery, quality assurance, data readiness, monitoring and human review. More articles by Dr. Aarzoo