Vendor demos are built on clean data and a happy path. These questions are the ones worth asking in the meeting, because the answers tend to predict what happens in month three.
On accuracy
What is the accuracy on documents like ours, not like yours? Ask to run the pilot on your own worst inputs — the faded scan, the supplier who changed their layout, the form with handwriting in the margin.
What happens when it is unsure? A tool that flags low confidence is safer than one that is always confident. Silent wrong answers are the expensive failure.
On your data
Where does our data go, and is it used for training? Get the answer in the contract, not the sales deck. If you handle personal, financial, or patient data, the question is a compliance requirement rather than a preference.
Can we export everything, in a usable format, if we leave? Ask what happens to the extracted and corrected data, not just the source documents.
On fitting your process
What does the exception queue look like? Somebody will work in it every day. If that screen is an afterthought, the tool was designed for demos.
Does it integrate with what we already run, or does it expect us to move? Most operations cannot abandon their ERP or accounting system, and a tool that assumes otherwise ends up as another place data is typed.
On the commercial shape
How is it priced as volume grows — per document, per user, per seat? Model your realistic volume in eighteen months, not today's.
What happens when the underlying model is updated? Behaviour can shift. Ask how changes are communicated and whether you can stay on a known version while you retest.
The question behind all of them
Is this replacing judgement or replacing typing? Tools that take over typing tend to work. Tools that claim to take over judgement usually hand it back the first time something unusual arrives, and by then the person who used to do it has moved on.
If a vendor cannot give straight answers to these, that is itself an answer.
