Your AI feature answers the demo question perfectly. A customer uploads a different document, asks a perfectly reasonable question, and gets a confident answer that is wrong.
Before rewriting the prompt, find out where the failure happened. The model may have received the wrong information, incomplete information, or a task with no answer in the available material.
Check the chain before blaming the final answer
Imagine a hypothetical assistant that answers questions about supplier documents. Your demo uses a clean, searchable PDF. A customer uploads a scan with a table across two pages. Text extraction drops a column, and the model produces an answer using the incomplete text.
A more forceful instruction to “be accurate” does not restore the missing column. The investigation should follow the input through extraction, search, the model request, and the displayed answer.
Ask which document passages were actually supplied to the model. Check whether they belong to the correct customer, are current, and contain enough information to answer the question. An attractive citation is not enough if the cited passage does not support the answer.
Build a small collection of realistic examples
Start with representative cases you have permission to use, removing sensitive information where possible. Include the questions your best demo avoids: missing information, conflicting documents, unusual formats, unclear wording, and requests outside the feature's scope.
For each case, describe what an acceptable result looks like. Sometimes that is the correct value with supporting evidence. Sometimes it is a request for clarification or an explicit statement that the information is missing.
Keep some examples aside when adjusting the system, so you can check whether it improved beyond the cases used to tune it. Anthropic's evaluation guide explains the value of defined success criteria and evaluations. Your business needs to supply the criteria; a technically fluent answer can still fail the customer's task.
Measure different failures separately
One overall accuracy percentage can hide the failure you most need to fix. Track whether the system found the right source, answered correctly, avoided unsupported claims, respected access permissions, and completed within a useful time.
If you use another AI model to grade results, check its judgments against human-reviewed examples. An automated judge is another system that can make mistakes, not an independent guarantee of correctness.
Also repeat a sample of cases. A feature that succeeds only intermittently needs a different release decision from one that handles the supported cases consistently but rejects unusual inputs.
Design what happens when it cannot answer
Give the user a useful next step: clarify the question, upload a clearer document, inspect the relevant source, or send the task for review. Do not present a model's self-reported confidence as a proven probability that its answer is correct.
The level of human review should reflect the consequences of a mistake. Drafting an internal note and making an irreversible external decision are not the same product behavior.
After a change to the model, prompt, extraction, or search, rerun the same evaluation cases before widening access. This gives you an explanation of what improved and what remains unsupported.
Five prompts to use with your AI coding assistant
Before you begin: Use the same project conversation for all five prompts. Inspect first, approve the plan yourself, and never test against live customer data or secrets.
These prompts assume your assistant can inspect the relevant project. Ask it to say clearly if it cannot. Use the prompts one at a time, reviewing each response before moving to the next step.
1. Map
Map the full feature pipeline: input, file extraction, data retrieval, prompt construction, model call, tool use, and output. Show which parts are implemented and where errors can arise.
2. Audit
Classify observed failures into input, extraction, retrieval, reasoning/output, and application issues using authorized sanitized examples. Separate proven causes from likely explanations.
3. Plan
Design a representative evaluation set and define pass criteria, failure categories, safe fallbacks, and human escalation. Propose targeted improvements rather than a general prompt rewrite.
4. Challenge
Compare the existing and proposed behavior on messy, incomplete, ambiguous, and adversarial-but-safe test inputs. Identify regressions and limits of automated scoring.
5. Implement
Once approved, implement the selected changes and evaluation checks in an isolated environment. Show quality results and remaining limitations; do not launch or claim perfect accuracy.
DitchNow's AI feature delivery includes the engineering around the model, from information flow to quality checks. Bring one successful example and one failure to a founder call; that contrast is a useful starting point for a focused assessment.
About DitchNow
DitchNow is based in Haarlem, Netherlands, and helps founders and teams fix and extend software and put AI to work. Meet DitchNow.