Your AI feature is getting used. That should be good news. Then the usage bill arrives, and you realise you do not know whether another customer makes the business healthier or simply more expensive to run.
The useful number is not just the price of one model request. It is the total cost of delivering a useful result to a customer, including failed attempts and work that gets repeated.
Follow one task from beginning to end
Consider a hypothetical document assistant. One customer asks a question. The app extracts text, searches for relevant passages, asks a model to answer, checks the answer, and possibly tries again. Each step may have a cost. The button looks like one action, but the system does several.
Ask an engineer to record the feature used, the model or service called, the measured usage, the duration, the number of retries, and whether the task finished acceptably. Use internal identifiers rather than putting customer documents or sensitive prompts into a general-purpose analytics tool.
Separate fixed operating costs from costs driven by usage. Also separate paying customers, free users, internal testing, and unsuccessful requests. Otherwise, a burst of testing can look like a change in customer economics.
Use a calculation that reflects the result
Here is an invented example, not a provider quote. If a feature costs €60 to operate for 1,000 attempts, the cost per attempt is €0.06. If only 750 attempts produce accepted results, the cost per successful result is €0.08. Human review and other operating costs would still need to be added where relevant.
That difference matters. Making the first request cheaper can increase the total bill if it leads to more retries, corrections, or abandoned tasks. Evaluate cost and output quality together; Anthropic's evaluation guidance includes price and task performance among the criteria worth measuring.
Look beyond the average. A few customers uploading very large documents may have a different cost pattern from everyone else. You need to understand the range before promising unlimited use.
Find unnecessary work before changing models
Ask whether the app extracts the same document repeatedly, sends an entire conversation when a smaller context would suffice, or retries a permanently invalid request. An engineer can investigate those specific causes before changing your pricing.
Reusing a result can help when the inputs and permissions still match. It can also return outdated or another customer's information if designed carelessly. Any cache needs clear ownership, freshness rules, and access boundaries.
A smaller model may suit a narrow classification task. A more capable model may remain necessary for other work. Compare candidates on representative tasks instead of selecting solely from a price table. Provider prices and charging rules change, so use current billing records and documentation when doing the calculation.
Put limits around the expensive paths
- Set limits on task size and the number of attempts a task can trigger.
- Make usage allowances understandable to customers before they hit them.
- Add alerts for unusual spending and a defined response owner.
- Use enforceable controls where a hard limit is required; a billing alert alone may not stop processing.
Your decision may be to optimize an implementation, narrow a feature, introduce usage-based packaging, or stop offering an unprofitable workflow. Those are different responses to different evidence.
Five prompts to use with your AI coding assistant
Before you begin: Use the same project conversation for all five prompts. Inspect first, approve the plan yourself, and never test against live customer data or secrets.
These prompts assume your assistant can inspect the relevant project. Ask it to say clearly if it cannot. Use the prompts one at a time, reviewing each response before moving to the next step.
1. Map
Map all code paths that incur model, API, storage, or compute charges. Connect each cost source to a customer action and identify where usage is logged or not logged.
2. Audit
Find repeated AI calls, oversized context, unnecessary reprocessing, retries, and missing usage controls. Estimate impact only from available measured usage and actual rates; flag unknowns.
3. Plan
Recommend prioritized changes such as caching, batching, model routing, input limits, and cost instrumentation. Include quality, latency, privacy, and customer-experience tradeoffs.
4. Challenge
Test the plan against plausible low, typical, and heavy-usage scenarios with explicitly stated assumptions. Identify where an optimization may reduce answer quality or increase failure and rework.
5. Implement
After approval, implement selected instrumentation and optimizations in a test environment. Compare cost per successful task and relevant quality measures. Do not change production quotas or billing without separate approval.
DitchNow can help establish a measurement and iteration loop around your AI product. Book a call to work out what to measure before spending more time on speculative optimization.
About DitchNow
DitchNow is based in Haarlem, Netherlands, and helps founders and teams fix and extend software and put AI to work. Meet DitchNow.