Quick answer: AI API pricing is usually based on how much information you send to a model, how much it generates in return, and sometimes whether part of the input can be reused from a cache. The cheapest-looking price per million tokens is not necessarily the cheapest real workflow. A useful budget must also consider repeated requests, long documents, tool calls, retries, storage and human review.
This independent guide explains the cost components without assuming one vendor’s current price will stay fixed. All calculation examples below are illustrative, not quotations from an API provider. Check the provider’s live pricing before purchasing.
1. What exactly is a token?
First, a token is a unit of text processed by an AI model. It is not the same as a character, word or sentence: punctuation, uncommon spellings, code and different languages can split into tokens differently. When you ask an API to summarize a customer email, both the instruction and the email consume input tokens. The generated summary consumes output tokens.
In addition, some providers report cached input separately. Multimodal inputs, such as pictures, video and audio, can follow different accounting rules. For planning, use the tokenizer and usage fields supplied by your chosen API instead of making universal “one token equals X words” assumptions.
AI API Pricing: Input, Cached Input and Output Costs
| Usage component | What it means | Budget implication |
|---|---|---|
| Input tokens | Your instructions, context, history, retrieved pages or documents | Longer prompts generally cost more. |
| Cached input tokens | Eligible repeated prompt content reused by supported caching features | May receive a discounted rate, depending on model and cache behavior. |
| Output tokens | What the model generates, including eligible intermediate generation accounted for by the provider | Can cost more per token than input. |
However, different models and processing tiers have different price schedules. OpenAI’s official API pricing, for example, distinguishes input, cached input, cache writes where applicable, and output. Its prompt-caching documentation explains why reusing identical prefixes can change effective cost. This is a pricing structure example, not a recommendation to use a particular model.
3. The simple formula for estimating a request
For a token-priced text call, start with:
Estimated cost = (uncached input tokens ÷ 1,000,000 × input rate) + (cached input tokens ÷ 1,000,000 × cached rate) + (output tokens ÷ 1,000,000 × output rate).
Then add any separately charged services. Suppose, strictly as an example, your application uses 300,000 uncached input tokens and 80,000 output tokens across several calls. If your hypothetical rate card were $1 per million input tokens and $4 per million output tokens, the token charge would be $0.62 (0.30 + 0.32). This calculation excludes caching, tool use, taxes, storage and retries. It is not a quoted current model price.
For a monthly forecast, multiply by expected requests, account for the distribution between short and long prompts, and add a safety margin for unsuccessful calls and extra generations.
4. Why the context window can change the bill
Meanwhile, the context window is the amount of information a model can consider during a request, subject to that model’s limits. Bigger context capacity is useful for large documents, long conversations and codebases. But capacity is not free usage: sending an entire 100-page report repeatedly can cost more than sending only the relevant paragraphs.
For example, some applications resend previous messages with every conversational turn, while others retrieve relevant excerpts from a knowledge base. This difference affects latency and costs. Our Claude message threads API guide explains one documented way to reduce repeated request payloads, including expiry and replay considerations.
5. Prompt caching is useful, but don’t overestimate it
In turn, prompt caching can reduce repeated-input work when the service recognizes an eligible reused prefix. A common optimization is to place stable instructions at the beginning of a prompt and changing customer data afterward. That creates more opportunity for prefix reuse, although exact eligibility, minimum length, retention, and the way caches are charged vary by provider.
However, three limitations matter: cached content may expire; the first uncached call still incurs its applicable charge; and generated output usually has its own rate. If a provider charges for cache writes or offers a separate long-context tier, model those costs too.
Therefore, measure actual cached tokens in API usage records. A “possible cache hit” is not the same as savings you can book in your budget.
6. Hidden costs that basic pricing tables miss
- Retries: A workflow may make extra calls after invalid JSON, timeouts or user edits.
- Long outputs: Asking for ten alternatives instead of two increases generation volume.
- Tool calls: Search, file processing, image generation, transcription or hosted execution may have separate prices.
- Embeddings and retrieval: Knowledge-base indexing, vector storage and retrieval infrastructure can have their own bills.
- Quality control: Staff time spent checking facts, compliance and customer-facing copy is a real expense.
- Data movement and monitoring: Logs, cloud hosting, observability and integrations should appear in the operating budget.
- Currency and tax: Your actual invoice can differ from the headline USD rate.
Likewise, do not assume that a plan branded “unlimited” gives you unlimited API requests. Chat subscriptions, creator-software plans and API usage are often separate products. If you are reviewing video-generation subscriptions, see our complementary guide to AI video credits and their real costs.
AI API Pricing Worksheet for a Small Business
Before choosing a provider, record the number of users, requests per user, average input tokens, average output tokens, expected cached share and optional tool charges. A simple sheet should contain six columns: workflow, request count, input tokens, output tokens, extra services, and monthly estimate.
For example, a customer-support drafting system might handle 30 questions per workday while an internal document assistant handles only five but processes substantially more text. Both can have the same number of users and radically different costs. Forecast at low, expected and high demand, then test your assumptions with a small real workload.
Next, set alerts for budget thresholds and unusual spikes. Also set a maximum output length where supported, trim unnecessary conversation history, and store results rather than regenerating the same report whenever someone clicks a button.
8. When a smaller model may be the better purchase
For example, many routine tasks—classifying feedback, extracting dates, routing support emails—do not always need a premium reasoning model. Start by identifying measurable success criteria: accuracy, latency, acceptable formatting, human-review hours and cost per accepted result.
Then evaluate a lower-cost model and a higher-capability model on the same sample tasks. A cheaper model that fails often can cost more after retries and staff correction. Conversely, using the largest model for every simple categorization task may waste budget.
9. Privacy and safety belong in the cost decision
Finally, before putting customer records, contracts or proprietary data into an API, review retention, model-training policies, regional processing and access controls. Mask unnecessary personal information. An inexpensive service is not a good deal if it cannot meet your data-handling requirements. Our AI tool privacy checklist provides a separate vendor-assessment framework.
10. Buyer checklist: questions to ask before signing up
A subscription, a metered API and local inference can have very different operating costs. Our Kimi K3.1 Pro API pricing guide illustrates another model-specific pricing workflow to compare against your own token consumption and quota needs. Always check the current official vendor rate before budgeting.
- Which model and processing tier is included at the quoted price?
- Are input, output, cached-input and cache-write rates separate?
- What is the maximum context length, and does crossing a threshold change rates?
- How are images, audio, search and other tools billed?
- Is the same price available in my location and currency?
- Does the provider supply usage reporting, alerts and hard spending caps?
- What happens when requests fail, retry or exceed rate limits?
- How will we verify accuracy and prevent sensitive-data leakage?
Model choice changes these numbers more than most buyers expect. Our Claude Fable 5.2 vs Mythos 5.2 comparison sets out the price and access differences between the two tiers.
FAQ
Is an AI API the same as a ChatGPT subscription?
No. A consumer chat plan and usage through a developer API typically have different billing arrangements. Confirm which product you are purchasing.
Does a larger context window automatically mean better answers?
No. It makes more information available, but irrelevant context may increase cost and can make retrieval and evaluation harder. Better source selection often matters more than maximum length.
Does caching make every repeated prompt free?
No. Eligibility varies, cached requests can still incur charges, and outputs are accounted for separately. Check provider documentation and usage records.
What should I optimize first?
Start with excessive prompt history, avoidable retries and output length. Measure cost per successful task, not just the cheapest headline rate.
Conclusion
Ultimately, the right AI API is the one that reliably completes your task within your accuracy, privacy and monthly-cost limits. Compare the real usage formula, use small-scale trials and revisit provider pricing before expanding. Token rates change; your measurement process should remain stable.
Source verification: The general billing structure and caching explanation above were checked against OpenAI’s pricing documentation and prompt-caching guide on October 18, 2026. No provider price in the hypothetical example is presented as a current rate.
