You paste the same 500-word prompt into two different model APIs, run the exact same request, and the bill is not the same. Not close, not a rounding difference, meaningfully different. Nothing about your text changed. What changed is how each model's tokenizer sliced that text apart before counting it, and that single step is responsible for more billing surprises than any pricing page ever gets credit for.
It's an easy thing to overlook because the pricing pages themselves look straightforward: a rate per million tokens, input and output listed separately. The rate is not the confusing part. The confusing part is that "how many tokens is this text" does not have one universal answer. It depends entirely on which model you asked.
What a Token Actually Is
A token is not a word, and it is not a character. It is a chunk of text, somewhere between the two, that a model's tokenizer decided to treat as a single unit. Most modern large language models use a variant of byte pair encoding, a compression technique that starts with individual characters and repeatedly merges the most frequently co-occurring pairs into single symbols, building up a vocabulary of common chunks along the way. Wikipedia's entry on byte pair encoding covers the underlying algorithm in more depth than any pricing page will.
The practical result: common English words like "the" or "and" usually become one token each. Rarer words, unusual names, code identifiers, and non-English text often split into two, three, or more tokens. A word like "tokenization" itself might become two or three pieces depending on the model, because the vocabulary that gets trained into a tokenizer reflects the text that model's builders trained it on.
Why Tokenizers Differ Between Models
Here's the part that actually causes billing confusion: every major model provider builds and trains its own tokenizer, on its own text corpus, with its own target vocabulary size. There is no shared standard.
GPT-4o and the tiktoken Family
OpenAI's newer models use a tokenizer built for a larger, more efficient vocabulary than earlier GPT generations, which tends to compress English text a bit more tightly per token than older encodings did. OpenAI documents which tokenizer backs which model family, and the encoding has changed more than once across model generations, which is itself a reason a cost estimate from a year ago may no longer be accurate.
Claude's Tokenizer
Anthropic's Claude models use their own tokenizer, trained independently of OpenAI's. It handles ordinary English reasonably close to GPT-4o's tokenizer in most cases, but the two diverge noticeably on code, non-English languages, and unusual formatting like heavily nested JSON or markdown tables. Anthropic publishes guidance on how Claude handles longer context windows, where these small per-token differences compound the most.
Gemini and Multilingual Text
Google's Gemini models were built with a strong emphasis on multilingual performance, and their tokenizer reflects that: text in non-Latin scripts, and some non-English languages generally, often tokenizes more efficiently on Gemini than on tokenizers trained with a heavier English-language bias. Google AI for Developers has model-specific documentation that's worth checking before assuming a token count from one provider transfers to another.
Photo by Alexander Popadin on Pexels
A Concrete Example of the Same Text, Different Counts
Take a plain-English paragraph of a few sentences, no code, no special formatting. Run it through GPT-4o's tokenizer and Claude's tokenizer separately, and you'll typically see counts that are close but not identical, often within a few percent of each other for ordinary prose. Now do the same thing with a paragraph full of code, camelCase variable names, or a foreign language, and that small gap tends to widen, sometimes substantially, because that's exactly the kind of text where the training corpora behind each tokenizer diverge most.
The takeaway isn't a specific multiplier to memorize. It's that token counts are content-dependent and model-dependent at the same time, which means a cost estimate that assumes "roughly four characters per token, universally" will drift the moment your actual traffic includes code snippets, structured data, or a non-English user base.
Input Tokens vs Output Tokens, and Why Output Costs More
Nearly every provider prices output tokens higher than input tokens, sometimes two to four times higher depending on the model tier. That asymmetry matters more than it looks like on a pricing page, because a chat feature or coding assistant often generates output that's comparable in length to, or longer than, the prompt that triggered it. A cost model that only accounts for input length will systematically underestimate the real bill, sometimes by a wide margin, once generation length is factored in properly.
Photo by Alexey Demidov on Pexels
How Small Per-Token Differences Compound at Scale
A gap that looks trivial on a single request stops being trivial once you multiply it out. Say a feature sends roughly a thousand requests a day, and one model's tokenizer happens to count a given class of input a few percent higher than another's, purely as a result of how that text breaks apart. On one request, that difference is invisible. Run the same pattern across a full month of production traffic, with output tokens priced higher than input on top of it, and what started as a rounding error on paper becomes a line item someone in finance is going to ask about.
This is why testing token counts on a single short prompt during a demo tends to be misleading. A demo prompt is usually shorter, cleaner, and more English-heavy than the messy, code-laden, occasionally multilingual traffic a feature actually sees once real users start typing into it. The gap between "what we tested" and "what we shipped" is exactly where estimates go wrong, and it's rarely because anyone did the arithmetic incorrectly. It's because the input they measured wasn't representative of the input they'd actually be billed for.
What Happens When You Switch Models Mid-Project
Comparing providers on price alone, without re-checking token counts on your actual prompts, is one of the more common ways a "we're switching to save money" decision quietly backfires. A lower per-token rate on paper doesn't guarantee a lower bill if that provider's tokenizer happens to break your specific content into noticeably more tokens than the one you switched from. Code-heavy prompts, prompts with a lot of structured JSON, and prompts involving non-English text are the categories most likely to see a meaningful swing between tokenizers, which makes them exactly the categories worth re-testing before signing off on a provider switch.
The safest way to compare isn't to trust either provider's marketing page. It's to run your own representative prompt templates, system prompt included, through each candidate model's tokenizer and look at the actual counts side by side. A rate that's twenty percent cheaper per token doesn't help much if the same content also tokenizes twenty-five percent higher on that model.
The Hidden Multipliers: System Prompts and Chat History
The prompt a user types is rarely the only thing sent to the model. A system prompt setting behavior and tone gets prepended to every single request. In a multi-turn chat interface, unless the application is specifically trimming context, the entire conversation history gets resent with every new message, which means token cost per turn tends to climb as a conversation gets longer, not stay flat.
This is the single most common source of an unexpectedly large bill on a feature that seemed cheap to prototype. A five-message conversation with a modest system prompt can easily carry two to three times the token weight of the visible user message by the final turn, and that weight compounds across every active user session running in parallel.
"The estimate that gets teams into trouble isn't the one for a single request, it's the one that forgets the system prompt and chat history get resent every single turn. Multiply that oversight by a few thousand active sessions and the gap between projected and actual spend gets very real, very fast." - Dennis Traina, founder of 137Foundry
Common Cost-Estimation Mistakes Teams Make
A few patterns show up repeatedly in postmortems on AI feature costs that ran well over projection:
- Estimating from a single test prompt instead of a representative sample of real user traffic, which tends to skew shorter and cleaner than production input.
- Forgetting output tokens are priced differently than input tokens, and often significantly higher.
- Not accounting for chat history resending on every turn of a multi-turn conversation.
- Assuming one model's token-per-character ratio applies to another model, especially when switching providers mid-project to compare pricing.
- Ignoring code and structured data in the mix, which tokenizes less efficiently than plain prose on most models.
Photo by Pavel Zuyonok on Pexels
Where a Token Calculator Fits Into the Workflow
None of this requires memorizing tokenizer internals to plan around. A free AI Token Calculator by EvvyTools lets you paste in representative text and see token counts and estimated costs across GPT-4o, Claude, Gemini, Llama, and other major models side by side, along with a batch mode for projecting cost across a larger volume of requests rather than just one. Running your actual prompt templates, including the system prompt and a realistic chat history length, through a tool like that before committing to a model choice catches most of the surprises described above before they show up on an invoice.
A Simple Pre-Launch Cost Checklist
Before shipping an AI feature that will run at any real volume, it's worth confirming:
- You've measured token counts on realistic sample prompts, not a single short test message.
- Your estimate includes the system prompt on every request, not just the visible user input.
- If the feature is conversational, you've modeled how token cost grows as conversation length increases.
- You've priced input and output tokens separately, using the provider's actual current rates.
- You've compared at least two model providers on the same representative sample, since the cheaper model per-token isn't always the cheaper model per-request once tokenization differences are factored in.
The Estimate Is Only as Good as the Text You Test It On
Token pricing pages make the math look simple: a rate, multiplied by a count. The part that actually determines your bill is the counting step itself, and that step is quietly model-specific in a way that catches most teams off guard exactly once, usually the first time a feature scales past the handful of test prompts it was estimated on. Testing with realistic text, across the actual models you're considering, is the difference between a cost projection and a guess.
For more developer utilities like this one, browse the EvvyTools tools directory, or check the EvvyTools blog for more deep dives into how the tools under the hood actually work. You can also start from the EvvyTools homepage to see the full catalog of free calculators.
