TensorZero：别只比每百万 token 单价，LLM API 的隐性成本差距可达 5.3 倍

• Token pricing is misleading: the same input produces 2.65x+ more tokens depending on the model. We got wildly different token counts from identical content using OpenAI, Anthropic, and Google’s official token counting APIs.

• Token efficiency varies by content type. Text, JSON, YAML, and tool definitions all tokenize differently. The cheapest provider changes depending on what you’re sending. The only way to know what you’re actually paying is to measure it.

• OpenAI has the most efficient tokenizer. On tool-heavy workloads, claude-opus-4-7 costs 5.3x more than gpt-5.4 despite list prices being only 2x apart.

Most engineers know they need to evaluate models on their specific task because performance varies. But so does cost: the same input can cost several times more on one provider than another, even when their list prices look similar.

The typical metric for comparing LLM API costs is price per million tokens ($/MTok). Here’s what the major providers charge today:

But not all tokens are equal! Different providers use different tokenizers, so the same input produces wildly different token counts.

What is a tokenizer?

A tokenizer splits text into smaller units (tokens) that the LLM processes. Different tokenizers split the same text differently, producing different token counts.

For example, gpt-5.4 uses the tokenization below:

We sent identical inputs through each provider’s official token counting API and normalized against OpenAI’s:

Details about the input data

We used the following input data for this experiment:

Multiplying list price by tokenizer efficiency gives you what you actually pay to process the same input.

The differences are dramatic. On tool-heavy workloads, claude-opus-4-7 costs 5.3x more than gpt-5.4 even though their list prices are only 2x apart. The rankings also flip depending on what you’re sending: Gemini is the cheapest option for text and structured data, but becomes 46% more expensive than OpenAI on tool definitions.

This analysis only considers base input token prices. In practice, cost comparisons get even more complex when you factor in prompt caching discounts, long-context pricing tiers, output tokens, and thinking tokens.

When choosing the right model for your task, you should compare both performance and cost in a real setting. A model that looks cheaper on paper might cost several times more once you account for how it counts tokens.

The only way to know what you’re actually paying is to measure it.

Our 11.5Kopen-source LLM gateway tracks real-world token usage and cost across providers. You can configure multiple variants and track usage and costs side by side.