How many tokens is a word?
For ordinary English text, about 1.3 tokens per word on average, or roughly 4 characters per token. Code, numbers, other languages and rare words usually take more. Use the provider token counter for exact numbers.
A small chunk of text, often part of a word, that models process; limits and prices are counted in tokens.
A token is the unit of text a language model reads and writes. Before text reaches the model, a tokenizer splits it into pieces: common words are often one token, longer or rarer words are split into several, and spaces, punctuation and code symbols count too. For English prose with common tokenizers, a token averages roughly three quarters of a word, or about four characters, so 1,000 words is around 1,300 to 1,500 tokens. Code, other languages and unusual text often use more tokens per word. Context window sizes, rate limits and API prices are all measured in tokens.
Tokens are how AI usage is limited and billed, so they explain both cost and capacity. A long instructions file, a big log paste or a large file read all cost tokens on every turn they remain in context. Estimating tokens helps you see why trimming context saves money and keeps agents focused. The 2prompt linter estimates the token cost of an instructions file for this reason.
The sentence "Run the tests before you finish." is about 8 tokens. A 200-line CLAUDE.md file of about 1,800 words is roughly 2,400 tokens, loaded every session. A 5,000-line JSON fixture pasted into a chat could easily be 50,000 tokens or more, a sizable share of a context window. Pasting only the relevant records keeps the cost and the noise down.
Tokens are not the same as words or characters, and the count depends on the model tokenizer. The same text can be a different number of tokens for Claude, GPT and Gemini. Treat any rule of thumb as an estimate.
All 28 terms are in the AI glossary. For models side by side, see frontier vs open-weight.
For ordinary English text, about 1.3 tokens per word on average, or roughly 4 characters per token. Code, numbers, other languages and rare words usually take more. Use the provider token counter for exact numbers.
Each output token requires a separate pass through the model, while input can be processed in parallel. Providers typically price output tokens several times higher than input tokens.
Yes. Spaces are often merged into the following word token, but punctuation, indentation, line breaks and symbols in code all take tokens, which is why code is token-heavy.