Tokenizer
Type text and see it split into tokens, with the characters-per-token ratio.
Approximation: split on words, numbers and punctuation. A real tokenizer merges frequent subwords, so its count will differ — use the model's tokenizer for billing.
Retrievalaugmentedgenerationgroundsanswersinevidence.
Tokens8
Characters59
Chars / token7.4
Related: Tokenization