How words are found and counted
The text is scanned for runs of Unicode letters and digits, optionally joined by an apostrophe between letters (straight or curly, with the curly form converted to a straight one), using the pattern [\p{L}\p{N}]+(?:['’][\p{L}\p{N}]+)*, so it works for accented and non-Latin scripts too. Each run is one token. By default tokens are lower-cased so "The" and "the" combine. Then the filters run in order: tokens shorter than the minimum length are skipped, and with "Exclude common words" on, any token in a built-in list of 61 English function words (the, and, of, to, is, you, can and similar) is skipped. What remains is counted, sorted by count high to low, and ties are broken alphabetically.
Worked example
Input: The cat saw the dog. The dog saw the cat run, and the cat didn't stop.
| Word | Count | % of 16 words |
|---|---|---|
| the | 5 | 31.3% |
| cat | 3 | 18.8% |
| dog | 2 | 12.5% |
| saw | 2 | 12.5% |
| and, didn't, run, stop | 1 each | 6.3% each |
With "Exclude common words" on, "the" and "and" disappear and the total falls to 10, so the percentages are recalculated: cat is 30.0%, dog and saw 20.0% each. A minimum length of 4 on top of that leaves only "didn't" and "stop" at 50.0% each. Percentages are always based on the filtered total, not the original word count.
Details that affect the numbers
- Apostrophes are handled. Straight and curly apostrophes between letters stay inside the word, and the two forms are merged, so "don’t" from Word and "don't" from a plain text file are one entry.
- Quote marks are stripped. Single quotes around a word, as in
'quoted', are not part of the token, so it is counted with the bare word. - Hyphens split. "well-known" counts as "well" and "known".
- No stemming. "run", "runs" and "running" are three different entries.
- Single words only. The table cannot show two- or three-word phrases.
- Display limits. You can show the top 25, 50, 100 or 500 entries. The words-counted figure still covers everything that passed the filters. "Copy table" copies each row as the word, a tab and the count, without the percentage.
Uses
In an essay or article, the top of the list quickly reveals crutch words. In a long document, enabling "Exclude common words" surfaces the topic words instead. In ordinary English prose, "the" is usually the most common word at roughly 5 to 7 percent of all words. For the overall size of a text, use the word counter. To check a single target phrase for SEO, the keyword density checker is the better fit, and readability score measures how hard the text is to read.