ngrams

ngrams, a Python code which analyzes a string or text against observed frequences of ngrams (strings of 1, 2, 3, 4, or 5 alphabetic characters.)

Licensing:

The information on this web page is distributed under the MIT license.

Languages:

ngrams is available in a Python version.

Related Data and Programs:

german, a dataset directory which contains some short German texts;

ngrams, a dataset directory which contains information about the observed frequency of "ngrams" (particular sequences of n letters) in English text.

text, a dataset directory which contains some short texts in English;

text_to_wordlist, a Python code which shows how to start with a text file, read its information into a single long string, and divide that string into individual words. This allows an investigator to analyze the text for patterns.

words, a dataset directory which contains lists of words;

Source Code:

• ngrams.py, the source code.
• ngrams.sh, runs all the tests.
• ngrams.txt, the output file.

• desiderata.txt, a medium length text file.
• HELLOWORLD.txt, a very short text file.
• english_monograms.txt a datafile for English monograms.
• english_bigrams.txt a datafile for English bigrams.
• english_trigrams.txt a datafile for English trigrams.
• english_quadgrams.txt a datafile for English quadgrams.
• english_quintgrams.txt a datafile for English quintgrams.