# Package weka.core.tokenizers

package weka.core.tokenizers

Related Packages
Package
Description
weka.core
Classes
Class
Description
AlphabeticTokenizer
Alphabetic string tokenizer, tokens are to be formed only from contiguous alphabetic sequences.
CharacterDelimitedTokenizer
Abstract superclass for tokenizers that take characters as delimiters.
NGramTokenizer
Splits a string into an n-gram with min and max grams.
Tokenizer
A superclass for all tokenizer algorithms.
WordTokenizer
A simple tokenizer that is using the java.util.StringTokenizer class to tokenize the strings.