# Package weka.core.tokenizers

package weka.core.tokenizers

- Related Packages
  Package
  Description
  weka.core
- Classes
  Class
  Description
  AlphabeticTokenizer
  Alphabetic string tokenizer, tokens are to be formed only from contiguous alphabetic sequences.
  CharacterDelimitedTokenizer
  Abstract superclass for tokenizers that take characters as delimiters.
  CharacterNGramTokenizer
  Splits a string into an n-gram with min and max grams.
  NGramTokenizer
  Splits a string into an n-gram with min and max grams.
  Tokenizer
  A superclass for all tokenizer algorithms.
  WordTokenizer
  A simple tokenizer that is using the java.util.StringTokenizer class to tokenize the strings.