Package weka.core.tokenizers

# Class AlphabeticTokenizer

java.lang.Object
weka.core.tokenizers.Tokenizer
weka.core.tokenizers.AlphabeticTokenizer

All Implemented Interfaces:
  `Serializable`, `Enumeration`, `OptionHandler`, `RevisionHandler`

public class AlphabeticTokenizer extends Tokenizer
Alphabetic string tokenizer, tokens are to be formed only from contiguous alphabetic sequences.

Version:
  $Revision: 1.2 $
Author:
  Asrhaf M. Kibriya (amk14@cs.waikato.ac.nz), FracPete (fracpete at waikato dot ac dot nz)
See Also:
  - Serialized Form

## Constructor Summary

Constructors
Constructor
Description
`AlphabeticTokenizer()`

## Method Summary

Modifier and Type
Method
Description
`String`
`getRevision()`
Returns the revision string.
`String`
`globalInfo()`
Returns a string describing the stemmer
`boolean`
`hasMoreElements()`
returns whether there are more elements still
`static void`
`main(String[] args)`
Runs the tokenizer with the given options and strings to tokenize.
`Object`
`nextElement()`
returns the next element
`void`
`tokenize(String s)`
Sets the string to tokenize.

### Methods inherited from class weka.core.tokenizers.Tokenizer

`getOptions, listOptions, runTokenizer, setOptions, tokenize`

### Methods inherited from class java.lang.Object

`equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Methods inherited from interface java.util.Enumeration

`asIterator`

## Constructor Details

### AlphabeticTokenizer

public AlphabeticTokenizer()

## Method Details

### globalInfo

public String globalInfo()
Returns a string describing the stemmer

Specified by:
  `globalInfo` in class `Tokenizer`
Returns:
  a description suitable for displaying in the explorer/experimenter gui

### hasMoreElements

public boolean hasMoreElements()
returns whether there are more elements still

Specified by:
  `hasMoreElements` in interface `Enumeration`
Specified by:
  `hasMoreElements` in class `Tokenizer`
Returns:
  true if there are still more elements

### nextElement

public Object nextElement()
returns the next element

Specified by:
  `nextElement` in interface `Enumeration`
Specified by:
  `nextElement` in class `Tokenizer`
Returns:
  the next element

### tokenize

public void tokenize(String s)
Sets the string to tokenize. Tokenization happens immediately.

Specified by:
  `tokenize` in class `Tokenizer`
Parameters:
  `s` - the string to tokenize

### getRevision

public String getRevision()
Returns the revision string.

Returns:
  the revision

### main

public static void main(String[] args)
Runs the tokenizer with the given options and strings to tokenize. The tokens are printed to stdout.

Parameters:
  `args` - the commandline options and strings to tokenize