## Uses of Class
org.apache.lucene.analysis.Tokenizer

- Packages that use Tokenizer
  Package | Description
  --- | ---
  org.apache.lucene.analysis | API and code to convert text into indexable/searchable tokens.
  org.apache.lucene.analysis.standard | A fast grammar-based tokenizer constructed with JFlex.

- - ### Uses of Tokenizer in org.apache.lucene.analysis

    Subclasses of Tokenizer in org.apache.lucene.analysis
    Modifier and Type | Class and Description
    --- | ---
    `class` | `CharTokenizer` An abstract base class for simple, character-oriented tokenizers.
    `class` | `KeywordTokenizer` Emits the entire input as a single token.
    `class` | `LetterTokenizer` A LetterTokenizer is a tokenizer that divides text at non-letters.
    `class` | `LowerCaseTokenizer` LowerCaseTokenizer performs the function of LetterTokenizer and LowerCaseFilter together.
    `class` | `SinkTokenizer` Deprecated. Use `TeeSinkTokenFilter` instead
    `class` | `WhitespaceTokenizer` A WhitespaceTokenizer is a tokenizer that divides text at whitespace.

  - ### Uses of Tokenizer in org.apache.lucene.analysis.standard

    Subclasses of Tokenizer in org.apache.lucene.analysis.standard
    Modifier and Type | Class and Description
    --- | ---
    `class` | `StandardTokenizer` A grammar-based tokenizer constructed with JFlex

Copyright © 2000-2016 Apache Software Foundation. All Rights Reserved.