Lucene++ - a full-featured, c++ search engine
API Documentation

Public Member Functions | Static Public Member Functions | Protected Attributes
Lucene::NumericTokenStream Class Reference

This class provides a TokenStream for indexing numeric values that can be used by NumericRangeQuery or NumericRangeFilter. More...

`#include <NumericTokenStream.h>`

Inheritance diagram for Lucene::NumericTokenStream:

## Public Member Functions

| NumericTokenStream ()
| Creates a token stream for numeric values using the default precisionStep {} (4). The stream is not yet initialized, before using set a value using the various setValue() methods. More...
| NumericTokenStream (int32_t precisionStep)
| Creates a token stream for numeric values with the specified precisionStep. The stream is not yet initialized, before using set a value using the various setValue() methods. More...
| NumericTokenStream (const AttributeSourcePtr &source, int32_t precisionStep)
| Creates a token stream for numeric values with the specified precisionStep using the given {}. The stream is not yet initialized, before using set a value using the various setValue() methods. More...
| NumericTokenStream (const AttributeFactoryPtr &factory, int32_t precisionStep)
| Creates a token stream for numeric values with the specified precisionStep using the given {}. The stream is not yet initialized, before using set a value using the various setValue() methods. More...
virtual | ~NumericTokenStream ()
virtual String | getClassName ()
boost::shared_ptr< NumericTokenStream > | shared_from_this ()
NumericTokenStreamPtr | setLongValue (int64_t value)
| Initializes the token stream with the supplied long value. More...
NumericTokenStreamPtr | setIntValue (int32_t value)
| Initializes the token stream with the supplied int value. More...
NumericTokenStreamPtr | setDoubleValue (double value)
| Initializes the token stream with the supplied double value. More...
virtual void | reset ()

Resets this stream to the beginning. This is an optional operation, so subclasses may or may not implement this method. reset() is not needed for the standard indexing process. However, if the tokens of a TokenStream are intended to be consumed more than once, it is necessary to implement reset(). Note that if your TokenStream caches tokens and feeds them back again after a reset, it is imperative that you clone the tokens when you store them away (on the first pass) as well as when you return them (on future passes after reset()). More...

virtual bool | incrementToken ()
| Consumers (ie., IndexWriter) use this method to advance the stream to the next token. Implementing classes must implement this method and update the appropriate Attributes with the attributes of the next token. More...
virtual String | toString ()
| Returns a string representation of the object. More...
Public Member Functions inherited from Lucene::TokenStream
virtual | ~TokenStream ()
boost::shared_ptr< TokenStream > | shared_from_this ()
virtual void | end ()
| This method is called by the consumer after the last token has been consumed, after incrementToken() returned false (using the new TokenStream API). Streams implementing the old API should upgrade to use this feature. More...
virtual void | close ()
| Releases resources associated with this stream. More...
Public Member Functions inherited from Lucene::AttributeSource
| AttributeSource ()
| An AttributeSource using the default attribute factory DefaultAttributeFactory. More...
| AttributeSource (const AttributeSourcePtr &input)
| An AttributeSource that uses the same attributes as the supplied one. More...
| AttributeSource (const AttributeFactoryPtr &factory)
| An AttributeSource using the supplied AttributeFactory for creating new Attribute instances. More...
virtual | ~AttributeSource ()
boost::shared_ptr< AttributeSource > | shared_from_this ()
AttributeFactoryPtr | getAttributeFactory ()
| returns the used AttributeFactory. More...
template<class ATTR >
boost::shared_ptr< ATTR > | addAttribute ()
| This method first checks if an instance of that class is already in this AttributeSource and returns it. Otherwise a new instance is created, added to this AttributeSource and returned. More...
void | addAttribute (const String &className, const AttributePtr &attrImpl)
| Adds a custom Attribute instance. More...
bool | hasAttributes ()
| Returns true if this AttributeSource has any attributes. More...
template<class ATTR >
bool | hasAttribute ()
| Returns true, if this AttributeSource contains the passed-in Attribute. More...
template<class ATTR >
boost::shared_ptr< ATTR > | getAttribute ()
| Returns the instance of the passed in Attribute contained in this AttributeSource. More...
void | clearAttributes ()
| Resets all Attributes in this AttributeSource by calling AttributeImpl#clear() on each Attribute implementation. More...
AttributeSourceStatePtr | captureState ()
| Captures the state of all Attributes. The return value can be passed to restoreState to restore the state of this or another AttributeSource. More...
void | restoreState (const AttributeSourceStatePtr &state)

Restores this state by copying the values of all attribute implementations that this state contains into the attributes implementations of the targetStream. The targetStream must contain a corresponding instance for each argument contained in this state (eg. it is not possible to restore the state of an AttributeSource containing a TermAttribute into a AttributeSource using a Token instance as implementation). More...

virtual int32_t | hashCode ()
| Return hash code for this object. More...
virtual bool | equals (const LuceneObjectPtr &other)
| Return whether two objects are equal. More...
AttributeSourcePtr | cloneAttributes ()
| Performs a clone of all AttributeImpl instances returned in a new AttributeSource instance. This method can be used to eg. create another TokenStream with exactly the same attributes (using {}) More...
Collection< AttributePtr > | getAttributes ()
| Return a vector of attributes based on currentState. More...
Public Member Functions inherited from Lucene::LuceneObject
virtual | ~LuceneObject ()
virtual void | initialize ()
| Called directly after instantiation to create objects that depend on this object being fully constructed. More...
virtual LuceneObjectPtr | clone (const LuceneObjectPtr &other=LuceneObjectPtr())
| Return clone of this object. More...
virtual int32_t | compareTo (const LuceneObjectPtr &other)
| Compare two objects. More...
Public Member Functions inherited from Lucene::LuceneSync
virtual | ~LuceneSync ()
virtual SynchronizePtr | getSync ()
| Return this object synchronize lock. More...
virtual LuceneSignalPtr | getSignal ()
| Return this object signal. More...
virtual void | lock (int32_t timeout=0)
| Lock this object using an optional timeout. More...
virtual void | unlock ()
| Unlock this object. More...
virtual bool | holdsLock ()
| Returns true if this object is currently locked by current thread. More...
virtual void | wait (int32_t timeout=0)
| Wait for signal using an optional timeout. More...
virtual void | notifyAll ()
| Notify all threads waiting for signal. More...

## Static Public Member Functions

static String | _getClassName ()
static const String & | TOKEN_TYPE_FULL_PREC ()
| The full precision token gets this token type assigned. More...
static const String & | TOKEN_TYPE_LOWER_PREC ()
| The lower precision tokens gets this token type assigned. More...
Static Public Member Functions inherited from Lucene::TokenStream
static String | _getClassName ()
Static Public Member Functions inherited from Lucene::AttributeSource
static String | _getClassName ()

## Protected Attributes

TermAttributePtr | termAtt
TypeAttributePtr | typeAtt
PositionIncrementAttributePtr | posIncrAtt
int32_t | shift
int32_t | valSize
int32_t | precisionStep
int64_t | value
Protected Attributes inherited from Lucene::AttributeSource
AttributeFactoryPtr | factory
MapStringAttribute | attributes
AttributeSourceStatePtr | currentState
Protected Attributes inherited from Lucene::LuceneSync
SynchronizePtr | objectLock
LuceneSignalPtr | objectSignal

## Additional Inherited Members

Protected Member Functions inherited from Lucene::TokenStream
| TokenStream ()
| A TokenStream using the default attribute factory. More...
| TokenStream (const AttributeSourcePtr &input)
| A TokenStream that uses the same attributes as the supplied one. More...
| TokenStream (const AttributeFactoryPtr &factory)
| A TokenStream using the supplied AttributeFactory for creating new Attribute instances. More...
Protected Member Functions inherited from Lucene::AttributeSource
AttributePtr | getAttribute (const String &className)
| The caller must pass in a className value. This method checks if an instance of that class is already in this AttributeSource and returns it. More...
bool | hasAttribute (const String &className)
| Returns true, if this AttributeSource contains the passed-in Attribute. More...
void | computeCurrentState ()
Protected Member Functions inherited from Lucene::LuceneObject
| LuceneObject ()

## Detailed Description

This class provides a TokenStream for indexing numeric values that can be used by NumericRangeQuery or NumericRangeFilter.

Note that for simple usage, NumericField is recommended. NumericField disables norms and term freqs, as they are not usually needed during searching. If you need to change these settings, you should use this class.

See NumericField for capabilities of fields indexed numerically.

Here's an example usage, for an int field:

FieldPtr field = newLucene<Field>(name, newLucene<NumericTokenStream>(precisionStep)->setIntValue(value)); field->setOmitNorms(true); field->setOmitTermFreqAndPositions(true); document->add(field);

For optimal performance, re-use the TokenStream and Field instance for more than one document:

NumericTokenStreamPtr stream = newLucene<NumericTokenStream>(precisionStep); FieldPtr field = newLucene<Field>(name, stream); field->setOmitNorms(true); field->setOmitTermFreqAndPositions(true); DocumentPtr document = newLucene<Document>(); document->add(field);

for (all documents) { stream->setIntValue(value); writer->addDocument(document); }

This stream is not intended to be used in analyzers; it's more for iterating the different precisions during indexing a specific numeric value.

NOTE: as token streams are only consumed once the document is added to the index, if you index more than one numeric field, use a separate NumericTokenStream * instance for each.

See NumericRangeQuery for more details on the precisionStep parameter as well as how numeric fields work under the hood.

## Constructor & Destructor Documentation

## ◆ NumericTokenStream() [1/4]

Lucene::NumericTokenStream::NumericTokenStream | ( |  | ) |

Creates a token stream for numeric values using the default precisionStep {} (4). The stream is not yet initialized, before using set a value using the various setValue() methods.

## ◆ NumericTokenStream() [2/4]

Lucene::NumericTokenStream::NumericTokenStream | ( | int32_t | precisionStep | ) |

Creates a token stream for numeric values with the specified precisionStep. The stream is not yet initialized, before using set a value using the various setValue() methods.

## ◆ NumericTokenStream() [3/4]

Lucene::NumericTokenStream::NumericTokenStream | ( | const AttributeSourcePtr & | source,
|  | int32_t | precisionStep
| ) |  |

Creates a token stream for numeric values with the specified precisionStep using the given {}. The stream is not yet initialized, before using set a value using the various setValue() methods.

## ◆ NumericTokenStream() [4/4]

Lucene::NumericTokenStream::NumericTokenStream | ( | const AttributeFactoryPtr & | factory,
|  | int32_t | precisionStep
| ) |  |

Creates a token stream for numeric values with the specified precisionStep using the given {}. The stream is not yet initialized, before using set a value using the various setValue() methods.

## ◆ ~NumericTokenStream()

virtual Lucene::NumericTokenStream::~NumericTokenStream | ( |  | ) |

virtual

## Member Function Documentation

## ◆ _getClassName()

static String Lucene::NumericTokenStream::_getClassName | ( |  | ) |

inline static

## ◆ getClassName()

virtual String Lucene::NumericTokenStream::getClassName | ( |  | ) |

inline virtual

Reimplemented from Lucene::TokenStream.

## ◆ incrementToken()

virtual bool Lucene::NumericTokenStream::incrementToken | ( |  | ) |

virtual

Consumers (ie., IndexWriter) use this method to advance the stream to the next token. Implementing classes must implement this method and update the appropriate Attributes with the attributes of the next token.

The producer must make no assumptions about the attributes after the method has been returned: the caller may arbitrarily change it. If the producer needs to preserve the state for subsequent calls, it can use {} to create a copy of the current attribute state. This method is called for every token of a document, so an efficient implementation is crucial for good performance. To avoid calls to #addAttribute(Class)} and #getAttribute(Class)}, references to all Attribute}s that this stream uses should be retrieved during instantiation. To ensure that filters and consumers know which attributes are available, the attributes must be added during instantiation. Filters and consumers are not required to check for availability of attributes in { #incrementToken()}. false for end of stream; true otherwise

Implements Lucene::TokenStream.

## ◆ reset()

virtual void Lucene::NumericTokenStream::reset | ( |  | ) |

virtual

Resets this stream to the beginning. This is an optional operation, so subclasses may or may not implement this method. reset() is not needed for the standard indexing process. However, if the tokens of a TokenStream are intended to be consumed more than once, it is necessary to implement reset(). Note that if your TokenStream caches tokens and feeds them back again after a reset, it is imperative that you clone the tokens when you store them away (on the first pass) as well as when you return them (on future passes after reset()).

Reimplemented from Lucene::TokenStream.

## ◆ setDoubleValue()

NumericTokenStreamPtr Lucene::NumericTokenStream::setDoubleValue | ( | double | value | ) |

Initializes the token stream with the supplied double value.

Parameters

  value | the value, for which this TokenStream should enumerate tokens.

Returns
  this instance, because of this you can use it the following way: newLucene<Field>(name, newLucene<NumericTokenStream>(precisionStep)->setDoubleValue(value))

## ◆ setIntValue()

NumericTokenStreamPtr Lucene::NumericTokenStream::setIntValue | ( | int32_t | value | ) |

Initializes the token stream with the supplied int value.

Parameters

  value | the value, for which this TokenStream should enumerate tokens.

Returns
  this instance, because of this you can use it the following way: newLucene<Field>(name, newLucene<NumericTokenStream>(precisionStep)->setIntValue(value))

## ◆ setLongValue()

NumericTokenStreamPtr Lucene::NumericTokenStream::setLongValue | ( | int64_t | value | ) |

Initializes the token stream with the supplied long value.

Parameters

  value | the value, for which this TokenStream should enumerate tokens.

Returns
  this instance, because of this you can use it the following way: newLucene<Field>(name, newLucene<NumericTokenStream>(precisionStep)->setLongValue(value))

## ◆ shared_from_this()

boost::shared_ptr< NumericTokenStream > Lucene::NumericTokenStream::shared_from_this | ( |  | ) |

inline

## ◆ TOKEN_TYPE_FULL_PREC()

static const String& Lucene::NumericTokenStream::TOKEN_TYPE_FULL_PREC | ( |  | ) |

static

The full precision token gets this token type assigned.

## ◆ TOKEN_TYPE_LOWER_PREC()

static const String& Lucene::NumericTokenStream::TOKEN_TYPE_LOWER_PREC | ( |  | ) |

static

The lower precision tokens gets this token type assigned.

## ◆ toString()

virtual String Lucene::NumericTokenStream::toString | ( |  | ) |

virtual

Returns a string representation of the object.

Reimplemented from Lucene::AttributeSource.

## Field Documentation

## ◆ posIncrAtt

PositionIncrementAttributePtr Lucene::NumericTokenStream::posIncrAtt

protected

## ◆ precisionStep

int32_t Lucene::NumericTokenStream::precisionStep

protected

## ◆ shift

int32_t Lucene::NumericTokenStream::shift

protected

## ◆ termAtt

TermAttributePtr Lucene::NumericTokenStream::termAtt

protected

## ◆ typeAtt

TypeAttributePtr Lucene::NumericTokenStream::typeAtt

protected

## ◆ valSize

int32_t Lucene::NumericTokenStream::valSize

protected

## ◆ value

int64_t Lucene::NumericTokenStream::value

protected

The documentation for this class was generated from the following file:

- NumericTokenStream.h

clucene.sourceforge.net