## org.apache.lucene.benchmark.byTask.feeds
Class ContentSource

    java.lang.Object
      org.apache.lucene.benchmark.byTask.feeds.ContentSource

Direct Known Subclasses:
  DirContentSource, EnwikiContentSource, LineDocSource, ReutersContentSource, SingleDocSource, TrecContentSource

    public abstract class ContentSource

extends java.lang.Object

Represents content from a specified source, such as TREC, Reuters etc. A `ContentSource` is responsible for creating `DocData` objects for its documents to be consumed by `DocMaker`. It also keeps track of various statistics, such as how many documents were generated, size in bytes etc.

Supports the following configuration parameters:

- content.source.forever - specifies whether to generate documents forever (default=true).
- content.source.verbose - specifies whether messages should be output by the content source (default=false).
- content.source.encoding - specifies which encoding to use when reading the files of that content source. Certain implementations may define a default value if this parameter is not specified. (default=null).
- content.source.log.step - specifies for how many documents a message should be logged. If set to 0 it means no logging should occur. NOTE: if verbose is set to false, logging should not occur even if logStep is not 0 (default=0).

Field Summary
`protected static int` `BUFFER_SIZE`
`protected java.lang.String` `encoding`
`protected boolean` `forever`
`protected int` `logStep`
`protected boolean` `verbose`

Constructor Summary
`ContentSource()`

Method Summary
`protected void` `addBytes(long numBytes)`
`protected void` `addDoc()`
`abstract void` `close()` Called when reading from this content source is no longer required.
`protected void` `collectFiles(java.io.File dir, java.util.ArrayList files)` A convenience method for collecting all the files of a content source from a given directory.
`long` `getBytesCount()` Returns the number of bytes generated since last reset.
`Config` `getConfig()`
`int` `getDocsCount()` Returns the number of generated documents since last reset.
`protected java.io.InputStream` `getInputStream(java.io.File file)` Returns an `InputStream` over the requested file.
`abstract DocData` `getNextDocData(DocData docData)` Returns the next `DocData` from the content source.
`long` `getTotalBytesCount()` Returns the total number of bytes that were generated by this source.
`int` `getTotalDocsCount()` Returns the total number of generated documents.
`void` `resetInputs()` Resets the input for this content source, so that the test would behave as if it was just started, input-wise.
`void` `setConfig(Config config)` Sets the `Config` for this content source.
`protected boolean` `shouldLog()` Returns true whether it's time to log a message (depending on verbose and the number of documents generated).

Methods inherited from class java.lang.Object
`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

Field Detail

### BUFFER_SIZE

    protected static final int BUFFER_SIZE

See Also:
  Constant Field Values

### forever

    protected boolean forever

### logStep

    protected int logStep

### verbose

    protected boolean verbose

### encoding

    protected java.lang.String encoding

Constructor Detail

### ContentSource

    public ContentSource()

Method Detail

### addBytes

    protected final void addBytes(long numBytes)

### addDoc

    protected final void addDoc()

### collectFiles

    protected final void collectFiles(java.io.File dir,
                                      java.util.ArrayList files)

  A convenience method for collecting all the files of a content source from a given directory. The collected `File` instances are stored in the given `files`.

### getInputStream

    protected java.io.InputStream getInputStream(java.io.File file)
                                          throws java.io.IOException

  Returns an `InputStream` over the requested file. This method attempts to identify the appropriate `InputStream` instance to return based on the file name (e.g., if it ends with .bz2 or .bzip, return a 'bzip' `InputStream`).

  Throws:
    `java.io.IOException`

### shouldLog

    protected final boolean shouldLog()

  Returns true whether it's time to log a message (depending on verbose and the number of documents generated).

### close

    public abstract void close()
                        throws java.io.IOException

  Called when reading from this content source is no longer required.

  Throws:
    `java.io.IOException`

### getBytesCount

    public final long getBytesCount()

  Returns the number of bytes generated since last reset.

### getDocsCount

    public final int getDocsCount()

  Returns the number of generated documents since last reset.

### getConfig

    public final Config getConfig()

### getNextDocData

    public abstract DocData getNextDocData(DocData docData)
                                    throws NoMoreDataException,
                                           java.io.IOException

  Returns the next `DocData` from the content source.

  Throws:
    `NoMoreDataException`
    `java.io.IOException`

### getTotalBytesCount

    public final long getTotalBytesCount()

  Returns the total number of bytes that were generated by this source.

### getTotalDocsCount

    public final int getTotalDocsCount()

  Returns the total number of generated documents.

### resetInputs

    public void resetInputs()
                     throws java.io.IOException

  Resets the input for this content source, so that the test would behave as if it was just started, input-wise.

  NOTE: the default implementation resets the number of bytes and documents generated since the last reset, so it's important to call super.resetInputs in case you override this method.

  Throws:
    `java.io.IOException`

### setConfig

    public void setConfig(Config config)

  Sets the `Config` for this content source. If you override this method, you must call super.setConfig.

Copyright © 2000-2011 Apache Software Foundation. All Rights Reserved.