Lucene++ - a full-featured, c++ search engine
API Documentation

- Main Page
- Related Pages
- Namespaces
- Data Structures
- Files

- Data Structures
- Class Hierarchy
- Data Fields

All Data Structures Namespaces Files Functions Variables Typedefs Enumerations Enumerator Macros Pages
Public Member Functions | Static Public Member Functions | Data Fields | Protected Member Functions | Protected Attributes | Static Protected Attributes
Lucene::IndexFileDeleter Class Reference

This class keeps track of each SegmentInfos instance that is still "live", either because it corresponds to a segments_N file in the Directory (a "commit", ie. a committed SegmentInfos) or because it's an in-memory SegmentInfos that a writer is actively updating but has not yet committed. This class uses simple reference counting to map the live SegmentInfos instances to individual files in the Directory. More...

`#include <IndexFileDeleter.h>`

Inheritance diagram for Lucene::IndexFileDeleter:

## Public Member Functions

| IndexFileDeleter (DirectoryPtr directory, IndexDeletionPolicyPtr policy, SegmentInfosPtr segmentInfos, InfoStreamPtr infoStream, DocumentsWriterPtr docWriter, HashSet< String > synced)
| Initialize the deleter: find all previous commits in the Directory, incref the files they reference, call the policy to let it delete commits. This will remove any files not referenced by any of the commits. More...
virtual | ~IndexFileDeleter ()
virtual String | getClassName ()
boost::shared_ptr < IndexFileDeleter > | shared_from_this ()
void | setInfoStream (InfoStreamPtr infoStream)
SegmentInfosPtr | getLastSegmentInfos ()
void | refresh (const String &segmentName)
| Writer calls this when it has hit an error and had to roll back, to tell us that there may now be unreferenced files in the filesystem. So we re-list the filesystem and delete such files. If segmentName is non-null, we will only delete files corresponding to that segment. More...
void | refresh ()
void | close ()
void | checkpoint (SegmentInfosPtr segmentInfos, bool isCommit)
| For definition of "check point" see IndexWriter comments: "Clarification: Check Points (and commits)". Writer calls this when it has made a "consistent change" to the index, meaning new files are written to the index and the in-memory SegmentInfos have been modified to point to those files. More...
void | incRef (SegmentInfosPtr segmentInfos, bool isCommit)
void | incRef (HashSet< String > files)
void | incRef (const String &fileName)
void | decRef (HashSet< String > files)
void | decRef (const String &fileName)
void | decRef (SegmentInfosPtr segmentInfos)
bool | exists (const String &fileName)
void | deleteFiles (HashSet< String > files)
void | deleteNewFiles (HashSet< String > files)
| Deletes the specified files, but only if they are new (have not yet been incref'd). More...
void | deleteFile (const String &fileName)
Public Member Functions inherited from Lucene::LuceneObject
virtual | ~LuceneObject ()
virtual void | initialize ()
| Called directly after instantiation to create objects that depend on this object being fully constructed. More...
virtual LuceneObjectPtr | clone (LuceneObjectPtr other=LuceneObjectPtr())
| Return clone of this object. More...
virtual int32_t | hashCode ()
| Return hash code for this object. More...
virtual bool | equals (LuceneObjectPtr other)
| Return whether two objects are equal. More...
virtual int32_t | compareTo (LuceneObjectPtr other)
| Compare two objects. More...
virtual String | toString ()
| Returns a string representation of the object. More...
Public Member Functions inherited from Lucene::LuceneSync
virtual | ~LuceneSync ()
virtual SynchronizePtr | getSync ()
| Return this object synchronize lock. More...
virtual LuceneSignalPtr | getSignal ()
| Return this object signal. More...
virtual void | lock (int32_t timeout=0)
| Lock this object using an optional timeout. More...
virtual void | unlock ()
| Unlock this object. More...
virtual bool | holdsLock ()
| Returns true if this object is currently locked by current thread. More...
virtual void | wait (int32_t timeout=0)
| Wait for signal using an optional timeout. More...
virtual void | notifyAll ()
| Notify all threads waiting for signal. More...

## Static Public Member Functions

static String | _getClassName ()

## Data Fields

bool | startingCommitDeleted

## Protected Member Functions

void | message (const String &message)
void | deleteCommits ()
| Remove the CommitPoints in the commitsToDelete List by DecRef'ing all files from each SegmentInfos. More...
void | deletePendingFiles ()
RefCountPtr | getRefCount (const String &fileName)
Protected Member Functions inherited from Lucene::LuceneObject
| LuceneObject ()

## Protected Attributes

HashSet< String > | deletable
| Files that we tried to delete but failed (likely because they are open and we are running on Windows), so we will retry them again later. More...
MapStringRefCount | refCounts
| Reference count for all files in the index. Counts how many existing commits reference a file. More...
Collection< IndexCommitPtr > | commits
| Holds all commits (segments_N) currently in the index. This will have just 1 commit if you are using the default delete policy (KeepOnlyLastCommitDeletionPolicy). Other policies may leave commit points live for longer in which case this list would be longer than 1. More...
Collection< HashSet< String > > | lastFiles
| Holds files we had incref'd from the previous non-commit checkpoint. More...
Collection< CommitPointPtr > | commitsToDelete
| Commits that the IndexDeletionPolicy have decided to delete. More...
InfoStreamPtr | infoStream
DirectoryPtr | directory
IndexDeletionPolicyPtr | policy
DocumentsWriterPtr | docWriter
SegmentInfosPtr | lastSegmentInfos
HashSet< String > | synced

## Static Protected Attributes

static bool | VERBOSE_REF_COUNTS
| Change to true to see details of reference counts when infoStream != null. More...

## Detailed Description

This class keeps track of each SegmentInfos instance that is still "live", either because it corresponds to a segments_N file in the Directory (a "commit", ie. a committed SegmentInfos) or because it's an in-memory SegmentInfos that a writer is actively updating but has not yet committed. This class uses simple reference counting to map the live SegmentInfos instances to individual files in the Directory.

The same directory file may be referenced by more than one IndexCommit, i.e. more than one SegmentInfos. Therefore we count how many commits reference each file. When all the commits referencing a certain file have been deleted, the refcount for that file becomes zero, and the file is deleted.

A separate deletion policy interface (IndexDeletionPolicy) is consulted on creation (onInit) and once per commit (onCommit), to decide when a commit should be removed.

It is the business of the IndexDeletionPolicy to choose when to delete commit points. The actual mechanics of file deletion, retrying, etc, derived from the deletion of commit points is the business of the IndexFileDeleter.

The current default deletion policy is KeepOnlyLastCommitDeletionPolicy, which removes all prior commits when a new commit has completed. This matches the behavior before 2.2.

Note that you must hold the write.lock before instantiating this class. It opens segments_N file(s) directly with no retry logic.

## Constructor & Destructor Documentation

Lucene::IndexFileDeleter::IndexFileDeleter | ( | DirectoryPtr | directory,
|  | IndexDeletionPolicyPtr | policy,
|  | SegmentInfosPtr | segmentInfos,
|  | InfoStreamPtr | infoStream,
|  | DocumentsWriterPtr | docWriter,
|  | HashSet< String > | synced
| ) |  |

Initialize the deleter: find all previous commits in the Directory, incref the files they reference, call the policy to let it delete commits. This will remove any files not referenced by any of the commits.

virtual Lucene::IndexFileDeleter::~IndexFileDeleter | ( | ) |  |

virtual

## Member Function Documentation

static String Lucene::IndexFileDeleter::_getClassName | ( | ) |  |

inline static

void Lucene::IndexFileDeleter::checkpoint | ( | SegmentInfosPtr | segmentInfos,
|  | bool | isCommit
| ) |  |

For definition of "check point" see IndexWriter comments: "Clarification: Check Points (and commits)". Writer calls this when it has made a "consistent change" to the index, meaning new files are written to the index and the in-memory SegmentInfos have been modified to point to those files.

This may or may not be a commit (segments_N may or may not have been written).

We simply incref the files referenced by the new SegmentInfos and decref the files we had previously seen (if any).

If this is a commit, we also call the policy to give it a chance to remove other commits. If any commits are removed, we decref their files as well.

void Lucene::IndexFileDeleter::close | ( | ) |  |

void Lucene::IndexFileDeleter::decRef | ( | HashSet< String > | files) |  |

void Lucene::IndexFileDeleter::decRef | ( | const String & | fileName) |  |

void Lucene::IndexFileDeleter::decRef | ( | SegmentInfosPtr | segmentInfos) |  |

void Lucene::IndexFileDeleter::deleteCommits | ( | ) |  |

protected

Remove the CommitPoints in the commitsToDelete List by DecRef'ing all files from each SegmentInfos.

void Lucene::IndexFileDeleter::deleteFile | ( | const String & | fileName) |  |

void Lucene::IndexFileDeleter::deleteFiles | ( | HashSet< String > | files) |  |

void Lucene::IndexFileDeleter::deleteNewFiles | ( | HashSet< String > | files) |  |

Deletes the specified files, but only if they are new (have not yet been incref'd).

void Lucene::IndexFileDeleter::deletePendingFiles | ( | ) |  |

protected

bool Lucene::IndexFileDeleter::exists | ( | const String & | fileName) |  |

virtual String Lucene::IndexFileDeleter::getClassName | ( | ) |  |

inline virtual

SegmentInfosPtr Lucene::IndexFileDeleter::getLastSegmentInfos | ( | ) |  |

RefCountPtr Lucene::IndexFileDeleter::getRefCount | ( | const String & | fileName) |  |

protected

void Lucene::IndexFileDeleter::incRef | ( | SegmentInfosPtr | segmentInfos,
|  | bool | isCommit
| ) |  |

void Lucene::IndexFileDeleter::incRef | ( | HashSet< String > | files) |  |

void Lucene::IndexFileDeleter::incRef | ( | const String & | fileName) |  |

void Lucene::IndexFileDeleter::message | ( | const String & | message) |  |

protected

void Lucene::IndexFileDeleter::refresh | ( | const String & | segmentName) |  |

Writer calls this when it has hit an error and had to roll back, to tell us that there may now be unreferenced files in the filesystem. So we re-list the filesystem and delete such files. If segmentName is non-null, we will only delete files corresponding to that segment.

void Lucene::IndexFileDeleter::refresh | ( | ) |  |

void Lucene::IndexFileDeleter::setInfoStream | ( | InfoStreamPtr | infoStream) |  |

boost::shared_ptr< IndexFileDeleter > Lucene::IndexFileDeleter::shared_from_this | ( | ) |  |

inline

## Field Documentation

Collection<IndexCommitPtr> Lucene::IndexFileDeleter::commits

protected

Holds all commits (segments_N) currently in the index. This will have just 1 commit if you are using the default delete policy (KeepOnlyLastCommitDeletionPolicy). Other policies may leave commit points live for longer in which case this list would be longer than 1.

Collection<CommitPointPtr> Lucene::IndexFileDeleter::commitsToDelete

protected

Commits that the IndexDeletionPolicy have decided to delete.

HashSet<String> Lucene::IndexFileDeleter::deletable

protected

Files that we tried to delete but failed (likely because they are open and we are running on Windows), so we will retry them again later.

DirectoryPtr Lucene::IndexFileDeleter::directory

protected

DocumentsWriterPtr Lucene::IndexFileDeleter::docWriter

protected

InfoStreamPtr Lucene::IndexFileDeleter::infoStream

protected

Collection< HashSet<String> > Lucene::IndexFileDeleter::lastFiles

protected

Holds files we had incref'd from the previous non-commit checkpoint.

SegmentInfosPtr Lucene::IndexFileDeleter::lastSegmentInfos

protected

IndexDeletionPolicyPtr Lucene::IndexFileDeleter::policy

protected

MapStringRefCount Lucene::IndexFileDeleter::refCounts

protected

Reference count for all files in the index. Counts how many existing commits reference a file.

bool Lucene::IndexFileDeleter::startingCommitDeleted

HashSet<String> Lucene::IndexFileDeleter::synced

protected

bool Lucene::IndexFileDeleter::VERBOSE_REF_COUNTS

static protected

Change to true to see details of reference counts when infoStream != null.

The documentation for this class was generated from the following file:

- IndexFileDeleter.h

clucene.sourceforge.net