edu.isi.pegasus.planner.catalog.replica.impl

## Class FlushedCache

- java.lang.Object
- - edu.isi.pegasus.planner.catalog.replica.impl.FlushedCache

All Implemented Interfaces:
  Catalog, ReplicaCatalog

    public class FlushedCache
    extends java.lang.Object
    implements ReplicaCatalog

This class implements a replica catalog which directly writes to the output file. This is done so that the Cache file generation would not require storing all RC entries in memory. The implementation does not maintain any internal state, and is not aware if same entries are being written multiple times. The class using this implementation needs to ensure only valid entries are passed through to the insert method, and that entries are not being duplicated.

Version:
  $Revision: 5907 $
Author:
  Rajiv Mayani

### Field Summary

Fields
`private static int` `m_avg_line_size` No.
`private int` `m_buffer_size` No.
`private int` `m_line_count` No.
`protected java.io.BufferedWriter` `m_out` A buffered writer, to buffer all file writes being performed.

### Fields inherited from interface edu.isi.pegasus.planner.catalog.ReplicaCatalog

`BATCH_KEY, c_prefix, DB_PREFIX, PROXY_KEY`

### Fields inherited from interface edu.isi.pegasus.planner.catalog.Catalog

`DB_ALL_PREFIX`

### Constructor Summary

Constructors
Constructor and Description
`FlushedCache()`

### Method Summary

All Methods Instance Methods Concrete Methods
`int` `clear()` Removes everything.
`void` `close()` This operation will dump the in-memory representation back onto disk.
`boolean` `connect(java.util.Properties props)` Establishes a connection to the database from the properties.
`boolean` `connect(java.lang.String filename)` Opens the file for writing.
`int` `delete(java.util.Map x, boolean matchAttributes)` Deletes multiple mappings into the replica catalog.
`int` `delete(java.lang.String lfn, ReplicaCatalogEntry tuple)` Deletes a very specific mapping from the replica catalog.
`int` `delete(java.lang.String lfn, java.lang.String pfn)` Deletes a specific mapping from the replica catalog.
`int` `delete(java.lang.String lfn, java.lang.String name, java.lang.Object value)` Deletes all PFN entries for a given LFN from the replica catalog where the PFN attribute is found, and matches exactly the object value.
`int` `deleteByResource(java.lang.String lfn, java.lang.String handle)` Deletes all PFN entries for a given LFN from the replica catalog where the resource handle is found.
`private void` `flush()` Check if the buffer is full i.e.
`int` `insert(java.util.Map x)` Inserts multiple mappings into the replica catalog.
`int` `insert(java.lang.String lfn, ReplicaCatalogEntry tuple)` Inserts a new mapping into the replica catalog.
`int` `insert(java.lang.String lfn, java.lang.String pfn, java.lang.String handle)` Inserts a new mapping into the replica catalog.
`boolean` `isClosed()` Predicate to check, if the connection with the catalog's implementation is still active.
`java.util.Set` `list()` Lists all logical filenames in the catalog.
`java.util.Set` `list(java.lang.String constraint)` Lists a subset of all logical filenames in the catalog.
`java.util.Map` `lookup(java.util.Map constraints)` Retrieves multiple entries for a given logical filename, up to the complete catalog.
`java.util.Map` `lookup(java.util.Set lfns)` Retrieves multiple entries for a given logical filename, up to the complete catalog.
`java.util.Map` `lookup(java.util.Set lfns, java.lang.String handle)` Retrieves multiple entries for a given logical filename, up to the complete catalog.
`java.util.Collection` `lookup(java.lang.String lfn)` Retrieves all entries for a given LFN from the replica catalog.
`java.lang.String` `lookup(java.lang.String lfn, java.lang.String handle)` Retrieves the entry for a given filename and resource handle from the replica catalog.
`java.util.Map` `lookupNoAttributes(java.util.Set lfns)` Retrieves multiple entries for a given logical filename, up to the complete catalog.
`java.util.Map` `lookupNoAttributes(java.util.Set lfns, java.lang.String handle)` Retrieves multiple entries for a given logical filename, up to the complete catalog.
`java.util.Set` `lookupNoAttributes(java.lang.String lfn)` Retrieves all entries for a given LFN from the replica catalog.
`java.lang.String` `quote(Escape e, java.lang.String s)` Quotes a string only if necessary.
`int` `remove(java.util.Set lfns)` Removes all mappings for a set of LFNs.
`int` `remove(java.lang.String lfn)` Removes all mappings for an LFN from the replica catalog.
`int` `removeByAttribute(java.lang.String handle)` Removes all entries associated with a particular resource handle.
`int` `removeByAttribute(java.lang.String name, java.lang.Object value)` Removes all entries from the replica catalog where the PFN attribute is found, and matches exactly the object value.
`void` `setBufferSize(int bufferSize)` Set buffer size i.e.
`private void` `write(java.lang.String content)`
`protected java.lang.String` `writeReplicaCatalogEntry(java.lang.String lfn, ReplicaCatalogEntry rce)` The method generate a String representation of the replica catalog entry.

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Field Detail

#### m_out

    protected java.io.BufferedWriter m_out

A buffered writer, to buffer all file writes being performed.

#### m_buffer_size

    private int m_buffer_size

No. of entries to buffer.

#### m_avg_line_size

    private static final int m_avg_line_size

No. of entries to buffer.

See Also:
  Constant Field Values

#### m_line_count

    private int m_line_count

No. of un-flushed entries.

### Constructor Detail

#### FlushedCache

    public FlushedCache()

### Method Detail

#### setBufferSize

    public void setBufferSize(int bufferSize)

Set buffer size i.e. The no of lines that the implementation will buffer.

#### connect

    public boolean connect(java.lang.String filename)

Opens the file for writing.

Parameters:
  `filename` - is the name of the file to write too.
Returns:
  true, if the file can be opened for writing

#### connect

    public boolean connect(java.util.Properties props)

Establishes a connection to the database from the properties. You will need to specify a "file" property to point to the location of the on-disk instance.

Specified by:
  `connect` in interface `Catalog`
Parameters:
  `props` - is the property table with sufficient settings to establish a link with the database.
Returns:
  true if connected, false if failed to connect.

#### quote

    public java.lang.String quote(Escape e,
                                  java.lang.String s)

Quotes a string only if necessary. This methods first determines, if a strings requires quoting, because it contains whitespace, an equality sign, quotes, or a backslash. If not, the string is not quoted. If the input contains forbidden characters, it is placed into quotes and quote and backslash are backslash escaped.

However, if the property "quote" had a `true` value when connecting to the database, output will always be quoted.

Parameters:
  `e` - is the Escape instance used to escape strings.
  `s` - is the string that may require quoting
Returns:
  either the original string, or a newly allocated instance to an escaped string.

#### writeReplicaCatalogEntry

    protected java.lang.String writeReplicaCatalogEntry(java.lang.String lfn,
                                                        ReplicaCatalogEntry rce)

The method generate a String representation of the replica catalog entry. The String representation follows the format expected by SimpleFile replica catalog implementation.

Parameters:
  `lfn` - The logical file name
  `rce` - The replica catalog entry consisting of the physical file name name and key, value pairs.
Returns:
  A String representation of the replica-catalog entry.

#### flush

    private void flush()

Check if the buffer is full i.e. No. of unwritten lines is same as m_buffer_size

See Also:
  `setBufferSize`

#### close

    public void close()

This operation will dump the in-memory representation back onto disk. The store operation is strict in what it produces. The LFN and PFN records are only quoted, if they require quotes, because they contain special characters. The attributes are always quoted and thus quote-escaped.

Specified by:
  `close` in interface `Catalog`

#### isClosed

    public boolean isClosed()

Predicate to check, if the connection with the catalog's implementation is still active. This helps determining, if it makes sense to call `close()`.

Specified by:
  `isClosed` in interface `Catalog`
Returns:
  true, if the implementation is disassociated, false otherwise.
See Also:
  `close()`

#### lookup

    public java.lang.String lookup(java.lang.String lfn,
                                   java.lang.String handle)

Description copied from interface: `ReplicaCatalog`
Retrieves the entry for a given filename and resource handle from the replica catalog.

Specified by:
  `lookup` in interface `ReplicaCatalog`
Parameters:
  `lfn` - is the logical filename to obtain information for.
  `handle` - is the resource handle to obtain entries for.
Returns:
  the (first) matching physical filename, or `null` if no match was found.

#### lookup

    public java.util.Collection lookup(java.lang.String lfn)

Description copied from interface: `ReplicaCatalog`
Retrieves all entries for a given LFN from the replica catalog. Each entry in the result set is a tuple of a PFN and all its attributes.

Specified by:
  `lookup` in interface `ReplicaCatalog`
Parameters:
  `lfn` - is the logical filename to obtain information for.
Returns:
  a collection of replica catalog entries
See Also:
  `ReplicaCatalogEntry`

#### lookupNoAttributes

    public java.util.Set lookupNoAttributes(java.lang.String lfn)

Description copied from interface: `ReplicaCatalog`
Retrieves all entries for a given LFN from the replica catalog. Each entry in the result set is just a PFN string. Duplicates are reduced through the set paradigm.

Specified by:
  `lookupNoAttributes` in interface `ReplicaCatalog`
Parameters:
  `lfn` - is the logical filename to obtain information for.
Returns:
  a set of PFN strings

#### lookup

    public java.util.Map lookup(java.util.Set lfns)

Description copied from interface: `ReplicaCatalog`
Retrieves multiple entries for a given logical filename, up to the complete catalog. Retrieving full catalogs should be harmful, but may be helpful in an online display or portal.

Specified by:
  `lookup` in interface `ReplicaCatalog`
Parameters:
  `lfns` - is a set of logical filename strings to look up.
Returns:
  a map indexed by the LFN. Each value is a collection of replica catalog entries for the LFN.
See Also:
  `ReplicaCatalogEntry`

#### lookupNoAttributes

    public java.util.Map lookupNoAttributes(java.util.Set lfns)

Description copied from interface: `ReplicaCatalog`
Retrieves multiple entries for a given logical filename, up to the complete catalog. Retrieving full catalogs should be harmful, but may be helpful in an online display or portal.

Specified by:
  `lookupNoAttributes` in interface `ReplicaCatalog`
Parameters:
  `lfns` - is a set of logical filename strings to look up.
Returns:
  a map indexed by the LFN. Each value is a set of PFN strings.

#### lookup

    public java.util.Map lookup(java.util.Set lfns,
                                java.lang.String handle)

Description copied from interface: `ReplicaCatalog`
Retrieves multiple entries for a given logical filename, up to the complete catalog. Retrieving full catalogs should be harmful, but may be helpful in online display or portal.

Specified by:
  `lookup` in interface `ReplicaCatalog`
Parameters:
  `lfns` - is a set of logical filename strings to look up.
  `handle` - is the resource handle, restricting the LFNs.
Returns:
  a map indexed by the LFN. Each value is a collection of replica catalog entries (all attributes).
See Also:
  `ReplicaCatalogEntry`

#### lookupNoAttributes

    public java.util.Map lookupNoAttributes(java.util.Set lfns,
                                            java.lang.String handle)

Description copied from interface: `ReplicaCatalog`
Retrieves multiple entries for a given logical filename, up to the complete catalog. Retrieving full catalogs should be harmful, but may be helpful in online display or portal.

Specified by:
  `lookupNoAttributes` in interface `ReplicaCatalog`
Parameters:
  `lfns` - is a set of logical filename strings to look up.
  `handle` - is the resource handle, restricting the LFNs.
Returns:
  a map indexed by the LFN. Each value is a set of physical filenames.

#### lookup

    public java.util.Map lookup(java.util.Map constraints)

Description copied from interface: `ReplicaCatalog`
Retrieves multiple entries for a given logical filename, up to the complete catalog. Retrieving full catalogs should be harmful, but may be helpful in online display or portal.

Specified by:
  `lookup` in interface `ReplicaCatalog`
Parameters:
  `constraints` - is mapping of keys 'lfn', 'pfn', or any attribute name, e.g. the resource handle 'site', to a string that has some meaning to the implementing system. This can be a SQL wildcard for queries, or a regular expression for Java-based memory collections. Unknown keys are ignored. Using an empty map requests the complete catalog.
Returns:
  a map indexed by the LFN. Each value is a collection of replica catalog entries.
See Also:
  `ReplicaCatalogEntry`

#### list

    public java.util.Set list()

Description copied from interface: `ReplicaCatalog`
Lists all logical filenames in the catalog.

Specified by:
  `list` in interface `ReplicaCatalog`
Returns:
  A set of all logical filenames known to the catalog.

#### list

    public java.util.Set list(java.lang.String constraint)

Description copied from interface: `ReplicaCatalog`
Lists a subset of all logical filenames in the catalog.

Specified by:
  `list` in interface `ReplicaCatalog`
Parameters:
  `constraint` - is a constraint for the logical filename only. It is a string that has some meaning to the implementing system. This can be a SQL wildcard for queries, or a regular expression for Java-based memory collections.
Returns:
  A set of logical filenames that match. The set may be empty

#### insert

    public int insert(java.lang.String lfn,
                      ReplicaCatalogEntry tuple)

Inserts a new mapping into the replica catalog. Any existing mapping of the same LFN, PFN, and HANDLE will be replaced, including all of its attributes.

Specified by:
  `insert` in interface `ReplicaCatalog`
Parameters:
  `lfn` - is the logical filename under which to book the entry.
  `tuple` - is the physical filename and associated PFN attributes.
Returns:
  number of insertions, should always be 1. On failure, throw an exception, don't use zero.

#### insert

    public int insert(java.lang.String lfn,
                      java.lang.String pfn,
                      java.lang.String handle)

Inserts a new mapping into the replica catalog. This is a convenience function exposing the resource handle. Internally, the `ReplicaCatalogEntry` element will be constructed, and passed to the appropriate insert function.

Specified by:
  `insert` in interface `ReplicaCatalog`
Parameters:
  `lfn` - is the logical filename under which to book the entry.
  `pfn` - is the physical filename associated with it.
  `handle` - is a resource handle where the PFN resides.
Returns:
  number of insertions, should always be 1. On failure, throw an exception, don't use zero.
See Also:
  `insert(String, ReplicaCatalogEntry )`, `ReplicaCatalogEntry`

#### insert

    public int insert(java.util.Map x)

Inserts multiple mappings into the replica catalog. The input is a map indexed by the LFN. The value for each LFN key is a collection of replica catalog entries. Note that this operation will not replace existing entries.

Specified by:
  `insert` in interface `ReplicaCatalog`
Parameters:
  `x` - is a map from logical filename string to list of replica catalog entries.
Returns:
  the number of insertions.
See Also:
  `org.griphyn.common.catalog.ReplicaCatalogEntry`

#### write

    private void write(java.lang.String content)

#### delete

    public int delete(java.lang.String lfn,
                      java.lang.String pfn)

Description copied from interface: `ReplicaCatalog`
Deletes a specific mapping from the replica catalog. We don't care about the resource handle. More than one entry could theoretically be removed. Upon removal of an entry, all attributes associated with the PFN also evaporate (cascading deletion).

Specified by:
  `delete` in interface `ReplicaCatalog`
Parameters:
  `lfn` - is the logical filename in the tuple.
  `pfn` - is the physical filename in the tuple.
Returns:
  the number of removed entries.

#### delete

    public int delete(java.util.Map x,
                      boolean matchAttributes)

Description copied from interface: `ReplicaCatalog`
Deletes multiple mappings into the replica catalog. The input is a map indexed by the LFN. The value for each LFN key is a collection of replica catalog entries. On setting matchAttributes to false, all entries having matching lfn pfn mapping to an entry in the Map are deleted. However, upon removal of an entry, all attributes associated with the pfn also evaporate (cascaded deletion).

Specified by:
  `delete` in interface `ReplicaCatalog`
Parameters:
  `x` - is a map from logical filename string to list of replica catalog entries.
  `matchAttributes` - whether mapping should be deleted only if all attributes match.
Returns:
  the number of deletions.
See Also:
  `ReplicaCatalogEntry`

#### delete

    public int delete(java.lang.String lfn,
                      ReplicaCatalogEntry tuple)

Description copied from interface: `ReplicaCatalog`
Deletes a very specific mapping from the replica catalog. The LFN must be matches, the PFN, and all PFN attributes specified in the replica catalog entry. More than one entry could theoretically be removed. Upon removal of an entry, all attributes associated with the PFN also evaporate (cascading deletion).

Specified by:
  `delete` in interface `ReplicaCatalog`
Parameters:
  `lfn` - is the logical filename in the tuple.
  `tuple` - is a description of the PFN and its attributes.
Returns:
  the number of removed entries, either 0 or 1.

#### delete

    public int delete(java.lang.String lfn,
                      java.lang.String name,
                      java.lang.Object value)

Description copied from interface: `ReplicaCatalog`
Deletes all PFN entries for a given LFN from the replica catalog where the PFN attribute is found, and matches exactly the object value. This method may be useful to remove all replica entries that have a certain MD5 sum associated with them. It may also be harmful overkill.

Specified by:
  `delete` in interface `ReplicaCatalog`
Parameters:
  `lfn` - is the logical filename to look for.
  `name` - is the PFN attribute name to look for.
  `value` - is an exact match of the attribute value to match.
Returns:
  the number of removed entries.

#### deleteByResource

    public int deleteByResource(java.lang.String lfn,
                                java.lang.String handle)

Description copied from interface: `ReplicaCatalog`
Deletes all PFN entries for a given LFN from the replica catalog where the resource handle is found. Karan requested this convenience method, which can be coded like

      delete( lfn, RESOURCE_HANDLE, handle )

Specified by:
  `deleteByResource` in interface `ReplicaCatalog`
Parameters:
  `lfn` - is the logical filename to look for.
  `handle` - is the resource handle
Returns:
  the number of entries removed.

#### remove

    public int remove(java.lang.String lfn)

Description copied from interface: `ReplicaCatalog`
Removes all mappings for an LFN from the replica catalog.

Specified by:
  `remove` in interface `ReplicaCatalog`
Parameters:
  `lfn` - is the logical filename to remove all mappings for.
Returns:
  the number of removed entries.

#### remove

    public int remove(java.util.Set lfns)

Description copied from interface: `ReplicaCatalog`
Removes all mappings for a set of LFNs.

Specified by:
  `remove` in interface `ReplicaCatalog`
Parameters:
  `lfns` - is a set of logical filename to remove all mappings for.
Returns:
  the number of removed entries.

#### removeByAttribute

    public int removeByAttribute(java.lang.String name,
                                 java.lang.Object value)

Description copied from interface: `ReplicaCatalog`
Removes all entries from the replica catalog where the PFN attribute is found, and matches exactly the object value.

Specified by:
  `removeByAttribute` in interface `ReplicaCatalog`
Parameters:
  `name` - is the PFN attribute name to look for.
  `value` - is an exact match of the attribute value to match.
Returns:
  the number of removed entries.

#### removeByAttribute

    public int removeByAttribute(java.lang.String handle)

Description copied from interface: `ReplicaCatalog`
Removes all entries associated with a particular resource handle. This is useful, if a site goes offline. It is a convenience method, which calls the generic `removeByAttribute` method.

Specified by:
  `removeByAttribute` in interface `ReplicaCatalog`
Parameters:
  `handle` - is the site handle to remove all entries for.
Returns:
  the number of removed entries.
See Also:
  `ReplicaCatalog.removeByAttribute( String, Object )`

#### clear

    public int clear()

Description copied from interface: `ReplicaCatalog`
Removes everything. Use with caution!

Specified by:
  `clear` in interface `ReplicaCatalog`
Returns:
  the number of removed entries.