edu.isi.pegasus.planner.catalog.replica.impl

## Class JDBCRC

- java.lang.Object
- - edu.isi.pegasus.planner.catalog.replica.impl.JDBCRC

All Implemented Interfaces:
  Catalog, ReplicaCatalog

    public class JDBCRC
    extends java.lang.Object
    implements ReplicaCatalog

This class implements a replica catalog on top of a simple table in a JDBC database. This enables a variety of replica catalog implementations in a transactionally safe, concurrent environment. The table must be defined using the statements appropriate for your database - they are part of the setup in $PEGASUS_HOME/sql/. If you chose to use an unsupported database, please check, if your database either supports sequence number, or if it supports auto increment columns. If your database supports sequences (e.g. PostGreSQL), you can use a setup similar to the following (for Oracle, the autoinc can be implemented via a trigger).

     create sequence rc_lfn_id;

     create table rc_lfn (
       id      bigint default nextval('rc_lfn_id'::text),
       lfn     varchar(255) not null,
       pfn     varchar(255) not null,
       site    varchar(245),

       constraint pk_rc_lfn primary key(id),
       constraint sk_rc_lfn unique(lfn,pfn,site)
     );

     create index idx_rc_lfn on rc_lfn(lfn);

     create table rc_attr (
       id      bigint,
       name    varchar(64) not null,
       value   varchar(255) not null,

       constraint pk_rc_attr primary key(id,name),
       constraint fk_rc_attr foreign key(id) references rc_lfn(id) on delete cascade
     );

     create index idx_rc_attr on rc_attr(name);

In case of databases that do not support sequences (e.g. MySQL), do not specify the `create sequence`, and use an auto-increment column for the primary key instead, e.g.:

     create table rc_lfn (
       id      bigint default null auto_increment,
       lfn     varchar(255) not null,
       pfn     varchar(255) not null,
       site    varchar(245),

       constraint pk_rc_lfn primary key(id),
       constraint sk_rc_lfn unique(lfn,pfn,site)
     );

     create index idx_rc_lfn on rc_lfn(lfn);

     create table rc_attr (
       id      bigint,
       name    varchar(64) not null,
       value   varchar(255) not null,

       constraint pk_rc_attr primary key(id,name),
       constraint fk_rc_attr foreign key id references rc_lfn(id) on delete cascade
     );

     create index idx_rc_attr on rc_attr(name);

The site attribute should be specified whenever possible. For the shell planner, it will always be of value "local".

Version:
  $Revision$
Author:
  Jens-S. Vöckler, Yong Zhao

### Field Summary

Fields
`private static java.lang.String` `c_error` This message is sent whenever one of the member function is executed which relies on an established database context.
`private boolean` `m_autoinc` Remembers if obtaining generated keys will work or not.
`protected java.sql.Connection` `mConnection` Maintains the connection to the database over the lifetime of this instance.
`private static java.lang.String[]` `mCStatements` The statement to prepare to slurp attributes.
`protected LogManager` `mLogger` The handle to the logging object.
`protected java.sql.PreparedStatement[]` `mStatements` Maintains an essential set of prepared statement, ready to use.

### Fields inherited from interface edu.isi.pegasus.planner.catalog.ReplicaCatalog

`BATCH_KEY, c_prefix, DB_PREFIX, PROXY_KEY`

### Fields inherited from interface edu.isi.pegasus.planner.catalog.Catalog

`DB_ALL_PREFIX`

### Constructor Summary

Constructors
Constructor and Description
`JDBCRC()` Default empty constructor creates an object that is not yet connected to any database.
`JDBCRC(java.lang.String jdbc, java.lang.String url, java.lang.String username, java.lang.String password)` Convenience c'tor: Establishes the connection to the replica catalog database.

### Method Summary

All Methods Instance Methods Concrete Methods
`private java.lang.String` `addItem(java.lang.Object value, java.lang.String obj, boolean where)` Helper function to assemble various pieces.
`private java.util.Map` `attributes(java.lang.String id, java.lang.String handle)` Slurps all attributes from related to a mapping into a map.
`int` `clear()` Removes everything.
`void` `close()` Explicitely free resources before the garbage collection hits.
`boolean` `connect(java.util.Properties props)` Establishes a connection to the database from the properties.
`void` `connect(java.lang.String url, java.lang.String username, java.lang.String password)` Connects to the database.
`int` `delete(java.util.Map x, boolean matchAttributes)` Deletes multiple mappings into the replica catalog.
`int` `delete(java.lang.String lfn, ReplicaCatalogEntry tuple)` Deletes a very specific mapping from the replica catalog.
`int` `delete(java.lang.String lfn, java.lang.String pfn)` Deletes a specific mapping from the replica catalog.
`int` `delete(java.lang.String lfn, java.lang.String name, java.lang.Object value)` Deletes all PFN entries for a given LFN from the replica catalog where the PFN attribute is found, and matches exactly the object value.
`int` `deleteByResource(java.lang.String lfn, java.lang.String handle)` Deletes all PFN entries for a given LFN from the replica catalog where the resource handle is found.
`protected java.sql.PreparedStatement` `getStatement(int i)` Singleton manager for prepared statements.
`int` `insert(java.util.Map x)` Inserts multiple mappings into the replica catalog.
`int` `insert(java.lang.String lfn, ReplicaCatalogEntry tuple)` Inserts a new mapping into the replica catalog.
`int` `insert(java.lang.String lfn, java.lang.String pfn, java.lang.String handle)` Inserts a new mapping into the replica catalog.
`boolean` `isClosed()` Predicate to check, if the connection with the catalog's implementation is still active.
`java.util.Set` `list()` Lists all logical filenames in the catalog.
`java.util.Set` `list(java.lang.String constraint)` Lists a subset of all logical filenames in the catalog.
`java.util.Map` `lookup(java.util.Map constraints)` Retrieves multiple entries for a given logical filename, up to the complete catalog.
`java.util.Map` `lookup(java.util.Set lfns)` Retrieves multiple entries for a given logical filename, up to the complete catalog.
`java.util.Map` `lookup(java.util.Set lfns, java.lang.String handle)` Retrieves multiple entries for a given logical filename, up to the complete catalog.
`java.util.Collection` `lookup(java.lang.String lfn)` Retrieves all entries for a given LFN from the replica catalog.
`java.lang.String` `lookup(java.lang.String lfn, java.lang.String handle)` Retrieves the entry for a given filename and site handle from the replica catalog.
`java.util.Map` `lookupNoAttributes(java.util.Set lfns)` Retrieves multiple entries for a given logical filename, up to the complete catalog.
`java.util.Map` `lookupNoAttributes(java.util.Set lfns, java.lang.String handle)` Retrieves multiple entries for a given logical filename, up to the complete catalog.
`java.util.Set` `lookupNoAttributes(java.lang.String lfn)` Retrieves all entries for a given LFN from the replica catalog.
`protected java.lang.String` `quote(java.lang.String s)` Quotes a string that may contain special SQL characters.
`int` `remove(java.util.Set lfns)` Removes all mappings for a set of LFNs.
`int` `remove(java.lang.String lfn)` Removes all mappings for an LFN from the replica catalog.
`int` `removeByAttribute(java.lang.String handle)` Removes all entries associated with a particular resource handle.
`int` `removeByAttribute(java.lang.String name, java.lang.Object value)` Removes all entries from the replica catalog where the PFN attribute is found, and matches exactly the object value.

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Field Detail

#### c_error

    private static final java.lang.String c_error

This message is sent whenever one of the member function is executed which relies on an established database context.

See Also:
  Constant Field Values

#### mConnection

    protected java.sql.Connection mConnection

Maintains the connection to the database over the lifetime of this instance.

#### mStatements

    protected java.sql.PreparedStatement[] mStatements

Maintains an essential set of prepared statement, ready to use.

#### mLogger

    protected LogManager mLogger

The handle to the logging object.

#### mCStatements

    private static final java.lang.String[] mCStatements

The statement to prepare to slurp attributes.

#### m_autoinc

    private boolean m_autoinc

Remembers if obtaining generated keys will work or not.

### Constructor Detail

#### JDBCRC

    public JDBCRC(java.lang.String jdbc,
                  java.lang.String url,
                  java.lang.String username,
                  java.lang.String password)
           throws java.lang.LinkageError,
                  java.lang.ExceptionInInitializerError,
                  java.lang.ClassNotFoundException,
                  java.sql.SQLException

Convenience c'tor: Establishes the connection to the replica catalog database. The usual suspects for the class name include:

     org.postgresql.Driver
     com.mysql.jdbc.Driver
     com.microsoft.jdbc.sqlserver.SQLServerDriver
     SQLite.JDBCDriver
     sun.jdbc.odbc.JdbcOdbcDriver

Parameters:
  `jdbc` - is a string containing the full name of the java class that must be dynamically loaded. This is usually an external jar file which contains the Java database driver.
  `url` - is the database driving URL. This string is database specific, and tell the JDBC driver, at which host and port the database listens, permits additional arguments, and selects the database inside the rDBMS to connect to. Please refer to your JDBC driver documentation for the format and permitted values.
  `username` - is the database user account name to connect with.
  `password` - is the database account password to use.
Throws:
  `java.lang.LinkageError` - if linking the dynamically loaded driver fails. This is a run-time error, and does not need to be caught.
  `java.lang.ExceptionInInitializerError` - if the initialization function of the driver's instantiation threw an exception itself. This is a run-time error, and does not need to be caught.
  `java.lang.ClassNotFoundException` - if the class in your jdbc parameter cannot be found in your given CLASSPATH environment. Callers must catch this exception.
  `java.sql.SQLException` - if something goes awry with the database. Callers must catch this exception.

#### JDBCRC

    public JDBCRC()

Default empty constructor creates an object that is not yet connected to any database. You must use support methods to connect before this instance becomes usable.

See Also:
  `connect( String, String, String )`

### Method Detail

#### connect

    public void connect(java.lang.String url,
                        java.lang.String username,
                        java.lang.String password)
                 throws java.sql.SQLException

Connects to the database. This is effectively an accessor to initialize the internal connection instance variable. Warning! You must call `Class.forName( String )` yourself to load the database JDBC driver jar!

Parameters:
  `url` - is the database driving URL. This string is database specific, and tell the JDBC driver, at which host and port the database listens, permits additional arguments, and selects the database inside the rDBMS to connect to. Please refer to your JDBC driver documentation for the format and permitted values.
  `username` - is the database user account name to connect with.
  `password` - is the database account password to use.
Throws:
  `java.sql.SQLException` - if something goes awry with the database. Callers must catch this exception.
See Also:
  `JDBCRC( String, String, String, String )`, `DriverManager.getConnection( String, String, String )`

#### connect

    public boolean connect(java.util.Properties props)

Establishes a connection to the database from the properties. You can specify a `driver` property to contain the class name of the JDBC driver for your database. This property will be removed before attempting to connect. You must speficy a `url` property to describe the connection. It will be removed before attempting to connect.

Specified by:
  `connect` in interface `Catalog`
Parameters:
  `props` - is the property table with sufficient settings to establish a link with the database. The minimum key required key is "url", and possibly "driver". Any other keys depend on the database driver.
Returns:
  true if connected, false if failed to connect.
Throws:
  `java.lang.Error` - subclasses for runtime errors in the class loader.
See Also:
  `DriverManager.getConnection( String, Properties )`

#### close

    public void close()

Explicitely free resources before the garbage collection hits.

Specified by:
  `close` in interface `Catalog`

#### isClosed

    public boolean isClosed()

Predicate to check, if the connection with the catalog's implementation is still active. This helps determining, if it makes sense to call `close()`.

Specified by:
  `isClosed` in interface `Catalog`
Returns:
  true, if the implementation is disassociated, false otherwise.
See Also:
  `close()`

#### quote

    protected java.lang.String quote(java.lang.String s)

Quotes a string that may contain special SQL characters.

Parameters:
  `s` - is the raw string.
Returns:
  the quoted string, which may be just the input string.

#### getStatement

    protected java.sql.PreparedStatement getStatement(int i)
                                               throws java.sql.SQLException

Singleton manager for prepared statements. This instruction checks that a prepared statement is ready to use, and will create an instance of the prepared statement, if it was unused previously.

Parameters:
  `i` - is the index which prepared statement to check.
Returns:
  a handle to the prepared statement.
Throws:
  `java.sql.SQLException`

#### lookup

    public java.lang.String lookup(java.lang.String lfn,
                                   java.lang.String handle)

Retrieves the entry for a given filename and site handle from the replica catalog.

Specified by:
  `lookup` in interface `ReplicaCatalog`
Parameters:
  `lfn` - is the logical filename to obtain information for.
  `handle` - is the resource handle to obtain entries for.
Returns:
  the (first) matching physical filename, or `null` if no match was found.

#### attributes

    private java.util.Map attributes(java.lang.String id,
                                     java.lang.String handle)
                              throws java.sql.SQLException

Slurps all attributes from related to a mapping into a map.

Parameters:
  `id` - is the reference id to slurp from as string. Especially Postgres's indexing mechanism goes from tables scans to btrees, if the numeric key is represented as a string. Strings should be safe for other databases, too.
Returns:
  a Map with the attributes, which may be empty.
Throws:
  `java.sql.SQLException`

#### lookup

    public java.util.Collection lookup(java.lang.String lfn)

Retrieves all entries for a given LFN from the replica catalog. Each entry in the result set is a tuple of a PFN and all its attributes.

Specified by:
  `lookup` in interface `ReplicaCatalog`
Parameters:
  `lfn` - is the logical filename to obtain information for.
Returns:
  a collection of replica catalog entries
See Also:
  `ReplicaCatalogEntry`

#### lookupNoAttributes

    public java.util.Set lookupNoAttributes(java.lang.String lfn)

Retrieves all entries for a given LFN from the replica catalog. Each entry in the result set is just a PFN string. Duplicates are reduced through the set paradigm.

Specified by:
  `lookupNoAttributes` in interface `ReplicaCatalog`
Parameters:
  `lfn` - is the logical filename to obtain information for.
Returns:
  a set of PFN strings

#### lookup

    public java.util.Map lookup(java.util.Set lfns)

Retrieves multiple entries for a given logical filename, up to the complete catalog. Retrieving full catalogs should be harmful, but may be helpful in an online display or portal.

Specified by:
  `lookup` in interface `ReplicaCatalog`
Parameters:
  `lfns` - is a set of logical filename strings to look up.
Returns:
  a map indexed by the LFN. Each value is a collection of replica catalog entries for the LFN.
See Also:
  `org.griphyn.common.catalog.ReplicaCatalogEntry`

#### lookupNoAttributes

    public java.util.Map lookupNoAttributes(java.util.Set lfns)

Retrieves multiple entries for a given logical filename, up to the complete catalog. Retrieving full catalogs should be harmful, but may be helpful in an online display or portal.

Specified by:
  `lookupNoAttributes` in interface `ReplicaCatalog`
Parameters:
  `lfns` - is a set of logical filename strings to look up.
Returns:
  a map indexed by the LFN. Each value is a set of PFN strings.

#### lookup

    public java.util.Map lookup(java.util.Set lfns,
                                java.lang.String handle)

Retrieves multiple entries for a given logical filename, up to the complete catalog. Retrieving full catalogs should be harmful, but may be helpful in online display or portal.

Specified by:
  `lookup` in interface `ReplicaCatalog`
Parameters:
  `lfns` - is a set of logical filename strings to look up.
  `handle` - is the resource handle, restricting the LFNs.
Returns:
  a map indexed by the LFN. Each value is a collection of replica catalog entries (all attributes).
See Also:
  `ReplicaCatalogEntry`

#### lookupNoAttributes

    public java.util.Map lookupNoAttributes(java.util.Set lfns,
                                            java.lang.String handle)

Retrieves multiple entries for a given logical filename, up to the complete catalog. Retrieving full catalogs should be harmful, but may be helpful in online display or portal.

Specified by:
  `lookupNoAttributes` in interface `ReplicaCatalog`
Parameters:
  `lfns` - is a set of logical filename strings to look up.
  `handle` - is the resource handle, restricting the LFNs.
Returns:
  a map indexed by the LFN. Each value is a set of physical filenames.

#### addItem

    private java.lang.String addItem(java.lang.Object value,
                                     java.lang.String obj,
                                     boolean where)

Helper function to assemble various pieces.

Parameters:
  `value` - is the value of the object from the map.
  `obj` - is the name of the table column
  `where` - is the decision, if we had a previous WHERE clause or not.
See Also:
  `lookup( Map )`

#### lookup

    public java.util.Map lookup(java.util.Map constraints)

Retrieves multiple entries for a given logical filename, up to the complete catalog. Retrieving full catalogs should be harmful, but may be helpful in online display or portal.

Specified by:
  `lookup` in interface `ReplicaCatalog`
Parameters:
  `constraints` - is mapping of keys 'lfn', 'pfn', or any attribute name, e.g. the resource handle 'site', to a string that has some meaning to the implementing system. This can be a SQL wildcard for queries, or a regular expression for Java-based memory collections. Unknown keys are ignored. Using an empty map requests the complete catalog.
Returns:
  a map indexed by the LFN. Each value is a collection of replica catalog entries.
See Also:
  `ReplicaCatalogEntry`

#### list

    public java.util.Set list()

Lists all logical filenames in the catalog.

Specified by:
  `list` in interface `ReplicaCatalog`
Returns:
  A set of all logical filenames known to the catalog.

#### list

    public java.util.Set list(java.lang.String constraint)

Lists a subset of all logical filenames in the catalog.

Specified by:
  `list` in interface `ReplicaCatalog`
Parameters:
  `constraint` - is a constraint for the logical filename only. It is a string that has some meaning to the implementing system. This can be a SQL wildcard for queries, or a regular expression for Java-based memory collections.
Returns:
  A set of logical filenames that match. The set may be empty

#### insert

    public int insert(java.lang.String lfn,
                      ReplicaCatalogEntry tuple)

Inserts a new mapping into the replica catalog.

Specified by:
  `insert` in interface `ReplicaCatalog`
Parameters:
  `lfn` - is the logical filename under which to book the entry.
  `tuple` - is the physical filename and associated PFN attributes.
Returns:
  number of insertions, should always be 1. On failure, throw an exception, don't use zero.

#### insert

    public int insert(java.lang.String lfn,
                      java.lang.String pfn,
                      java.lang.String handle)

Inserts a new mapping into the replica catalog. This is a convenience function exposing the resource handle. Internally, the `ReplicaCatalogEntry` element will be contructed, and passed to the appropriate insert function.

Specified by:
  `insert` in interface `ReplicaCatalog`
Parameters:
  `lfn` - is the logical filename under which to book the entry.
  `pfn` - is the physical filename associated with it.
  `handle` - is a resource handle where the PFN resides.
Returns:
  number of insertions, should always be 1. On failure, throw an exception, don't use zero.
See Also:
  `insert( String, ReplicaCatalogEntry )`, `ReplicaCatalogEntry`

#### insert

    public int insert(java.util.Map x)

Inserts multiple mappings into the replica catalog. The input is a map indexed by the LFN. The value for each LFN key is a collection of replica catalog entries.

Specified by:
  `insert` in interface `ReplicaCatalog`
Parameters:
  `x` - is a map from logical filename string to list of replica catalog entries.
Returns:
  the number of insertions.
See Also:
  `org.griphyn.common.catalog.ReplicaCatalogEntry`

#### delete

    public int delete(java.util.Map x,
                      boolean matchAttributes)

Deletes multiple mappings into the replica catalog. The input is a map indexed by the LFN. The value for each LFN key is a collection of replica catalog entries. On setting matchAttributes to false, all entries having matching lfn pfn mapping to an entry in the Map are deleted. However, upon removal of an entry, all attributes associated with the pfn also evaporate (cascaded deletion).

Specified by:
  `delete` in interface `ReplicaCatalog`
Parameters:
  `x` - is a map from logical filename string to list of replica catalog entries.
  `matchAttributes` - whether mapping should be deleted only if all attributes match.
Returns:
  the number of deletions.
See Also:
  `ReplicaCatalogEntry`

#### delete

    public int delete(java.lang.String lfn,
                      java.lang.String pfn)

Deletes a specific mapping from the replica catalog. We don't care about the resource handle. More than one entry could theoretically be removed. Upon removal of an entry, all attributes associated with the PFN also evaporate (cascading deletion).

Specified by:
  `delete` in interface `ReplicaCatalog`
Parameters:
  `lfn` - is the logical filename in the tuple.
  `pfn` - is the physical filename in the tuple.
Returns:
  the number of removed entries.

#### delete

    public int delete(java.lang.String lfn,
                      ReplicaCatalogEntry tuple)

Deletes a very specific mapping from the replica catalog. The LFN must be matches, the PFN, and all PFN attributes specified in the replica catalog entry. More than one entry could theoretically be removed. Upon removal of an entry, all attributes associated with the PFN also evaporate (cascading deletion).

Specified by:
  `delete` in interface `ReplicaCatalog`
Parameters:
  `lfn` - is the logical filename in the tuple.
  `tuple` - is a description of the PFN and its attributes.
Returns:
  the number of removed entries, either 0 or 1.

#### delete

    public int delete(java.lang.String lfn,
                      java.lang.String name,
                      java.lang.Object value)

Deletes all PFN entries for a given LFN from the replica catalog where the PFN attribute is found, and matches exactly the object value. This method may be useful to remove all replica entries that have a certain MD5 sum associated with them. It may also be harmful overkill.

Specified by:
  `delete` in interface `ReplicaCatalog`
Parameters:
  `lfn` - is the logical filename to look for.
  `name` - is the PFN attribute name to look for.
  `value` - is an exact match of the attribute value to match.
Returns:
  the number of removed entries.

#### deleteByResource

    public int deleteByResource(java.lang.String lfn,
                                java.lang.String handle)

Deletes all PFN entries for a given LFN from the replica catalog where the resource handle is found. Karan requested this convenience method, which can be coded like

      delete( lfn, RESOURCE_HANDLE, handle )

Specified by:
  `deleteByResource` in interface `ReplicaCatalog`
Parameters:
  `lfn` - is the logical filename to look for.
  `handle` - is the resource handle
Returns:
  the number of entries removed.

#### remove

    public int remove(java.lang.String lfn)

Removes all mappings for an LFN from the replica catalog.

Specified by:
  `remove` in interface `ReplicaCatalog`
Parameters:
  `lfn` - is the logical filename to remove all mappings for.
Returns:
  the number of removed entries.

#### remove

    public int remove(java.util.Set lfns)

Removes all mappings for a set of LFNs.

Specified by:
  `remove` in interface `ReplicaCatalog`
Parameters:
  `lfns` - is a set of logical filename to remove all mappings for.
Returns:
  the number of removed entries.

#### removeByAttribute

    public int removeByAttribute(java.lang.String name,
                                 java.lang.Object value)

Removes all entries from the replica catalog where the PFN attribute is found, and matches exactly the object value.

Specified by:
  `removeByAttribute` in interface `ReplicaCatalog`
Parameters:
  `name` - is the PFN attribute name to look for.
  `value` - is an exact match of the attribute value to match.
Returns:
  the number of removed entries.

#### removeByAttribute

    public int removeByAttribute(java.lang.String handle)

Removes all entries associated with a particular resource handle. This is useful, if a site goes offline. It is a convenience method, which calls the generic `removeByAttribute` method.

Specified by:
  `removeByAttribute` in interface `ReplicaCatalog`
Parameters:
  `handle` - is the site handle to remove all entries for.
Returns:
  the number of removed entries.
See Also:
  `removeByAttribute( String, Object )`

#### clear

    public int clear()

Removes everything. Use with caution!

Specified by:
  `clear` in interface `ReplicaCatalog`
Returns:
  the number of removed entries.