edu.isi.pegasus.planner.refiner

## Class InterPoolEngine

- java.lang.Object
- - edu.isi.pegasus.planner.refiner.Engine
  - - edu.isi.pegasus.planner.refiner.InterPoolEngine

All Implemented Interfaces:
  Refiner

    public class InterPoolEngine
    extends Engine
    implements Refiner

This engine calls out to the Site Selector selected by the user and maps the jobs in the workflow to the execution pools.

Version:
  $Revision$
Author:
  Karan Vahi, Gaurang Mehta

### Field Summary

Fields
`private ADag` `mDag` ADag object corresponding to the Dag whose jobs we want to schedule.
`protected TransformationStore` `mDAXTransformationStore` Handle to the transformation store that stores the transformation catalog user specifies in the DAX
`private java.util.Set` `mExecPools` Set of the execution pools which the user has specified.
`private SiteSelector` `mSiteSelector` Handle to the site selector.
`private SLS` `mSLS` The handle to the SLS implementor
`private Mapper` `mTCMapper` The handle to the transformation catalog mapper object that caches the queries to the transformation catalog, and indexes them according to lfn's.
`private TransformationSelector` `mTXSelector` The handle to the transformation selector, that ends up selecting what transformations to pick up.
`private boolean` `mWorkerNodeExecution` A boolean indicating whether to have worker node execution or not.
`private XMLProducer` `mXMLStore` The XML Producer object that records the actions.
`static java.lang.String` `REFINER_NAME` The name of the refiner for purposes of error logging

### Fields inherited from class edu.isi.pegasus.planner.refiner.Engine

`mBag, mLogger, mLogMsg, mOutputPool, mPoolFile, mPOptions, mProps, mRLIUrl, mSiteStore, mTCFile, mTCHandle, mTCMode, REGISTRATION_UNIVERSE, TRANSFER_UNIVERSE`

### Fields inherited from interface edu.isi.pegasus.planner.refiner.Refiner

`VERSION`

### Constructor Summary

Constructors
Constructor and Description
`InterPoolEngine(ADag aDag, PegasusBag bag)` Overloaded constructor.
`InterPoolEngine(PegasusBag bag)` Default constructor.

### Method Summary

All Methods Instance Methods Concrete Methods
`java.util.List` `convertToList(java.util.Set s)` Converts a Set to a List.
`java.util.List` `convertToList(java.util.Vector v)` Converts a Vector to a List.
`void` `determineSites()` This is where the callout to the Partitioner should take place, that partitions the workflow into clusters and sends to the site selector only those list of jobs that are ready to be scheduled.
`private java.lang.String` `getJobManager(java.lang.String site, java.lang.String universe)` It returns a jobmanager for the given pool.
`PegasusBag` `getPegasusBag()` Returns the bag of intialization objects.
`java.lang.String` `getStagingSite(Job job)` Returns the staging site to be used for a job.
`ADag` `getWorkflow()` Returns a reference to the workflow that is being refined by the refiner.
`XMLProducer` `getXMLProducer()` Returns a reference to the XMLProducer, that generates the XML fragment capturing the actions of the refiner.
`private void` `handleDependantExecutables(Job job)` Handles the dependant executables that need to be staged.
`private boolean` `incorporateHint(Job job, java.lang.String key)` It incorporates a hint in the namespace to the job.
`private boolean` `incorporateProfiles(Job job)` Incorporates the profiles from the various sources into the job.
`protected void` `logRefinerAction(Job job)` Logs the action taken by the refiner on a job as a XML fragment in the XML Producer.
`void` `scheduleJobs(ADag dag, java.util.List sites)` It schedules a list of jobs on the execution pools by calling out to the site selector specified.
`private TransformationCatalogEntry` `selectTCEntry(java.util.List entries, Job job, java.lang.String selector)` Calls out to the transformation selector to select an entry from a list of valid transformation catalog entries.

### Methods inherited from class edu.isi.pegasus.planner.refiner.Engine

`addVector, appendArrayList, complainForHeadNodeURLPrefix, complainForHeadNodeURLPrefix, loadProperties, printVector, stringInList, stringInPegVector, stringInVector, vectorToString`

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Field Detail

#### REFINER_NAME

    public static final java.lang.String REFINER_NAME

The name of the refiner for purposes of error logging

See Also:
  Constant Field Values

#### mDag

    private ADag mDag

ADag object corresponding to the Dag whose jobs we want to schedule.

#### mExecPools

    private java.util.Set mExecPools

Set of the execution pools which the user has specified.

#### mSiteSelector

    private SiteSelector mSiteSelector

Handle to the site selector.

#### mTXSelector

    private TransformationSelector mTXSelector

The handle to the transformation selector, that ends up selecting what transformations to pick up.

#### mTCMapper

    private Mapper mTCMapper

The handle to the transformation catalog mapper object that caches the queries to the transformation catalog, and indexes them according to lfn's. There is no purge policy in the TC Mapper, so per se it is not a classic cache.

#### mXMLStore

    private XMLProducer mXMLStore

The XML Producer object that records the actions.

#### mSLS

    private SLS mSLS

The handle to the SLS implementor

#### mWorkerNodeExecution

    private boolean mWorkerNodeExecution

A boolean indicating whether to have worker node execution or not.

#### mDAXTransformationStore

    protected TransformationStore mDAXTransformationStore

Handle to the transformation store that stores the transformation catalog user specifies in the DAX

### Constructor Detail

#### InterPoolEngine

    public InterPoolEngine(PegasusBag bag)

Default constructor.

Parameters:
  `bag` - the bag of initialization objects.

#### InterPoolEngine

    public InterPoolEngine(ADag aDag,
                           PegasusBag bag)

Overloaded constructor.

Parameters:
  `aDag` - the `ADag` object corresponding to the Dag for which we want to determine on which pools to run the nodes of the Dag.
  `bag` - the bag of initialization objects

### Method Detail

#### getPegasusBag

    public PegasusBag getPegasusBag()

Returns the bag of intialization objects.

Returns:
  PegasusBag

#### getWorkflow

    public ADag getWorkflow()

Returns a reference to the workflow that is being refined by the refiner.

Specified by:
  `getWorkflow` in interface `Refiner`
Returns:
  ADAG object.

#### getXMLProducer

    public XMLProducer getXMLProducer()

Returns a reference to the XMLProducer, that generates the XML fragment capturing the actions of the refiner. This is used for provenace purposes.

Specified by:
  `getXMLProducer` in interface `Refiner`
Returns:
  XMLProducer

#### determineSites

    public void determineSites()

This is where the callout to the Partitioner should take place, that partitions the workflow into clusters and sends to the site selector only those list of jobs that are ready to be scheduled.

#### scheduleJobs

    public void scheduleJobs(ADag dag,
                             java.util.List sites)

It schedules a list of jobs on the execution pools by calling out to the site selector specified. It is upto to the site selector to determine if the job can be run on the list of sites passed.

Parameters:
  `dag` - the abstract workflow.
  `sites` - the list of execution sites, specified by the user.

#### getStagingSite

    public java.lang.String getStagingSite(Job job)

Returns the staging site to be used for a job. If a staging site is not determined from the options it is set to be the execution site for the job

Parameters:
  `job` - the job for which to determine the staging site
Returns:
  the staging site

#### incorporateProfiles

    private boolean incorporateProfiles(Job job)

Incorporates the profiles from the various sources into the job. The profiles are incorporated in the order pool, transformation catalog, and properties file, with the profiles from the properties file having the highest priority. It is here where the transformation selector is called to select amongst the various transformations returned by the TC Mapper.

Parameters:
  `job` - the job into which the profiles have been incorporated.
Returns:
  true profiles were successfully incorporated. false otherwise

#### handleDependantExecutables

    private void handleDependantExecutables(Job job)

Handles the dependant executables that need to be staged.

Parameters:
  `job` - Job

#### selectTCEntry

    private TransformationCatalogEntry selectTCEntry(java.util.List entries,
                                                     Job job,
                                                     java.lang.String selector)

Calls out to the transformation selector to select an entry from a list of valid transformation catalog entries.

Parameters:
  `entries` - list of `TransformationCatalogEntry` objects.
  `job` - the job.
  `selectors` - the selector to be called
Returns:
  the selected `TransformationCatalogEntry` object null when transformation selector is unable to select any transformation

#### getJobManager

    private java.lang.String getJobManager(java.lang.String site,
                                           java.lang.String universe)

It returns a jobmanager for the given pool.

Parameters:
  `site` - the name of the pool.
  `universe` - the universe for which you need the scheduler on that particular pool.
Returns:
  the jobmanager for that pool and universe. null if not found.

#### incorporateHint

    private boolean incorporateHint(Job job,
                                    java.lang.String key)

It incorporates a hint in the namespace to the job. After the hint is incorporated the key is deleted from the hint namespace for that job.

Parameters:
  `job` - the job that needs the hint to be incorporated.
  `key` - the key in the hint namespace.
Returns:
  true the hint was successfully incorporated. false the hint was not set in job or was not successfully incorporated.

#### convertToList

    public java.util.List convertToList(java.util.Vector v)

Converts a Vector to a List. It only copies by reference.

Parameters:
  `v` - Vector
Returns:
  a ArrayList

#### convertToList

    public java.util.List convertToList(java.util.Set s)

Converts a Set to a List. It only copies by reference.

Parameters:
  `s` - Set
Returns:
  a ArrayList

#### logRefinerAction

    protected void logRefinerAction(Job job)

Logs the action taken by the refiner on a job as a XML fragment in the XML Producer.

Parameters:
  `job` - the `Job` containing the job that was mapped to a site.