edu.isi.pegasus.planner.cluster.aggregator

## Class Abstract

- java.lang.Object
- - edu.isi.pegasus.planner.cluster.aggregator.Abstract

All Implemented Interfaces:
  JobAggregator

Direct Known Subclasses:
  MPIExec, SeqExec

    public abstract class Abstract
    extends java.lang.Object
    implements JobAggregator

An abstract implementation of the JobAggregator interface, which the other implementations can choose to extend.

Version:
  $Revision$
Author:
  Karan Vahi vahi@isi.edu

### Field Summary

Fields
`static java.lang.String` `CLUSTERED_JOB_PREFIX` The prefix that is assigned to the jobname of the collapsed jobs to get the jobname for the fat job.
`static java.lang.String` `DERIVATION_NAMESPACE` The derivation namespace for the cluster jobs.
`static java.lang.String` `DERIVATION_VERSION` The version number for the derivations for cluster jobs.
`protected PegasusBag` `mBag` Bag of initialization objects.
`protected ADag` `mClusteredADag` The handle to the ADag object that contains the workflow being clustered.
`protected java.lang.String` `mDirectory` The directory, where the stdin file of the fat jobs are created.
`protected GridStartFactory` `mGridStartFactory` The handle to the GridStart Factory.
`protected LogManager` `mLogger` The handle to the LogManager that logs all the messages.
`static java.lang.String` `MONITORD_COMMENT_MARKER` The marker to designate a line in the input file reserved for monitord purposes.
`protected PegasusProperties` `mProps` The object holding all the properties pertaining to Pegasus.
`protected SiteStore` `mSiteStore` Handle to the site catalog store
`protected TransformationCatalog` `mTCHandle` The handle to the transformation catalog.
`static java.lang.String` `TRANSFORMATION_NAMESPACE` The transformation namespace for the cluster jobs.
`static java.lang.String` `TRANSFORMATION_VERSION` The version number for the derivations for cluster jobs

### Fields inherited from interface edu.isi.pegasus.planner.cluster.JobAggregator

`VERSION`

### Constructor Summary

Constructors
Constructor and Description
`Abstract()` The default constructor.

### Method Summary

All Methods Static Methods Instance Methods Abstract Methods Concrete Methods
`abstract java.lang.String` `aggregatedJobArguments(AggregatedJob job)` Returns the arguments with which the `AggregatedJob` needs to be invoked with.
`AggregatedJob` `constructAbstractAggregatedJob(java.util.List jobs, java.lang.String name, java.lang.String id)` Constructs a new aggregated job that contains all the jobs passed to it.
`AggregatedJob` `constructAbstractAggregatedJob(java.util.List jobs, java.lang.String name, java.lang.String id, java.lang.String mergeLFN)` Constructs an abstract aggregated job that has a handle to the appropriate JobAggregator that will be used to aggregate the jobs.
`private TransformationCatalogEntry` `defaultTCEntry(java.lang.String name, java.lang.String executableBasename, java.lang.String site)` Returns a default TC entry to be used in case entry is not found in the transformation catalog.
`protected boolean` `entryNotInTC(java.lang.String namespace, java.lang.String name, java.lang.String version, java.lang.String executableBasename, java.lang.String site)` Determines whether there is NOT an entry in the transformation catalog for a particular transformation on a particular site.
`protected java.lang.String` `getCommentString(Job job, int taskid)` Generates the comment string for the job .
`static java.lang.String` `getCompleteTranformationName(java.lang.String name)` A convenience method to return the complete transformation name being used to construct jobs in this class.
`protected TransformationCatalogEntry` `getTCEntry(Job job)` Helper method to get an entry from the transformation catalog for an installed executable.
`void` `initialize(ADag dag, PegasusBag bag)` Initializes the JobAggregator impelementation
`void` `makeAbstractAggregatedJobConcrete(AggregatedJob job)` Enables the abstract clustered job for execution and converts it to it's executable form
`protected void` `setDirectory(java.lang.String directory)` Sets the directory where the stdin files are to be generated.
`protected java.io.File` `writeOutInputFileForJobAggregator(AggregatedJob job)` Writes out the input file for the aggregated job

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Methods inherited from interface edu.isi.pegasus.planner.cluster.JobAggregator

`abortOnFristJobFailure, entryNotInTC, getClusterExecutableBasename, getClusterExecutableLFN, setAbortOnFirstJobFailure, topologicalOrderingRequired`

### Field Detail

#### CLUSTERED_JOB_PREFIX

    public static final java.lang.String CLUSTERED_JOB_PREFIX

The prefix that is assigned to the jobname of the collapsed jobs to get the jobname for the fat job.

See Also:
  Constant Field Values

#### TRANSFORMATION_NAMESPACE

    public static final java.lang.String TRANSFORMATION_NAMESPACE

The transformation namespace for the cluster jobs.

See Also:
  Constant Field Values

#### TRANSFORMATION_VERSION

    public static final java.lang.String TRANSFORMATION_VERSION

The version number for the derivations for cluster jobs

#### DERIVATION_NAMESPACE

    public static final java.lang.String DERIVATION_NAMESPACE

The derivation namespace for the cluster jobs.

See Also:
  Constant Field Values

#### DERIVATION_VERSION

    public static final java.lang.String DERIVATION_VERSION

The version number for the derivations for cluster jobs.

See Also:
  Constant Field Values

#### MONITORD_COMMENT_MARKER

    public static final java.lang.String MONITORD_COMMENT_MARKER

The marker to designate a line in the input file reserved for monitord purposes.

See Also:
  Constant Field Values

#### mDirectory

    protected java.lang.String mDirectory

The directory, where the stdin file of the fat jobs are created. It should be the submit file directory that the user mentions at runtime.

#### mProps

    protected PegasusProperties mProps

The object holding all the properties pertaining to Pegasus.

#### mLogger

    protected LogManager mLogger

The handle to the LogManager that logs all the messages.

#### mTCHandle

    protected TransformationCatalog mTCHandle

The handle to the transformation catalog.

#### mSiteStore

    protected SiteStore mSiteStore

Handle to the site catalog store

#### mClusteredADag

    protected ADag mClusteredADag

The handle to the ADag object that contains the workflow being clustered.

#### mGridStartFactory

    protected GridStartFactory mGridStartFactory

The handle to the GridStart Factory.

#### mBag

    protected PegasusBag mBag

Bag of initialization objects.

### Constructor Detail

#### Abstract

    public Abstract()

The default constructor.

### Method Detail

#### getCompleteTranformationName

    public static java.lang.String getCompleteTranformationName(java.lang.String name)

A convenience method to return the complete transformation name being used to construct jobs in this class.

Parameters:
  `name` - the name of the transformation
Returns:
  the complete transformation name

#### initialize

    public void initialize(ADag dag,
                           PegasusBag bag)

Initializes the JobAggregator impelementation

Specified by:
  `initialize` in interface `JobAggregator`
Parameters:
  `dag` - the workflow that is being clustered.
  `bag` - the bag of objects that is useful for initialization.

#### aggregatedJobArguments

    public abstract java.lang.String aggregatedJobArguments(AggregatedJob job)

Returns the arguments with which the `AggregatedJob` needs to be invoked with.

Parameters:
  `job` - the `AggregatedJob` for which the arguments have to be constructed.
Returns:
  argument string

#### constructAbstractAggregatedJob

    public AggregatedJob constructAbstractAggregatedJob(java.util.List jobs,
                                                        java.lang.String name,
                                                        java.lang.String id)

Constructs a new aggregated job that contains all the jobs passed to it. The new aggregated job, appears as a single job in the workflow and replaces the jobs it contains in the workflow.

Specified by:
  `constructAbstractAggregatedJob` in interface `JobAggregator`
Parameters:
  `jobs` - the list of `Job` objects that need to be collapsed. All the jobs being collapsed should be scheduled at the same pool, to maintain correct semantics.
  `name` - the logical name of the jobs in the list passed to this function.
  `id` - the id that is given to the new job.
Returns:
  the `Job` object corresponding to the aggregated job containing the jobs passed as List in the input, null if the list of jobs is empty

#### constructAbstractAggregatedJob

    public AggregatedJob constructAbstractAggregatedJob(java.util.List jobs,
                                                        java.lang.String name,
                                                        java.lang.String id,
                                                        java.lang.String mergeLFN)

Constructs an abstract aggregated job that has a handle to the appropriate JobAggregator that will be used to aggregate the jobs.

Parameters:
  `jobs` - the list of `SubInfo` objects that need to be collapsed. All the jobs being collapsed should be scheduled at the same pool, to maintain correct semantics.
  `name` - the logical name of the jobs in the list passed to this function.
  `id` - the id that is given to the new job.
  `mergeLFN` - the logical name for the aggregated job that has to be constructed.
Returns:
  the `SubInfo` object corresponding to the aggregated job containing the jobs passed as List in the input, null if the list of jobs is empty

#### makeAbstractAggregatedJobConcrete

    public void makeAbstractAggregatedJobConcrete(AggregatedJob job)

Enables the abstract clustered job for execution and converts it to it's executable form

Specified by:
  `makeAbstractAggregatedJobConcrete` in interface `JobAggregator`
Parameters:
  `job` - the abstract clustered job

#### getCommentString

    protected java.lang.String getCommentString(Job job,
                                                int taskid)

Generates the comment string for the job . It generates a comment of the format # task_id transformation derivation.

Parameters:
  `job` - the job for which
  `taskid` - the task id to put in.
Returns:
  the comment invocation

#### getTCEntry

    protected TransformationCatalogEntry getTCEntry(Job job)

Helper method to get an entry from the transformation catalog for an installed executable. It does the traversal from the list of entries to return a single TransformationCatalogEntry object, and dies with an appropriate error message if the object is not found. The pool and the name are retrieved from job object.

Parameters:
  `job` - the job whose corresponding TransformationCatalogEntry you want.
Returns:
  the TransformationCatalogEntry corresponding to the entry in the TC.

#### defaultTCEntry

    private TransformationCatalogEntry defaultTCEntry(java.lang.String name,
                                                      java.lang.String executableBasename,
                                                      java.lang.String site)

Returns a default TC entry to be used in case entry is not found in the transformation catalog.

Parameters:
  `name` - the logical name for the clustering transformation.
  `executableBasename` - the basename for the executable in the bin directory of a Pegasus installation
  `site` - the site for which the default entry is required.
Returns:
  the default entry.

#### entryNotInTC

    protected boolean entryNotInTC(java.lang.String namespace,
                                   java.lang.String name,
                                   java.lang.String version,
                                   java.lang.String executableBasename,
                                   java.lang.String site)

Determines whether there is NOT an entry in the transformation catalog for a particular transformation on a particular site.

Parameters:
  `namespace` - the logical namespace of the transformation.
  `name` - the logical name of the transformation.
  `version` - the version of the transformation.
  `executableBasename` - basename of the executable that does the clustering.
  `site` - the site at which existence check is required.
Returns:
  boolean true if an entry does not exists, false otherwise.

#### setDirectory

    protected void setDirectory(java.lang.String directory)

Sets the directory where the stdin files are to be generated.

Parameters:
  `directory` - the path to the directory to which it needs to be set.

#### writeOutInputFileForJobAggregator

    protected java.io.File writeOutInputFileForJobAggregator(AggregatedJob job)

Writes out the input file for the aggregated job

Parameters:
  `job` - the aggregated job
Returns:
  path to the input file