edu.isi.pegasus.planner.transfer.implementation

## Class Abstract

- java.lang.Object
- - edu.isi.pegasus.planner.transfer.implementation.Abstract

All Implemented Interfaces:
  Implementation

Direct Known Subclasses:
  AbstractMultipleFTPerXFERJob, AbstractSingleFTPerXFERJob

    public abstract class Abstract
    extends java.lang.Object
    implements Implementation

An abstract implementation that implements some of the common functions in the Implementation Interface that are required by all the implementations.

Version:
  $Revision$
Author:
  Karan Vahi

### Field Summary

Fields
`static java.lang.String` `CHANGE_XBIT_TRANSFORMATION` The logical name of the transformation that creates directories on the remote execution pools.
`protected boolean` `mAddNodesForSettingXBit` A boolean indicating whether chmod jobs should be created that set the xbit in case of executable staging.
`protected boolean` `mChmodDisabledForAllSites` A boolean indicating whether chmod is disabled for all sites.
`protected java.util.Set` `mDisabledChmodSites` The set of sites for which chmod job creation has to be disabled while doing executable staging.
`protected java.lang.String` `mLocalIrodsEnv` The path to the irodsEnv file on the submit host (local pool).
`protected java.lang.String` `mLocalIrodsEnvBasename` The basename of the user irodsEnv file
`protected java.lang.String` `mLocalS3cfg` The path to the s3cfg file on the submit host (local pool).
`protected java.lang.String` `mLocalS3cfgBasename` The basename of the user s3cfg file
`protected java.lang.String` `mLocalUserProxy` The path to the user proxy on the submit host (local pool), that is picked up for use in transfer of proxies.
`protected java.lang.String` `mLocalUserProxyBasename` The basename of the user proxy , that is picked up for use in transfer of proxies.
`protected LogManager` `mLogger` The logging object which is used to log all the messages.
`protected PlannerOptions` `mPOptions` Contains the various options to the Planner as passed by the user at runtime.
`protected PegasusProperties` `mProps` The handle to the properties object holding the properties relevant to Pegasus.
`protected Refiner` `mRefiner` The handle to the refiner that loaded this implementation.
`protected SiteStore` `mSiteStore` The handle to the Pool Info Provider.
`protected TransformationCatalog` `mTCHandle` The handle to the Transformation Catalog.
`static java.lang.String` `NOOP_PREFIX` The prefix for the NoOP jobs that are created.
`static java.lang.String` `SET_XBIT_PREFIX` The prefix for the jobs which are added to set X bit for the staged executables.
`static java.lang.String` `XBIT_DERIVATION_NS` The derivation namespace for the setXBit jobs.
`static java.lang.String` `XBIT_DERIVATION_VERSION` The version number for the derivations for setXBit jobs.
`static java.lang.String` `XBIT_EXECUTABLE_BASENAME` The basename of the pegasus dirmanager executable.
`static java.lang.String` `XBIT_TRANSFORMATION_NS` The transformation namespace for the setXBit jobs.
`static java.lang.String` `XBIT_TRANSFORMATION_VERSION` The version number for the derivations for setXBit jobs.

### Fields inherited from interface edu.isi.pegasus.planner.transfer.Implementation

`TRANSFER_UNIVERSE, VERSION`

### Constructor Summary

Constructors
Constructor and Description
`Abstract(PegasusBag bag)` The overloaded constructor, that is called by the Factory to load the class.

### Method Summary

All Methods Instance Methods Concrete Methods Deprecated Methods
`protected boolean` `addSetXBitJobs(Job computeJob, Job txJob, java.util.Collection execFiles)` Adds the dirmanager to the workflow, that do a chmod on the files being staged.
`boolean` `addSetXBitJobs(Job computeJob, java.lang.String txJobName, java.util.Collection execFiles, int transferClass)` Adds the dirmanager job to the workflow, that do a chmod on the files being staged.
`boolean` `addSetXBitJobs(Job computeJob, java.lang.String txJobName, java.util.Collection execFiles, int transferClass, int xbitIndex)` Adds the dirmanager job to the workflow, that do a chmod on the files being staged.
`void` `applyPriority(TransferJob job)` Applies priorities to the transfer jobs if a priority is specified in the properties file.
`boolean` `checkAndTransferIrodsEnvFile(TransferJob job)` Deprecated.
`boolean` `checkAndTransferProxy(TransferJob job)` Deprecated.
`protected void` `construct(Job job, java.lang.String key, java.lang.String value)` Constructs a condor variable in the condor profile namespace associated with the job.
`Job` `createNoOPJob(java.lang.String name)` It creates a NoOP job that runs on the submit host.
`protected Job` `createSetXBitJob(java.util.Collection<FileTransfer> files, java.lang.String name, java.lang.String site)` Creates a dirmanager job, that does a chmod on the file being staged.
`protected Job` `createSetXBitJob(FileTransfer file, java.lang.String name)` Creates a dirmanager job, that does a chmod on the file being staged.
`Job` `createSetXBitJob(Job computeJob, java.util.Collection<FileTransfer> execFiles, int transferClass, int xbitIndex)` Adds the dirmanager job to the workflow, that do a chmod on the files being staged.
`private TransformationCatalogEntry` `defaultXBitTCEntry(java.lang.String site)` Returns a default TC entry to be used in case entry is not found in the transformation catalog.
`protected java.util.Set` `determineDisabledChmodSites(java.lang.String sites)` Builds up a set of disabled chmod sites
`protected boolean` `disableChmodJobCreation(java.lang.String site)` Returns a boolean indicating whether to disable chmod job creation for a site or not.
`java.lang.String` `getNOOPJobName(java.lang.String name, int counter)` Generates the name of the noop job , that is unique for the given workflow.
`protected java.lang.String` `getPriority(TransferJob job)` Returns the priority for the transfer job as specified in the properties file.
`java.lang.String` `getSetXBitJobName(java.lang.String name, int counter)` Generates the name of the setXBitJob , that is unique for the given workflow.
`void` `setRefiner(Refiner refiner)` Sets the callback to the refiner, that has loaded this implementation.

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Methods inherited from interface edu.isi.pegasus.planner.transfer.Implementation

`createTransferJob, doesPreserveXBit, getDescription, getTransformationCatalogEntry, useThirdPartyTransferAlways`

### Field Detail

#### CHANGE_XBIT_TRANSFORMATION

    public static final java.lang.String CHANGE_XBIT_TRANSFORMATION

The logical name of the transformation that creates directories on the remote execution pools.

See Also:
  Constant Field Values

#### XBIT_EXECUTABLE_BASENAME

    public static final java.lang.String XBIT_EXECUTABLE_BASENAME

The basename of the pegasus dirmanager executable.

See Also:
  Constant Field Values

#### XBIT_TRANSFORMATION_NS

    public static final java.lang.String XBIT_TRANSFORMATION_NS

The transformation namespace for the setXBit jobs.

See Also:
  Constant Field Values

#### XBIT_TRANSFORMATION_VERSION

    public static final java.lang.String XBIT_TRANSFORMATION_VERSION

The version number for the derivations for setXBit jobs.

#### XBIT_DERIVATION_NS

    public static final java.lang.String XBIT_DERIVATION_NS

The derivation namespace for the setXBit jobs.

See Also:
  Constant Field Values

#### XBIT_DERIVATION_VERSION

    public static final java.lang.String XBIT_DERIVATION_VERSION

The version number for the derivations for setXBit jobs.

#### SET_XBIT_PREFIX

    public static final java.lang.String SET_XBIT_PREFIX

The prefix for the jobs which are added to set X bit for the staged executables.

See Also:
  Constant Field Values

#### NOOP_PREFIX

    public static final java.lang.String NOOP_PREFIX

The prefix for the NoOP jobs that are created.

See Also:
  Constant Field Values

#### mLocalUserProxy

    protected java.lang.String mLocalUserProxy

The path to the user proxy on the submit host (local pool), that is picked up for use in transfer of proxies.

#### mLocalUserProxyBasename

    protected java.lang.String mLocalUserProxyBasename

The basename of the user proxy , that is picked up for use in transfer of proxies.

#### mLocalS3cfg

    protected java.lang.String mLocalS3cfg

The path to the s3cfg file on the submit host (local pool).

#### mLocalS3cfgBasename

    protected java.lang.String mLocalS3cfgBasename

The basename of the user s3cfg file

#### mLocalIrodsEnv

    protected java.lang.String mLocalIrodsEnv

The path to the irodsEnv file on the submit host (local pool).

#### mLocalIrodsEnvBasename

    protected java.lang.String mLocalIrodsEnvBasename

The basename of the user irodsEnv file

#### mProps

    protected PegasusProperties mProps

The handle to the properties object holding the properties relevant to Pegasus.

#### mPOptions

    protected PlannerOptions mPOptions

Contains the various options to the Planner as passed by the user at runtime.

#### mSiteStore

    protected SiteStore mSiteStore

The handle to the Pool Info Provider. It is instantiated in this class

#### mTCHandle

    protected TransformationCatalog mTCHandle

The handle to the Transformation Catalog. It must be instantiated in the implementing class

#### mRefiner

    protected Refiner mRefiner

The handle to the refiner that loaded this implementation.

#### mLogger

    protected LogManager mLogger

The logging object which is used to log all the messages.

#### mDisabledChmodSites

    protected java.util.Set mDisabledChmodSites

The set of sites for which chmod job creation has to be disabled while doing executable staging.

#### mChmodDisabledForAllSites

    protected boolean mChmodDisabledForAllSites

A boolean indicating whether chmod is disabled for all sites.

#### mAddNodesForSettingXBit

    protected boolean mAddNodesForSettingXBit

A boolean indicating whether chmod jobs should be created that set the xbit in case of executable staging.

### Constructor Detail

#### Abstract

    public Abstract(PegasusBag bag)

The overloaded constructor, that is called by the Factory to load the class.

Parameters:
  `bag` - the bag of initialization objects.

### Method Detail

#### applyPriority

    public void applyPriority(TransferJob job)

Applies priorities to the transfer jobs if a priority is specified in the properties file.

Specified by:
  `applyPriority` in interface `Implementation`
Parameters:
  `job` - the transfer job .

#### checkAndTransferProxy

    public boolean checkAndTransferProxy(TransferJob job)

Deprecated.
Determines if there is a need to transfer proxy for the transfer job or not. If there is a need to transfer proxy, then the job is modified to create the correct condor commands to transfer the proxy. Proxy is usually transferred if the Pegasus profile TRANSFER_PROXY is set, or the job is being run in the condor vanilla universe. The proxy is transferred from the submit host (i.e site local). The location is determined from the value of the X509_USER_PROXY profile key associated in the env namespace.

Specified by:
  `checkAndTransferProxy` in interface `Implementation`
Parameters:
  `job` - the transfer job .
Returns:
  boolean true job was modified to transfer the proxy, else false when job is not modified.

#### checkAndTransferIrodsEnvFile

    public boolean checkAndTransferIrodsEnvFile(TransferJob job)

Deprecated.
Determines if there is a need to transfer the irodsEnvFile for the transfer job or not. If there is a need to transfert the file, then the job is modified to create the correct condor commands to transfer the file. The file is transferred from the submit host (i.e site local).

Parameters:
  `job` - the transfer job .
Returns:
  boolean true job was modified to transfer the irodsEnvFile, else false when job is not modified.

#### setRefiner

    public void setRefiner(Refiner refiner)

Sets the callback to the refiner, that has loaded this implementation.

Specified by:
  `setRefiner` in interface `Implementation`
Parameters:
  `refiner` - the transfer refiner that loaded the implementation.

#### addSetXBitJobs

    protected boolean addSetXBitJobs(Job computeJob,
                                     Job txJob,
                                     java.util.Collection execFiles)

Adds the dirmanager to the workflow, that do a chmod on the files being staged.

Parameters:
  `computeJob` - the computeJob for which the files are being staged.
  `txJob` - the transfer job that is staging the files.
  `execFiles` - the executable files that are being staged.
Returns:
  boolean indicating whether any XBitJobs were succesfully added or not.

#### addSetXBitJobs

    public boolean addSetXBitJobs(Job computeJob,
                                  java.lang.String txJobName,
                                  java.util.Collection execFiles,
                                  int transferClass)

Adds the dirmanager job to the workflow, that do a chmod on the files being staged.

Parameters:
  `computeJob` - the computeJob for which the files are being staged.
  `txJobName` - the name of the transfer job that is staging the files.
  `execFiles` - the executable files that are being staged.
  `transferClass` - the class of transfer job
Returns:
  boolean indicating whether any XBitJobs were succesfully added or not.

#### createSetXBitJob

    public Job createSetXBitJob(Job computeJob,
                                java.util.Collection<FileTransfer> execFiles,
                                int transferClass,
                                int xbitIndex)

Adds the dirmanager job to the workflow, that do a chmod on the files being staged.

Specified by:
  `createSetXBitJob` in interface `Implementation`
Parameters:
  `computeJob` - the computeJob for which the files are being staged.
  `execFiles` - the executable files that are being staged.
  `transferClass` - the class of transfer job
  `xbitIndex` - index to be used for creating the name of XBitJob.
Returns:
  the job object for the xBitJob

#### addSetXBitJobs

    public boolean addSetXBitJobs(Job computeJob,
                                  java.lang.String txJobName,
                                  java.util.Collection execFiles,
                                  int transferClass,
                                  int xbitIndex)

Adds the dirmanager job to the workflow, that do a chmod on the files being staged.

Specified by:
  `addSetXBitJobs` in interface `Implementation`
Parameters:
  `computeJob` - the computeJob for which the files are being staged.
  `txJobName` - the name of the transfer job that is staging the files.
  `execFiles` - the executable files that are being staged.
  `transferClass` - the class of transfer job
  `xbitIndex` - index to be used for creating the name of XBitJob.
Returns:
  boolean indicating whether any XBitJobs were succesfully added or not.

#### getSetXBitJobName

    public java.lang.String getSetXBitJobName(java.lang.String name,
                                              int counter)

Generates the name of the setXBitJob , that is unique for the given workflow.

Specified by:
  `getSetXBitJobName` in interface `Implementation`
Parameters:
  `name` - the name of the compute job
  `counter` - the index for the setXBit job.
Returns:
  the name of the setXBitJob .

#### getNOOPJobName

    public java.lang.String getNOOPJobName(java.lang.String name,
                                           int counter)

Generates the name of the noop job , that is unique for the given workflow.

Parameters:
  `name` - the name of the compute job
  `counter` - the index for the noop job.
Returns:
  the name of the setXBitJob .

#### createNoOPJob

    public Job createNoOPJob(java.lang.String name)

It creates a NoOP job that runs on the submit host.

Parameters:
  `name` - the name to be assigned to the noop job
Returns:
  the noop job.

#### createSetXBitJob

    protected Job createSetXBitJob(java.util.Collection<FileTransfer> files,
                                   java.lang.String name,
                                   java.lang.String site)

Creates a dirmanager job, that does a chmod on the file being staged. The file being staged should be of type executable. Though no explicit check is made for that. The staged file is the one whose X bit would be set on execution of this job. The site at which job is executed, is determined from the site associated with the destination URL.

Parameters:
  `files` - the collection `FileTransfer` containing the file that has to be X Bit Set.
  `name` - the name that has to be assigned to the job.
  `site` - the site at which the job has to be created
Returns:
  the chmod job, else null if it is not able to be created for some reason.

#### createSetXBitJob

    protected Job createSetXBitJob(FileTransfer file,
                                   java.lang.String name)

Creates a dirmanager job, that does a chmod on the file being staged. The file being staged should be of type executable. Though no explicit check is made for that. The staged file is the one whose X bit would be set on execution of this job. The site at which job is executed, is determined from the site associated with the destination URL.

Parameters:
  `file` - the `FileTransfer` containing the file that has to be X Bit Set.
  `name` - the name that has to be assigned to the job.
Returns:
  the chmod job, else null if it is not able to be created for some reason.

#### defaultXBitTCEntry

    private TransformationCatalogEntry defaultXBitTCEntry(java.lang.String site)

Returns a default TC entry to be used in case entry is not found in the transformation catalog.

Parameters:
  `site` - the site for which the default entry is required.
Returns:
  the default entry.

#### determineDisabledChmodSites

    protected java.util.Set determineDisabledChmodSites(java.lang.String sites)

Builds up a set of disabled chmod sites

Parameters:
  `sites` - comma separated list of sites.
Returns:
  a Set containing the site names.

#### disableChmodJobCreation

    protected boolean disableChmodJobCreation(java.lang.String site)

Returns a boolean indicating whether to disable chmod job creation for a site or not.

Parameters:
  `site` - the name of the site
Returns:
  boolean

#### getPriority

    protected java.lang.String getPriority(TransferJob job)

Returns the priority for the transfer job as specified in the properties file.

Parameters:
  `job` - the Transfer job.
Returns:
  the priority of the job as determined from properties, can be null if invalid value passed or property not set.

#### construct

    protected void construct(Job job,
                             java.lang.String key,
                             java.lang.String value)

Constructs a condor variable in the condor profile namespace associated with the job. Overrides any preexisting key values.

Parameters:
  `job` - contains the job description.
  `key` - the key of the profile.
  `value` - the associated value.