edu.isi.pegasus.planner.transfer.implementation

## Class AbstractMultipleFTPerXFERJob

- java.lang.Object
- - edu.isi.pegasus.planner.transfer.implementation.Abstract
  - - edu.isi.pegasus.planner.transfer.implementation.AbstractMultipleFTPerXFERJob

All Implemented Interfaces:
  Implementation, MultipleFTPerXFERJob

Direct Known Subclasses:
  GUC, T2, Transfer

    public abstract class AbstractMultipleFTPerXFERJob
    extends Abstract
    implements MultipleFTPerXFERJob

An abstract implementation for implementations that can handle multiple file transfers in a single file transfer job.

Version:
  $Revision$
Author:
  Karan Vahi

### Field Summary

### Fields inherited from class edu.isi.pegasus.planner.transfer.implementation.Abstract

`CHANGE_XBIT_TRANSFORMATION, mAddNodesForSettingXBit, mChmodDisabledForAllSites, mDisabledChmodSites, mLocalIrodsEnv, mLocalIrodsEnvBasename, mLocalS3cfg, mLocalS3cfgBasename, mLocalUserProxy, mLocalUserProxyBasename, mLogger, mPOptions, mProps, mRefiner, mSiteStore, mTCHandle, NOOP_PREFIX, SET_XBIT_PREFIX, XBIT_DERIVATION_NS, XBIT_DERIVATION_VERSION, XBIT_EXECUTABLE_BASENAME, XBIT_TRANSFORMATION_NS, XBIT_TRANSFORMATION_VERSION`

### Fields inherited from interface edu.isi.pegasus.planner.transfer.Implementation

`TRANSFER_UNIVERSE, VERSION`

### Constructor Summary

Constructors
Constructor and Description
`AbstractMultipleFTPerXFERJob(PegasusBag bag)` The overloaded constructor, that is called by the Factory to load the class.

### Method Summary

All Methods Instance Methods Abstract Methods Concrete Methods
`TransferJob` `createTransferJob(Job job, java.lang.String site, java.util.Collection files, java.util.Collection execFiles, java.lang.String txJobName, int jobClass)` Constructs a general transfer job that handles multiple transfers per transfer job.
`protected TransformationCatalogEntry` `defaultTCEntry(java.lang.String namespace, java.lang.String name, java.lang.String version, java.lang.String executableBasename, java.lang.String site)` Returns a default TC entry to be used in case entry is not found in the transformation catalog.
`protected abstract java.lang.String` `generateArgumentString(TransferJob job)` It constructs the arguments to the transfer executable that need to be passed to the executable referred to in this transfer mode.
`protected abstract java.lang.String` `getCompleteTCName()` Returns the complete name for the transformation that the implementation is using..
`protected abstract java.lang.String` `getDerivationName()` Returns the logical name of the derivation that this implementation refers to.
`protected abstract java.lang.String` `getDerivationNamespace()` Returns the namespace of the derivation that this implementation refers to.
`protected abstract java.lang.String` `getDerivationVersion()` Returns the version of the derivation that this implementation refers to.
`protected abstract java.util.List<Profile>` `getEnvironmentVariables(java.lang.String site)` Returns the environment profiles that are required for the default entry to sensibly work.
`void` `postProcess(TransferJob job)` An optional method that allows the derived classes to do their own post processing on the the transfer job before it is returned to the calling module.
`protected java.lang.String` `prepareSTDINAndAssociateCredentials(TransferJob job, java.util.Collection files, java.lang.String stagingSite, int jobClass)` Prepares the stdin for the transfer job.
`protected abstract void` `writeStdInAndAssociateCredentials(TransferJob job, java.io.FileWriter stdIn, java.util.Collection files, java.lang.String stagingSite, int jobClass)` Writes to a FileWriter stream the stdin which goes into the magic script via standard input

### Methods inherited from class edu.isi.pegasus.planner.transfer.implementation.Abstract

`addSetXBitJobs, addSetXBitJobs, addSetXBitJobs, applyPriority, checkAndTransferIrodsEnvFile, checkAndTransferProxy, construct, createNoOPJob, createSetXBitJob, createSetXBitJob, createSetXBitJob, determineDisabledChmodSites, disableChmodJobCreation, getNOOPJobName, getPriority, getSetXBitJobName, setRefiner`

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Methods inherited from interface edu.isi.pegasus.planner.transfer.Implementation

`doesPreserveXBit, getDescription, getTransformationCatalogEntry, useThirdPartyTransferAlways`

### Constructor Detail

#### AbstractMultipleFTPerXFERJob

    public AbstractMultipleFTPerXFERJob(PegasusBag bag)

The overloaded constructor, that is called by the Factory to load the class.

Parameters:
  `bag` - the bag of Pegasus initialization objects

### Method Detail

#### createTransferJob

    public TransferJob createTransferJob(Job job,
                                         java.lang.String site,
                                         java.util.Collection files,
                                         java.util.Collection execFiles,
                                         java.lang.String txJobName,
                                         int jobClass)

Constructs a general transfer job that handles multiple transfers per transfer job. There are appropriate callouts to generate the implementation specific details.

Specified by:
  `createTransferJob` in interface `Implementation`
Parameters:
  `job` - the Job object for the job, in relation to which the transfer node is being added. Either the transfer node can be transferring this jobs input files to the execution pool, or transferring this job's output files to the output pool.
  `site` - the site where the transfer job should run.
  `files` - collection of `FileTransfer` objects representing the data files and staged executables to be transferred.
  `execFiles` - subset collection of the files parameter, that identifies the executable files that are being transferred.
  `txJobName` - the name of transfer node.
  `jobClass` - the job Class for the newly added job. Can be one of the following: stage-in stage-out inter-pool transfer stage-in worker transfer
Returns:
  the created TransferJob.

#### defaultTCEntry

    protected TransformationCatalogEntry defaultTCEntry(java.lang.String namespace,
                                                        java.lang.String name,
                                                        java.lang.String version,
                                                        java.lang.String executableBasename,
                                                        java.lang.String site)

Returns a default TC entry to be used in case entry is not found in the transformation catalog.

Parameters:
  `namespace` - the namespace of the transfer transformation
  `name` - the logical name of the transfer transformation
  `version` - the version of the transfer transformation
  `executableBasename` - the basename of the executable
  `site` - the site for which the default entry is required.
Returns:
  the default entry.

#### getEnvironmentVariables

    protected abstract java.util.List<Profile> getEnvironmentVariables(java.lang.String site)

Returns the environment profiles that are required for the default entry to sensibly work.

Parameters:
  `site` - the site where the job is going to run.
Returns:
  List of environment variables, else null in case where the required environment variables could not be found.

#### postProcess

    public void postProcess(TransferJob job)

An optional method that allows the derived classes to do their own post processing on the the transfer job before it is returned to the calling module.

Parameters:
  `job` - the `TransferJob` that has been created.

#### prepareSTDINAndAssociateCredentials

    protected java.lang.String prepareSTDINAndAssociateCredentials(TransferJob job,
                                                                   java.util.Collection files,
                                                                   java.lang.String stagingSite,
                                                                   int jobClass)
                                                            throws java.lang.Exception

Prepares the stdin for the transfer job. Usually involves writing out a text file that Condor transfers to the remote end. Additionally, it associates credentials with the job that are requried to for the transfers.

Parameters:
  `name` - the name of the transfer job.
  `files` - Collection of `FileTransfer` objects containing the information about sourceam fin and destURL's.
  `stagingSite` - the site where the data will be populated by first level staging jobs.
  `jobClass` - the job Class for the newly added job. Can be one of the following: stage-in stage-out inter-pool transfer
Returns:
  the path to the prepared stdin file.
Throws:
  `java.lang.Exception` - in case of error.

#### getDerivationNamespace

    protected abstract java.lang.String getDerivationNamespace()

Returns the namespace of the derivation that this implementation refers to.

Returns:
  the namespace of the derivation.

#### getDerivationName

    protected abstract java.lang.String getDerivationName()

Returns the logical name of the derivation that this implementation refers to.

Returns:
  the name of the derivation.

#### getDerivationVersion

    protected abstract java.lang.String getDerivationVersion()

Returns the version of the derivation that this implementation refers to.

Returns:
  the version of the derivation.

#### generateArgumentString

    protected abstract java.lang.String generateArgumentString(TransferJob job)

It constructs the arguments to the transfer executable that need to be passed to the executable referred to in this transfer mode.

Parameters:
  `job` - the object containing the transfer node.
Returns:
  the argument string

#### writeStdInAndAssociateCredentials

    protected abstract void writeStdInAndAssociateCredentials(TransferJob job,
                                                              java.io.FileWriter stdIn,
                                                              java.util.Collection files,
                                                              java.lang.String stagingSite,
                                                              int jobClass)
                                                       throws java.lang.Exception

Writes to a FileWriter stream the stdin which goes into the magic script via standard input

Parameters:
  `job` - the transfer job .
  `stdIn` - the writer to the stdin file.
  `files` - Collection of `FileTransfer` objects containing the information about sourceam fin and destURL's.
  `stagingSite` - the site where the data will be populated by first level staging jobs.
  `jobClass` - the job Class for the newly added job. Can be one of the following: stage-in stage-out inter-pool transfer
Throws:
  `java.lang.Exception`

#### getCompleteTCName

    protected abstract java.lang.String getCompleteTCName()

Returns the complete name for the transformation that the implementation is using..

Returns:
  the complete name.