edu.isi.pegasus.planner.code.gridstart

## Class Kickstart

- java.lang.Object
- - edu.isi.pegasus.planner.code.gridstart.Kickstart

All Implemented Interfaces:
  GridStart

    public class Kickstart
    extends java.lang.Object
    implements GridStart

This enables a constituentJob to be run on the grid, by launching it through kickstart. The kickstart executable is a light-weight program which connects the stdin, stdout and stderr filehandles for Pegasus jobs on the remote site.

Sitting in between the remote scheduler and the executable, it is possible for kickstart to gather additional information about the executable run-time behavior, including the exit status of jobs.

Kickstart is an executable distributed with Pegasus that can generally be found at $PEGASUS_HOME/bin/kickstart

Version:
  $Revision$
Author:
  Karan Vahi vahi@isi.edu

### Field Summary

Fields
`static java.lang.String` `CLASSNAME` The basename of the class that is implmenting this.
`static java.lang.String` `COMPLETE_TRANSFORMATION_NAME` The complete TC name for kickstart.
`static java.lang.String` `EXECUTABLE_BASENAME` The basename of the kickstart executable.
`static java.lang.String` `KICKSTART_CLEANUP` The environment variable used to the set Kickstart CLEANUP JOB.
`static java.lang.String` `KICKSTART_INPUT_SUFFIX` The suffix for the kickstart input file, that is generated to use invoke at the remote end.
`static java.lang.String` `KICKSTART_POSTJOB` The environment variable used to the set Kickstart POSTJOB.
`static java.lang.String` `KICKSTART_PREJOB` The environment variable used to the set Kickstart PREJOB.
`static java.lang.String` `KICKSTART_SETUP` The environment variable used to the set Kickstart SETUP JOB.
`private ADag` `mConcDAG` The handle to the workflow that is being enabled.
`private boolean` `mDisableInvokeFunctionality` Boolean indicating whether to disable invoke functionality.
`private boolean` `mDoStat` A boolean indicating whether to stat files or not.
`private boolean` `mDynamicDeployment` A boolean indicating whether kickstart is deployed dynamically or not.
`private boolean` `mEnablingPartOfAggregatedJob` An instance variable to track if enabling is happening as part of a clustered constituentJob.
`private boolean` `mGenerateLOF` A boolean indicating whether to generate lof files or not.
`private boolean` `mInvokeAlways` A boolean indicating whether to use invoke always or not.
`private long` `mInvokeLength` The invoke limit trigger.
`private java.lang.String` `mKickstartLabel` The label that is passed to kickstart.
`private LogManager` `mLogger` The LogManager object which is used to log all the messages.
`private GridStart` `mNoGridStartImpl` Handle to NoGridStart implementation.
`private PlannerOptions` `mPOptions` The options passed to the planner.
`private PegasusProperties` `mProps` The object holding all the properties pertaining to Pegasus.
`private boolean` `mSetXBit` Whether kickstart should set the X Bit on the staged executables.
`private SiteStore` `mSiteStore` Handle to the site catalog store.
`private SLS` `mSLS` The handle to the SLS implementor
`private java.lang.String` `mSubmitDir` The submit exectionSiteDirectory where the submit files are being generated for the workflow.
`private TransformationCatalog` `mTCHandle` Handle to Transformation Catalog.
`private boolean` `mUseFullPathToGridStart` Boolean indicating whether to use full path or not
`(package private) boolean` `mWorkerNodeExecution` A boolean indicating whether to have worker node execution or not.
`static java.lang.String` `SHORT_NAME` The SHORTNAME for this implementation.
`static java.lang.String` `TRANSFORMATION_NAME` The logical name of kickstart
`static java.lang.String` `TRANSFORMATION_NAMESPACE` The transformation namespace for the kickstart
`static java.lang.String` `TRANSFORMATION_VERSION` The version number for kickstart.

### Fields inherited from interface edu.isi.pegasus.planner.code.GridStart

`mSeparator, VERSION`

### Constructor Summary

Constructors
Constructor and Description
`Kickstart()`

### Method Summary

All Methods Instance Methods Concrete Methods
`private void` `addCleanupPostScript(Job job, java.util.List files)` Adds a /bin/rm post constituentJob to kickstart that removes the files passed.
`boolean` `canSetXBit()` Indicates whether the enabling mechanism can set the X bit on the executable on the remote grid site, in addition to launching it on the remote grid site.
`private void` `construct(Job job, java.lang.String key, java.lang.String value)` Constructs a condor variable in the condor profile namespace associated with the constituentJob.
`protected java.lang.String` `constructCleanupJob(Job job, java.lang.String workerNodeTmp)` Constructs a kickstart setup constituentJob
`protected java.lang.String` `constructPREJob(Job job, java.lang.String headNodeURLPrefix, java.lang.String headNodeDirectory, java.lang.String workerNodeDirectory, java.lang.String slsFile)` Constructs the prejob that fetches sls file, and then invokes transfer again.
`protected java.lang.String` `constructSetupJob(Job job, java.lang.String workerNodeTmp)` Constructs a kickstart setup constituentJob
`java.lang.String` `defaultPOSTScript()` Returns the SHORT_NAME for the POSTScript implementation that is used to be as default with this GridStart implementation.
`boolean` `enable(AggregatedJob job, boolean isGlobusJob)` Enables a constituentJob to run on the grid.
`boolean` `enable(Job job, boolean isGlobusJob)` Enables a constituentJob to run on the grid by launching it through kickstart.
`protected boolean` `enable(Job job, boolean isGlobusJob, boolean stat, boolean addPostScript, boolean partOfClusteredJob)` Enables a constituentJob to run on the grid by launching it through kickstart.
`java.lang.String` `generateListofFilenamesFile(java.util.Set files, java.lang.String basename)` Writes out the list of filenames file for the constituentJob.
`protected java.lang.String` `getDirectory(Job job)` Returns the directory in which the job should run.
`private java.lang.String` `getDirectoryKey(Job job)` Returns the exectionSiteDirectory that is associated with the constituentJob to specify the exectionSiteDirectory in which the constituentJob needs to run
`java.lang.String` `getKickstartPath(SiteCatalogEntry site)` Returns the default path to kickstart as constructed from the environment variable associated with a site in the site catalog
`TransformationCatalogEntry` `getTransformationCatalogEntry(java.lang.String site)` Returns the transformation catalog entry for kickstart on a site
`java.lang.String` `getVDSKeyValue()` Returns the value of the vds profile with key as Pegasus.GRIDSTART_KEY, that would result in the loading of this particular implementation.
`java.lang.String` `getWorkerNodeDirectory(Job job)` Returns the exectionSiteDirectory in which the constituentJob executes on the worker node.
`protected java.lang.String` `handleTransferOfExecutable(Job job, java.lang.String path)` It changes the paths to the executable depending on whether we want to transfer the executable or not.
`void` `initialize(PegasusBag bag, ADag dag)` Initializes the GridStart implementation.
`private java.lang.String` `quote(java.lang.String string)` Condor Quotes a string
`protected boolean` `requiresToSetDirectory(Job job)` Returns a boolean indicating whether we need to set the directory for the job or not.
`java.lang.String` `shortDescribe()` Returns a short textual description in the form of the name of the class.
`void` `useFullPathToGridStarts(boolean fullPath)` Setter method to control whether a full path to Gridstart should be returned while wrapping a job or not.
`private boolean` `useInvoke(Job job, java.lang.String executable, java.lang.StringBuffer args)` Triggers the creation of the kickstart input file, that contains the the remote executable and the arguments with which it has to be invoked.

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Field Detail

#### TRANSFORMATION_NAMESPACE

    public static final java.lang.String TRANSFORMATION_NAMESPACE

The transformation namespace for the kickstart

See Also:
  Constant Field Values

#### TRANSFORMATION_NAME

    public static final java.lang.String TRANSFORMATION_NAME

The logical name of kickstart

See Also:
  Constant Field Values

#### TRANSFORMATION_VERSION

    public static final java.lang.String TRANSFORMATION_VERSION

The version number for kickstart.

#### EXECUTABLE_BASENAME

    public static final java.lang.String EXECUTABLE_BASENAME

The basename of the kickstart executable.

See Also:
  Constant Field Values

#### COMPLETE_TRANSFORMATION_NAME

    public static final java.lang.String COMPLETE_TRANSFORMATION_NAME

The complete TC name for kickstart.

#### KICKSTART_INPUT_SUFFIX

    public static final java.lang.String KICKSTART_INPUT_SUFFIX

The suffix for the kickstart input file, that is generated to use invoke at the remote end.

See Also:
  Constant Field Values

#### CLASSNAME

    public static final java.lang.String CLASSNAME

The basename of the class that is implmenting this. Could have been determined by reflection.

See Also:
  Constant Field Values

#### SHORT_NAME

    public static final java.lang.String SHORT_NAME

The SHORTNAME for this implementation.

See Also:
  Constant Field Values

#### KICKSTART_SETUP

    public static final java.lang.String KICKSTART_SETUP

The environment variable used to the set Kickstart SETUP JOB.

See Also:
  Constant Field Values

#### KICKSTART_PREJOB

    public static final java.lang.String KICKSTART_PREJOB

The environment variable used to the set Kickstart PREJOB.

See Also:
  Constant Field Values

#### KICKSTART_POSTJOB

    public static final java.lang.String KICKSTART_POSTJOB

The environment variable used to the set Kickstart POSTJOB.

See Also:
  Constant Field Values

#### KICKSTART_CLEANUP

    public static final java.lang.String KICKSTART_CLEANUP

The environment variable used to the set Kickstart CLEANUP JOB.

See Also:
  Constant Field Values

#### mLogger

    private LogManager mLogger

The LogManager object which is used to log all the messages.

#### mProps

    private PegasusProperties mProps

The object holding all the properties pertaining to Pegasus.

#### mPOptions

    private PlannerOptions mPOptions

The options passed to the planner.

#### mConcDAG

    private ADag mConcDAG

The handle to the workflow that is being enabled.

#### mSiteStore

    private SiteStore mSiteStore

Handle to the site catalog store.

#### mTCHandle

    private TransformationCatalog mTCHandle

Handle to Transformation Catalog.

#### mSubmitDir

    private java.lang.String mSubmitDir

The submit exectionSiteDirectory where the submit files are being generated for the workflow.

#### mInvokeAlways

    private boolean mInvokeAlways

A boolean indicating whether to use invoke always or not.

#### mDoStat

    private boolean mDoStat

A boolean indicating whether to stat files or not.

#### mGenerateLOF

    private boolean mGenerateLOF

A boolean indicating whether to generate lof files or not.

#### mInvokeLength

    private long mInvokeLength

The invoke limit trigger.

#### mWorkerNodeExecution

    boolean mWorkerNodeExecution

A boolean indicating whether to have worker node execution or not.

#### mSLS

    private SLS mSLS

The handle to the SLS implementor

#### mEnablingPartOfAggregatedJob

    private boolean mEnablingPartOfAggregatedJob

An instance variable to track if enabling is happening as part of a clustered constituentJob. See Bug 21 comments on Pegasus Bugzilla

#### mDynamicDeployment

    private boolean mDynamicDeployment

A boolean indicating whether kickstart is deployed dynamically or not.

#### mKickstartLabel

    private java.lang.String mKickstartLabel

The label that is passed to kickstart.

#### mSetXBit

    private boolean mSetXBit

Whether kickstart should set the X Bit on the staged executables.

#### mNoGridStartImpl

    private GridStart mNoGridStartImpl

Handle to NoGridStart implementation.

#### mUseFullPathToGridStart

    private boolean mUseFullPathToGridStart

Boolean indicating whether to use full path or not

#### mDisableInvokeFunctionality

    private boolean mDisableInvokeFunctionality

Boolean indicating whether to disable invoke functionality.

### Constructor Detail

#### Kickstart

    public Kickstart()

### Method Detail

#### initialize

    public void initialize(PegasusBag bag,
                           ADag dag)

Initializes the GridStart implementation.

Specified by:
  `initialize` in interface `GridStart`
Parameters:
  `bag` - the bag of objects that is used for initialization.
  `dag` - the concrete dag so far.

#### useFullPathToGridStarts

    public void useFullPathToGridStarts(boolean fullPath)

Setter method to control whether a full path to Gridstart should be returned while wrapping a job or not.

Specified by:
  `useFullPathToGridStarts` in interface `GridStart`
Parameters:
  `fullPath` - if set to true, indicates that full path would be used.

#### enable

    public boolean enable(AggregatedJob job,
                          boolean isGlobusJob)

Enables a constituentJob to run on the grid. This also determines how the stdin,stderr and stdout of the constituentJob are to be propogated. To grid enable a constituentJob, the constituentJob may need to be wrapped into another constituentJob, that actually launches the constituentJob. It usually results in the constituentJob description passed being modified modified.

Specified by:
  `enable` in interface `GridStart`
Parameters:
  `constituentJob` - the `Job` object containing the constituentJob description of the constituentJob that has to be enabled on the grid.
  `isGlobusJob` - is `true`, if the constituentJob generated a line `universe = globus`, and thus runs remotely. Set to `false`, if the constituentJob runs on the submit host in any way.
Returns:
  boolean true if enabling was successful,else false.

#### enable

    public boolean enable(Job job,
                          boolean isGlobusJob)

Enables a constituentJob to run on the grid by launching it through kickstart. Does the stdio, and stderr handling of the constituentJob to be run on the grid. It modifies the constituentJob description, and also constructs all the valid option to be passed to kickstart for launching the executable.

Specified by:
  `enable` in interface `GridStart`
Parameters:
  `constituentJob` - the `Job` object containing the constituentJob description of the constituentJob that has to be enabled on the grid.
  `isGlobusJob` - is `true`, if the constituentJob generated a line `universe = globus`, and thus runs remotely. Set to `false`, if the constituentJob runs on the submit host in any way.
Returns:
  boolean true if enabling was successful,else false in case when the path to kickstart could not be determined on the site where the constituentJob is scheduled.

#### enable

    protected boolean enable(Job job,
                             boolean isGlobusJob,
                             boolean stat,
                             boolean addPostScript,
                             boolean partOfClusteredJob)

Enables a constituentJob to run on the grid by launching it through kickstart. Does the stdio, and stderr handling of the constituentJob to be run on the grid. It modifies the constituentJob description, and also constructs all the valid option to be passed to kickstart for launching the executable.

Parameters:
  `constituentJob` - the `Job` object containing the constituentJob description of the constituentJob that has to be enabled on the grid.
  `isGlobusJob` - is `true`, if the constituentJob generated a line `universe = globus`, and thus runs remotely. Set to `false`, if the constituentJob runs on the submit host in any way.
  `stat` - boolean indicating whether to generate the lof files for kickstart stat option or not.
  `addPostScript` - boolean indicating whether to add a postscript or not.
  `partOfClusteredJob` - boolean indicating whether the job being enabled is part of a clustered job or not.
Returns:
  boolean true if enabling was successful,else false in case when the path to kickstart could not be determined on the site where the constituentJob is scheduled.

#### handleTransferOfExecutable

    protected java.lang.String handleTransferOfExecutable(Job job,
                                                          java.lang.String path)

It changes the paths to the executable depending on whether we want to transfer the executable or not. If the transfer_executable is set to true, then the executable needs to be shipped from the submit host meaning the local pool. This function changes the path of the executable to the one on the local pool, so that it can be shipped. If the worker package is being deployed dynamically, then the path is set to the exectionSiteDirectory where the worker package is deployed. Else, we pick up the path from the site catalog that is passed as input

Parameters:
  `constituentJob` - the `Job` containing the constituentJob description.
  `path` - the path to kickstart on the remote compute site, as determined from the site catalog.
Returns:
  the path that needs to be set as the executable

#### getTransformationCatalogEntry

    public TransformationCatalogEntry getTransformationCatalogEntry(java.lang.String site)

Returns the transformation catalog entry for kickstart on a site

Parameters:
  `site` - the site on which the entry is required
Returns:
  the entry if found else null

#### getKickstartPath

    public java.lang.String getKickstartPath(SiteCatalogEntry site)

Returns the default path to kickstart as constructed from the environment variable associated with a site in the site catalog

Parameters:
  `site` - the SiteCatalogEntry object for the site.
Returns:
  value if set else null

#### getWorkerNodeDirectory

    public java.lang.String getWorkerNodeDirectory(Job job)

Returns the exectionSiteDirectory in which the constituentJob executes on the worker node.

Specified by:
  `getWorkerNodeDirectory` in interface `GridStart`
Parameters:
  `constituentJob` -
Returns:
  the full path to the exectionSiteDirectory where the constituentJob executes

#### canSetXBit

    public boolean canSetXBit()

Indicates whether the enabling mechanism can set the X bit on the executable on the remote grid site, in addition to launching it on the remote grid site.

Specified by:
  `canSetXBit` in interface `GridStart`
Returns:
  true indicating Kickstart can set the X bit or not.

#### getVDSKeyValue

    public java.lang.String getVDSKeyValue()

Returns the value of the vds profile with key as Pegasus.GRIDSTART_KEY, that would result in the loading of this particular implementation. It is usually the name of the implementing class without the package name.

Specified by:
  `getVDSKeyValue` in interface `GridStart`
Returns:
  the value of the profile key.
See Also:
  `org.griphyn.cPlanner.namespace.Pegasus#GRIDSTART_KEY`

#### shortDescribe

    public java.lang.String shortDescribe()

Returns a short textual description in the form of the name of the class.

Specified by:
  `shortDescribe` in interface `GridStart`
Returns:
  short textual description.

#### defaultPOSTScript

    public java.lang.String defaultPOSTScript()

Returns the SHORT_NAME for the POSTScript implementation that is used to be as default with this GridStart implementation.

Specified by:
  `defaultPOSTScript` in interface `GridStart`
Returns:
  the identifier for the PegasusExitCode POSTScript implementation.
See Also:
  `POSTScript.shortDescribe()`

#### requiresToSetDirectory

    protected boolean requiresToSetDirectory(Job job)

Returns a boolean indicating whether we need to set the directory for the job or not.

Parameters:
  `job` - the job for which to set directory.
Returns:

#### getDirectory

    protected java.lang.String getDirectory(Job job)

Returns the directory in which the job should run.

Parameters:
  `job` - the job in which the directory has to run.
Returns:

#### getDirectoryKey

    private java.lang.String getDirectoryKey(Job job)

Returns the exectionSiteDirectory that is associated with the constituentJob to specify the exectionSiteDirectory in which the constituentJob needs to run

Parameters:
  `constituentJob` - the constituentJob
Returns:
  the condor key . can be initialdir or remote_initialdir

#### useInvoke

    private boolean useInvoke(Job job,
                              java.lang.String executable,
                              java.lang.StringBuffer args)

Triggers the creation of the kickstart input file, that contains the the remote executable and the arguments with which it has to be invoked. The kickstart input file is created in the submit directory.

Parameters:
  `constituentJob` - the `Job` object containing the constituentJob description.
  `executable` - the path to the executable used.
  `args` - the arguments buffer for gridstart invocation so far.
Returns:
  boolean indicating whether kickstart input file was generated or not. false in case of any error.

#### constructSetupJob

    protected java.lang.String constructSetupJob(Job job,
                                                 java.lang.String workerNodeTmp)

Constructs a kickstart setup constituentJob

Parameters:
  `constituentJob` - the constituentJob to be run.
  `workerNodeTmp` - the worker node tmp to run the constituentJob in.
Returns:
  String

#### constructCleanupJob

    protected java.lang.String constructCleanupJob(Job job,
                                                   java.lang.String workerNodeTmp)

Constructs a kickstart setup constituentJob

Parameters:
  `constituentJob` - the constituentJob to be run.
  `workerNodeTmp` - the worker node tmp to run the constituentJob in.
Returns:
  String

#### constructPREJob

    protected java.lang.String constructPREJob(Job job,
                                               java.lang.String headNodeURLPrefix,
                                               java.lang.String headNodeDirectory,
                                               java.lang.String workerNodeDirectory,
                                               java.lang.String slsFile)

Constructs the prejob that fetches sls file, and then invokes transfer again.

Parameters:
  `constituentJob` - the constituentJob for which the prejob is being created
  `headNodeURLPrefix` - String
  `headNodeDirectory` - String
  `workerNodeDirectory` - String
  `slsFile` - String
Returns:
  String containing the prescript invocation

#### generateListofFilenamesFile

    public java.lang.String generateListofFilenamesFile(java.util.Set files,
                                                        java.lang.String basename)

Writes out the list of filenames file for the constituentJob.

Parameters:
  `files` - the list of `PegasusFile` objects contains the files whose stat information is required.
  `basename` - the basename of the file that is to be created
Returns:
  the full path to lof file created, else null if no file is written out.

#### construct

    private void construct(Job job,
                           java.lang.String key,
                           java.lang.String value)

Constructs a condor variable in the condor profile namespace associated with the constituentJob. Overrides any preexisting key values.

Parameters:
  `constituentJob` - contains the constituentJob description.
  `key` - the key of the profile.
  `value` - the associated value.

#### quote

    private java.lang.String quote(java.lang.String string)

Condor Quotes a string

Parameters:
  `string` - the string to be quoted.
Returns:
  quoted string.

#### addCleanupPostScript

    private void addCleanupPostScript(Job job,
                                      java.util.List files)

Adds a /bin/rm post constituentJob to kickstart that removes the files passed. The post jobs is added as an environment variable.

Parameters:
  `constituentJob` - the constituentJob in which the post constituentJob needs to be added.
  `files` - the files to be deleted.