edu.isi.pegasus.planner.client

## Class CPlanner

- java.lang.Object
- - edu.isi.pegasus.planner.client.Executable
  - - edu.isi.pegasus.planner.client.CPlanner

    public class CPlanner
    extends Executable

This is the main program for the Pegasus. It parses the options specified by the user and calls out to the appropriate components to parse the abstract plan, concretize it and then write the submit files.

Version:
  $Revision$
Author:
  Gaurang Mehta, Karan Vahi

### Field Summary

Fields
`static java.lang.String` `CLEANUP_DIR` The basename of the directory that contains the submit files for the cleanup DAG that for the concrete dag generated for the workflow.
`static java.lang.String` `DEFAULT_MEGADAG_MODE` The default megadag mode that is used for generation of megadags in deferred planning.
`private static java.lang.String` `EMPTY_FINAL_WORKFLOW_MESSAGE` The final successful message that is to be logged.
`static java.lang.String` `JAVA_COMMAND_LINE_PROPERTY_REGEX` The regex used to match against a java property that is set using -Dpropertyname=value in the argument string
`private PegasusBag` `mBag` The object containing the bag of pegasus objects
`private java.text.NumberFormat` `mNumFormatter` The number formatter to format the run submit dir entries.
`private PlannerMetrics` `mPMetrics` The PlannerMetrics object storing the metrics about this planning instance.
`private PlannerOptions` `mPOptions` The object containing all the options passed to the Concrete Planner.
`private boolean` `mSendMetrics` A boolean indicating whether metrics should be sent to metrics server or not
`private java.lang.String` `mUser` The user name of the user running Pegasus.
`static java.lang.String` `NOOP_PREFIX` The prefix for the NoOP jobs that are created.
`static java.lang.String` `PEGASUS_MONITORD_LAUNCH_PROPERTY_KEY` The name of the property key that determines whether pegasus-run should monitord or not.
`private static java.lang.String` `SUCCESS_MESSAGE` The message to be logged in case of empty executable workflow.

### Fields inherited from class edu.isi.pegasus.planner.client.Executable

`mLogger, mLogMsg, mProps, mVersion`

### Constructor Summary

Constructors
Constructor and Description
`CPlanner()` Default constructor.
`CPlanner(LogManager logger)` The overload constructor.

### Method Summary

All Methods Static Methods Instance Methods Concrete Methods
`protected void` `construct(Job job, java.lang.String key, java.lang.String value)` Constructs a condor variable in the condor profile namespace associated with the job.
`protected Job` `createNoOPJob(java.lang.String name)` It creates a NoOP job that runs on the submit host.
`protected boolean` `createSymbolicLink(java.lang.String source, java.lang.String destination, java.io.File directory, boolean logErrorToDebug)` This method generates a symlink between two files
`protected java.lang.String` `determineRelativeSubmitDirectory(ADag dag, java.lang.String dir, java.lang.String user, java.lang.String vogroup, boolean timestampBased)` Creates the submit directory for the workflow.
`protected java.lang.String` `determineRelativeSubmitDirectory(java.lang.String label, java.lang.String dir, java.lang.String user, java.lang.String vogroup, boolean timestampBased)` Creates the submit directory for the workflow.
`private java.lang.String` `determineRootWorkflowUUID(ADag dag, PlannerOptions options, PegasusProperties properties)` Determines the workflow uuid for a workflow
`protected java.lang.String` `doBackupAndCreateSymbolicLinkForSubmitDirectory(java.lang.String baseDir, java.lang.String relativeSubmitDir)`
`void` `executeCommand()` Executes the command on the basis of the options specified.
`java.util.Collection<java.io.File>` `executeCommand(PlannerOptions options)` Executes the command on the basis of the options specified.
`private void` `generateStampedeEventsForAbstractWorkflow(ADag workflow, PegasusBag bag)` Generates events for the abstract workflow.
`gnu.getopt.LongOpt[]` `generateValidOptions()` Tt generates the LongOpt which contain the valid options that the command will accept.
`protected java.lang.String` `getDAGFilename(ADag dag, PlannerOptions options)` Returns the basename of the dag file
`java.lang.String` `getNOOPJobName(ADag dag)` Returns the name of the noop job.
`private java.lang.String` `getPegasusRunInvocation()` Returns the pegasus-run invocation on the workflow planned.
`protected java.lang.String` `getRandomDirectory(ADag dag)` Sets the basename of the random directory that is created on the remote sites per workflow.
`protected boolean` `handleRescueDAG(ADag dag, PlannerOptions options)` Checks for rescue dags, and determines whether to plan or not.
`protected boolean` `handleRescueDAG(java.lang.String dag, java.lang.String dir, int numOfRescues)` Checks for rescue dags, and determines whether to submit a rescue dag or not.
`void` `initialize(java.lang.String[] opts, char confChar)` Initialize the executable object
`void` `loadProperties()` Loads all the properties that are needed by this class.
`private SiteStore` `loadSiteStore(java.util.Set<java.lang.String> sites)` Loads the sites from the site catalog into the site store
`private void` `logMemoryUsage()` Logs memory usage of the JVM
`private void` `logSuccessfulCompletion(boolean emptyWorkflow)` Logs the successful completion message.
`static void` `main(java.lang.String[] args)` The main program for the CPlanner.
`PlannerOptions` `parseCommandLineArguments(java.lang.String[] args)` Parses the command line arguments using GetOpt and returns a `PlannerOptions` contains all the options passed by the user at the command line.
`PlannerOptions` `parseCommandLineArguments(java.lang.String[] args, boolean sanitizePath)` Parses the command line arguments using GetOpt and returns a `PlannerOptions` contains all the options passed by the user at the command line.
`private ADag` `parseDAX(java.lang.String dax)` Parses the DAX and returns the associated ADag object
`void` `printLongVersion()` Prints the long description, displaying in detail what the various options to the command stand for.
`void` `printShortVersion()` Prints out a short description of what the command does.
`protected static void` `sanityCheck(java.io.File dir)` Checks the destination location for existence, if it can be created, if it is writable etc.
`boolean` `submitWorkflow(java.lang.String invocation)` Submits the workflow for execution using pegasus-run, a wrapper around pegasus-submit-dag.

### Methods inherited from class edu.isi.pegasus.planner.client.Executable

`convertException, convertException, getCommandLineOptions, getEnvValue, getGVDSVersion, initialize, log, lookupConfProperty, sanityCheckOnProperties, setupLogging`

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Field Detail

#### DEFAULT_MEGADAG_MODE

    public static final java.lang.String DEFAULT_MEGADAG_MODE

The default megadag mode that is used for generation of megadags in deferred planning.

See Also:
  Constant Field Values

#### CLEANUP_DIR

    public static final java.lang.String CLEANUP_DIR

The basename of the directory that contains the submit files for the cleanup DAG that for the concrete dag generated for the workflow.

See Also:
  Constant Field Values

#### NOOP_PREFIX

    public static final java.lang.String NOOP_PREFIX

The prefix for the NoOP jobs that are created.

See Also:
  Constant Field Values

#### PEGASUS_MONITORD_LAUNCH_PROPERTY_KEY

    public static final java.lang.String PEGASUS_MONITORD_LAUNCH_PROPERTY_KEY

The name of the property key that determines whether pegasus-run should monitord or not.

See Also:
  Constant Field Values

#### JAVA_COMMAND_LINE_PROPERTY_REGEX

    public static final java.lang.String JAVA_COMMAND_LINE_PROPERTY_REGEX

The regex used to match against a java property that is set using -Dpropertyname=value in the argument string

See Also:
  Constant Field Values

#### EMPTY_FINAL_WORKFLOW_MESSAGE

    private static final java.lang.String EMPTY_FINAL_WORKFLOW_MESSAGE

The final successful message that is to be logged.

See Also:
  Constant Field Values

#### SUCCESS_MESSAGE

    private static final java.lang.String SUCCESS_MESSAGE

The message to be logged in case of empty executable workflow.

See Also:
  Constant Field Values

#### mPOptions

    private PlannerOptions mPOptions

The object containing all the options passed to the Concrete Planner.

#### mBag

    private PegasusBag mBag

The object containing the bag of pegasus objects

#### mPMetrics

    private PlannerMetrics mPMetrics

The PlannerMetrics object storing the metrics about this planning instance.

#### mNumFormatter

    private java.text.NumberFormat mNumFormatter

The number formatter to format the run submit dir entries.

#### mUser

    private java.lang.String mUser

The user name of the user running Pegasus.

#### mSendMetrics

    private boolean mSendMetrics

A boolean indicating whether metrics should be sent to metrics server or not

### Constructor Detail

#### CPlanner

    public CPlanner()

Default constructor.

#### CPlanner

    public CPlanner(LogManager logger)

The overload constructor.

Parameters:
  `logger` - the logger object to use. can be null.

### Method Detail

#### initialize

    public void initialize(java.lang.String[] opts,
                           char confChar)

Description copied from class: `Executable`
Initialize the executable object

Overrides:
  `initialize` in class `Executable`
Parameters:
  `opts` - the command line argument passed by the user
  `confChar` - the short option corresponding the conf property.

#### main

    public static void main(java.lang.String[] args)

The main program for the CPlanner.

Parameters:
  `args` - the main arguments passed to the planner.

#### loadProperties

    public void loadProperties()

Loads all the properties that are needed by this class.

Specified by:
  `loadProperties` in class `Executable`

#### executeCommand

    public void executeCommand()

Executes the command on the basis of the options specified.

Parameters:
  `args` - the command line options.

#### executeCommand

    public java.util.Collection<java.io.File> executeCommand(PlannerOptions options)

Executes the command on the basis of the options specified.

Parameters:
  `options` - the command line options.
Returns:
  the Collection of `File` objects for the files written out.

#### getNOOPJobName

    public java.lang.String getNOOPJobName(ADag dag)

Returns the name of the noop job.

Parameters:
  `dag` - the workflow
Returns:
  the name

#### createNoOPJob

    protected Job createNoOPJob(java.lang.String name)

It creates a NoOP job that runs on the submit host.

Parameters:
  `name` - the name to be assigned to the noop job
Returns:
  the noop job.

#### construct

    protected void construct(Job job,
                             java.lang.String key,
                             java.lang.String value)

Constructs a condor variable in the condor profile namespace associated with the job. Overrides any preexisting key values.

Parameters:
  `job` - contains the job description.
  `key` - the key of the profile.
  `value` - the associated value.

#### parseCommandLineArguments

    public PlannerOptions parseCommandLineArguments(java.lang.String[] args)

Parses the command line arguments using GetOpt and returns a `PlannerOptions` contains all the options passed by the user at the command line.

Parameters:
  `args` - the arguments passed by the user at command line.
Returns:
  the options.

#### parseCommandLineArguments

    public PlannerOptions parseCommandLineArguments(java.lang.String[] args,
                                                    boolean sanitizePath)

Parses the command line arguments using GetOpt and returns a `PlannerOptions` contains all the options passed by the user at the command line.

Parameters:
  `args` - the arguments passed by the user at command line.
  `sanitizePath` - whether to sanitize path during construction of options
Returns:
  the options.

#### submitWorkflow

    public boolean submitWorkflow(java.lang.String invocation)

Submits the workflow for execution using pegasus-run, a wrapper around pegasus-submit-dag.

Parameters:
  `invocation` - the pegasus run invocation
Returns:
  boolean indicating whether could successfully submit the workflow or not.

#### getRandomDirectory

    protected java.lang.String getRandomDirectory(ADag dag)

Sets the basename of the random directory that is created on the remote sites per workflow. The name is generated by default from teh flow ID, unless a basename prefix is specifed at runtime in the planner options.

Parameters:
  `dag` - the DAG containing the abstract workflow.
Returns:
  the basename of the random directory.

#### generateValidOptions

    public gnu.getopt.LongOpt[] generateValidOptions()

Tt generates the LongOpt which contain the valid options that the command will accept.

Specified by:
  `generateValidOptions` in class `Executable`
Returns:
  array of `LongOpt` objects , corresponding to the valid options

#### printShortVersion

    public void printShortVersion()

Prints out a short description of what the command does.

Specified by:
  `printShortVersion` in class `Executable`

#### printLongVersion

    public void printLongVersion()

Prints the long description, displaying in detail what the various options to the command stand for.

Specified by:
  `printLongVersion` in class `Executable`

#### determineRootWorkflowUUID

    private java.lang.String determineRootWorkflowUUID(ADag dag,
                                                       PlannerOptions options,
                                                       PegasusProperties properties)

Determines the workflow uuid for a workflow

Parameters:
  `dag` - the workflow
  `options` - the options passed to the planner
  `properties` - the properties passed to the planner
Returns:
  uuid for the root workflow instance

#### determineRelativeSubmitDirectory

    protected java.lang.String determineRelativeSubmitDirectory(ADag dag,
                                                                java.lang.String dir,
                                                                java.lang.String user,
                                                                java.lang.String vogroup,
                                                                boolean timestampBased)
                                                         throws java.io.IOException

Creates the submit directory for the workflow. This is not thread safe.

Parameters:
  `dag` - the workflow being worked upon.
  `dir` - the base directory specified by the user.
  `user` - the username of the user.
  `vogroup` - the vogroup to which the user belongs to.
  `timestampBased` - boolean indicating whether to have a timestamp based dir or not
Returns:
  the directory name created relative to the base directory passed as input.
Throws:
  `java.io.IOException` - in case of unable to create submit directory.

#### determineRelativeSubmitDirectory

    protected java.lang.String determineRelativeSubmitDirectory(java.lang.String label,
                                                                java.lang.String dir,
                                                                java.lang.String user,
                                                                java.lang.String vogroup,
                                                                boolean timestampBased)
                                                         throws java.io.IOException

Creates the submit directory for the workflow. This is not thread safe.

Parameters:
  `label` - the label of the workflow
  `dir` - the base directory specified by the user.
  `user` - the username of the user.
  `vogroup` - the vogroup to which the user belongs to.
  `timestampBased` - boolean indicating whether to have a timestamp based dir or not
Returns:
  the directory name created relative to the base directory passed as input.
Throws:
  `java.io.IOException` - in case of unable to create submit directory.

#### sanityCheck

    protected static void sanityCheck(java.io.File dir)
                               throws java.io.IOException

Checks the destination location for existence, if it can be created, if it is writable etc.

Parameters:
  `dir` - is the new base directory to optionally create.
Throws:
  `java.io.IOException` - in case of error while writing out files.

#### getDAGFilename

    protected java.lang.String getDAGFilename(ADag dag,
                                              PlannerOptions options)

Returns the basename of the dag file

Parameters:
  `dag` - the dag that was parsed.
  `options` - the planner options
Returns:
  boolean true means submit the rescue false do the planning operation

#### handleRescueDAG

    protected boolean handleRescueDAG(ADag dag,
                                      PlannerOptions options)

Checks for rescue dags, and determines whether to plan or not.

Parameters:
  `dag` - the dag that was parsed.
  `options` - the planner options
Returns:
  boolean true means submit the rescue false do the planning operation

#### handleRescueDAG

    protected boolean handleRescueDAG(java.lang.String dag,
                                      java.lang.String dir,
                                      int numOfRescues)

Checks for rescue dags, and determines whether to submit a rescue dag or not.

Parameters:
  `dag` - the dag file for the dax
  `dir` - the submit directory.
  `numOfRescues` - the number of rescues to handle.
Returns:
  true means submit the rescue false do the planning operation

#### createSymbolicLink

    protected boolean createSymbolicLink(java.lang.String source,
                                         java.lang.String destination,
                                         java.io.File directory,
                                         boolean logErrorToDebug)

This method generates a symlink between two files

Parameters:
  `source` - the file that has to be symlinked
  `destination` - the destination of the symlink
  `directory` - the directory in which to execute the command
  `logErrorToDebug` - whether to log messeage to debug or not
Returns:
  boolean indicating if creation of symlink was successful or not

#### generateStampedeEventsForAbstractWorkflow

    private void generateStampedeEventsForAbstractWorkflow(ADag workflow,
                                                           PegasusBag bag)

Generates events for the abstract workflow.

Parameters:
  `workflow` - the parsed dax
  `bag` - the initialized object bag

#### loadSiteStore

    private SiteStore loadSiteStore(java.util.Set<java.lang.String> sites)

Loads the sites from the site catalog into the site store

Parameters:
  `sites` -
Returns:
  SiteStore object containing the information about the sites.

#### logSuccessfulCompletion

    private void logSuccessfulCompletion(boolean emptyWorkflow)

Logs the successful completion message.

Parameters:
  `emptyWorkflow` - indicates whether the workflow created was empty or not.

#### getPegasusRunInvocation

    private java.lang.String getPegasusRunInvocation()

Returns the pegasus-run invocation on the workflow planned.

Returns:
  the pegasus-run invocation

#### doBackupAndCreateSymbolicLinkForSubmitDirectory

    protected java.lang.String doBackupAndCreateSymbolicLinkForSubmitDirectory(java.lang.String baseDir,
                                                                               java.lang.String relativeSubmitDir)
                                                                        throws java.io.IOException

Throws:
  `java.io.IOException`

#### logMemoryUsage

    private void logMemoryUsage()

Logs memory usage of the JVM

#### parseDAX

    private ADag parseDAX(java.lang.String dax)

Parses the DAX and returns the associated ADag object

Parameters:
  `dax` - path to the DAX file.
Returns: