ndcctools.work_queue.PythonTask Class Reference

Python PythonTask object. More...

Inheritance diagram for ndcctools.work_queue.PythonTask:

## Public Member Functions

| __init__ (self, func, *args, **kwargs)
| Creates a new python task.
| output (self)
| returns the result of a python task as a python variable
Public Member Functions inherited from ndcctools.work_queue.Task
| __init__ (self, command)
| Create a new task specification.
| clone (self)
| Return a copy of this task.
| specify_command (self, command)
| Set the command to be executed by the task.
| specify_coprocess (self, coprocess)
| Set the coprocess at the worker that should execute the task's command.
| specify_algorithm (self, algorithm)
| Set the worker selection algorithm for task.
| specify_tag (self, tag)
| Attach a user defined logical name to the task.
| specify_category (self, name)
| Label the task with the given category.
| specify_feature (self, name)
| Label the task with the given user-defined feature.
| specify_file (self, local_name, remote_name=None, type=None, flags=None, cache=None, failure_only=None)
| Add a file to the task.
| specify_url (self, url, remote_name, type=None, flags=None, cache=None, failure_only=None)
| Add a url to the task which will be provided as an input file.
| specify_file_command (self, cmd, remote_name, type=None, flags=None, cache=None, failure_only=None)
| Add an input file produced by a Unix shell command.
| specify_file_piece (self, local_name, remote_name=None, start_byte=0, end_byte=0, type=None, flags=None, cache=None, failure_only=None)
| Add a file piece to the task.
| specify_input_file (self, local_name, remote_name=None, flags=None, cache=None)
| Add a input file to the task.
| specify_output_file (self, local_name, remote_name=None, flags=None, cache=None, failure_only=None)
| Add a output file to the task.
| specify_directory (self, local_name, remote_name=None, type=None, flags=None, recursive=False, cache=None, failure_only=None)
| Add a directory to the task.
| specify_buffer (self, buffer, remote_name, flags=None, cache=None)
| Add an input bufer to the task.
| specify_snapshot_file (self, filename)
| When monitoring, indicates a json-encoded file that instructs the monitor to take a snapshot of the task resources.
| specify_max_retries (self, max_retries)
| Indicate the number of times the task should be retried.
| specify_cores (self, cores)
| Indicate the number of cores required by this task.
| specify_memory (self, memory)
| Indicate the memory (in MB) required by this task.
| specify_disk (self, disk)
| Indicate the disk space (in MB) required by this task.
| specify_gpus (self, gpus)
| Indicate the number of GPUs required by this task.
| specify_priority (self, priority)
| Indicate the the priority of this task (larger means better priority, default is 0).
| specify_environment_variable (self, name, value=None)
| Set this environment variable before running the task.
| specify_monitor_output (self, directory)
| Set a name for the resource summary output directory from the monitor.
| tag (self)
| Get the user-defined logical name for the task.
| category (self)
| Get the category name for the task.
| command (self)
| Get the shell command executed by the task.
| priority (self)
| Get the priority of the task.
| algorithm (self)
| Get the algorithm for choosing worker to run the task.
| std_output (self)
| Get the standard output of the task.
| id (self)
| Get the task id number.
| return_status (self)
| Get the exit code of the command executed by the task.
| result (self)
| Get the result of the task as an integer code, such as successful, missing file, etc.
| result_str (self)
| Return a string that explains the result of a task.
| total_submissions (self)
| Get the number of times the task has been resubmitted internally.
| exhausted_attempts (self)
| Get the number of times the task has been failed given resource exhaustion.
| host (self)
| Get the address and port of the host on which the task ran.
| hostname (self)
| Get the name of the host on which the task ran.
| submit_time (self)
| Get the time at which this task was submitted.
| finish_time (self)
| Get the time at which this task was finished.
| total_cmd_exhausted_execute_time (self)
| Get the total time the task executed and failed given resource exhaustion.
| app_delay (self)
| Get the time spent in upper-level application (outside of work_queue_wait).
| send_input_start (self)
| Get the time at which the task started to transfer input files.
| send_input_finish (self)
| Get the time at which the task finished transferring input files.
| execute_cmd_start (self)
| The time at which the task began.
| execute_cmd_finish (self)
| Get the time at which the task finished (discovered by the manager).
| receive_output_start (self)
| Get the time at which the task started to transfer output files.
| receive_output_finish (self)
| Get the time at which the task finished transferring output files.
| total_bytes_received (self)
| Get the number of bytes received since task started receiving input data.
| total_bytes_sent (self)
| Get the number of bytes sent since task started sending input data.
| total_bytes_transferred (self)
| Get the number of bytes transferred since task started transferring input data.
| total_transfer_time (self)
| Get the time comsumed in microseconds for transferring total_bytes_transferred.
| cmd_execution_time (self)
| Time spent in microseconds for executing the command until completion on a single worker.
| total_cmd_execution_time (self)
| Accumulated time spent in microseconds for executing the command on any worker, regardless of whether the task finished (i.e., this includes time running on workers that disconnected).
| resources_measured (self)
| Get the resources measured for the task execution if resource monitoring is enabled.
| limits_exceeded (self)
| Get the resources the task exceeded.
| resources_requested (self)
| Get the resources the task requested to run.
| resources_allocated (self)
| Get the resources allocated to the task in its latest attempt.

## Protected Member Functions

| _create_wrapper (self)
| creates the wrapper script which will execute the function.

## Detailed Description

Python PythonTask object.

this class is used to create a python task

## Constructor & Destructor Documentation

## __init__()

ndcctools.work_queue.PythonTask.__init__ | ( |  | self,
|  |  | func,
|  | * | args,
|  | ** | kwargs )

Creates a new python task.

Parameters

  self | Reference to the current python task object
  func | python function to be executed by task
  args | arguments used in function to be executed by task
  kwargs | keyword arguments used in function to be executed by task

## Member Function Documentation

## output()

ndcctools.work_queue.PythonTask.output | ( |  | self | ) |

returns the result of a python task as a python variable

Parameters

  self | reference to the current python task object

Reimplemented from ndcctools.work_queue.Task.

## _create_wrapper()

ndcctools.work_queue.PythonTask._create_wrapper | ( |  | self | ) |

protected

creates the wrapper script which will execute the function.

pickles output.

The documentation for this class was generated from the following file:

- work_queue.py