ndcctools.taskvine.task.PythonTask Class Reference
Inheritance diagram for ndcctools.taskvine.task.PythonTask:

## Public Member Functions

| __init__ (self, func, *args, **kwargs)
| Creates a new python task.
| submit_finalize (self)
| Finalizes the task definition once the manager that will execute is run.
| enable_temp_output (self)
| Marks the output of this task to stay at the worker.
| set_output_cache (self, cache=False)
| Set the cache behavior for the output of the task.
| output_file (self)
| Returns the ndcctools.taskvine.file.File object that represents the output of this task.
| output (self)
| returns the result of a python task as a python variable
| disable_output_serialization (self)
| Disables serialization of results to disk when writing to a file for transmission.
Public Member Functions inherited from ndcctools.taskvine.task.Task
| __init__ (self, command, **task_info)
| Create a new task specification.
| addref (self)
| Return a copy of this task.
| set_command (self, command)
| Set the command to be executed by the task.
| set_library_required (self, library)
| Set the name of the library at the worker that should execute the task's command.
| get_library_required (self)
| Get the name of the library at the worker that should execute the task's command.
| needs_library (self, library)
| Deprecated, see set_library_required.
| set_library_provided (self, library_name)
| Set the library name provided by this task.
| get_libray_provided (self)
| Get the name of the library at the worker that should execute the task's command.
| provides_library (self, library)
| Deprecated, see set_library_provided.
| set_function_slots (self, nslots)
| Set the number of concurrent functions a library can run.
| set_function_exec_mode_from_string (self, exec_mode)
| Set the execution mode of functions in a library.
| set_scheduler (self, scheduler)
| Set the worker selection scheduler for task.
| set_tag (self, tag)
| Attach a user defined logical name to the task.
| set_category (self, name)
| Label the task with the given category.
| add_feature (self, name)
| Label the task with the given user-defined feature.
| add_input (self, file, remote_name, strict_input=False, mount_symlink=False)
| Add any input object to a task.
| add_output (self, file, remote_name, watch=False, failure_only=None, success_only=None)
| Add any output object to a task.
| set_snapshot_file (self, filename)
| When monitoring, indicates a json-encoded file that instructs the monitor to take a snapshot of the task resources.
| add_starch_package (self, file)
| Add a Starch package as an execution context.
| add_poncho_package (self, file)
| Add a Poncho package as an execution context.
| add_execution_context (self, f)
| Adds an execution context to the task.
| set_retries (self, max_retries)
| Indicate the number of times the task should be retried.
| set_max_forsaken (self, max_forsaken)
| Indicate the number of times the task can be returned to the manager without being executed.
| set_cores (self, cores)
| Indicate the number of cores required by this task.
| set_memory (self, memory)
| Indicate the memory (in MB) required by this task.
| set_disk (self, disk)
| Indicate the disk space (in MB) required by this task.
| set_gpus (self, gpus)
| Indicate the number of GPUs required by this task.
| set_priority (self, priority)
| Indicate the the priority of this task (larger means better priority, default is 0).
| set_env_var (self, name, value=None)
| Set this environment variable before running the task.
| set_monitor_output (self, directory)
| Set a name for the resource summary output directory from the monitor.
| tag (self)
| Get the user-defined logical name for the task.
| category (self)
| Get the category name for the task.
| command (self)
| Get the shell command executed by the task.
| state (self)
| Get the state of the task.
| std_output (self)
| Get the standard output of the task.
| id (self)
| Get the task id number.
| exit_code (self)
| Get the exit code of the command executed by the task.
| result (self)
| Return a string that explains the result of a task.
| completed (self)
| Return True if task executed and its command terminated normally.
| successful (self)
| Return True if task executed successfully, (i.e.
| get_metric (self, name)
| Return various integer performance metrics about a completed task.
| addrport (self)
| Get the address and port of the host on which the task ran.
| hostname (self)
| Get the address and port of the host on which the task ran.
| resources_measured (self)
| Get the resources measured for the task execution if resource monitoring is enabled.
| limits_exceeded (self)
| Get the resources the task exceeded.
| resources_requested (self)
| Get the resources the task requested to run.
| resources_allocated (self)
| Get the resources allocated to the task in its latest attempt.
| add_nopen (self, manager)
| Adds inputs for nopen library and rules file and sets LD_PRELOAD.

## Additional Inherited Members

Protected Member Functions inherited from ndcctools.taskvine.task.Task
| _compute_library_name (self, library)
| Compute the name of a given library.

## Constructor & Destructor Documentation

## __init__()

ndcctools.taskvine.task.PythonTask.__init__ | ( |  | self,
|  |  | func,
|  | * | args,
|  | ** | kwargs )

Creates a new python task.

Parameters

  self | Reference to the current python task object
  func | python function to be executed by task
  args | arguments used in function to be executed by task
  kwargs | keyword arguments used in function to be executed by task

## Member Function Documentation

## submit_finalize()

ndcctools.taskvine.task.PythonTask.submit_finalize | ( |  | self | ) |

Finalizes the task definition once the manager that will execute is run.

This function is run by the manager before registering the task for execution.

Parameters

  self | Reference to the current python task object
  manager | Manager to which the task was submitted

Reimplemented from ndcctools.taskvine.task.Task.

Reimplemented in ndcctools.taskvine.futures.FutureFunctionCall, ndcctools.taskvine.futures.FuturePythonTask, and ndcctools.taskvine.task.FunctionCall.

## enable_temp_output()

ndcctools.taskvine.task.PythonTask.enable_temp_output | ( |  | self | ) |

Marks the output of this task to stay at the worker.

Functions that consume the output of this tasks have to add self.output_file as an input, and cloudpickle.load() it.

E.g.:

ta = PythonTask(fn, ...)
ta.enable_temp_output()
tid = m.submit(ta)
t = m.wait(...)
if t.id == tid:
tb = PythonTask(fn_with_tmp, "ta_output.file")
tb.add_input(ta.output_file, "ta_output.file")
m.submit(tb)
PythonTask
TaskVine PythonTask object.

where fn_with_tmp may look something like this:

def fn_with_tmp(filename):
import cloudpickle
with open(filename) as f:
data = cloudpickle.load(f)

Parameters

  self | Reference to the current python task object

## set_output_cache()

ndcctools.taskvine.task.PythonTask.set_output_cache | ( |  | self,
|  |  | cache = False )

Set the cache behavior for the output of the task.

Parameters

  cache | If True or 'workflow', cache the file at workers for reuse until the end of the workflow. If 'always', the file is cache until the end-of-life of the worker. Default is False (file is not cache).

## output_file()

ndcctools.taskvine.task.PythonTask.output_file | ( |  | self | ) |

Returns the ndcctools.taskvine.file.File object that represents the output of this task.

## output()

ndcctools.taskvine.task.PythonTask.output | ( |  | self | ) |

returns the result of a python task as a python variable

Parameters

  self | reference to the current python task object

Reimplemented from ndcctools.taskvine.task.Task.

Reimplemented in ndcctools.taskvine.futures.FutureFunctionCall, ndcctools.taskvine.futures.FuturePythonTask, and ndcctools.taskvine.task.FunctionCall.

## disable_output_serialization()

ndcctools.taskvine.task.PythonTask.disable_output_serialization | ( |  | self | ) |

Disables serialization of results to disk when writing to a file for transmission.

WARNING: Only do this if the function itself encodes the output in a way amenable for serialization.

Parameters

  self | Reference to the current python task object

The documentation for this class was generated from the following file:

- task.py