# parsl.executors.taskvine.TaskVineExecutor

class parsl.executors.taskvine.TaskVineExecutor(label: str = 'TaskVineExecutor', worker_launch_method: Literal['provider', 'factory', 'manual'] = 'factory', function_exec_mode: Literal['regular', 'serverless'] = 'regular', manager_config: TaskVineManagerConfig = TaskVineManagerConfig(), factory_config: TaskVineFactoryConfig = TaskVineFactoryConfig(), provider: ExecutionProvider | None = None, storage_access: List[Staging] | None = None, remote_monitoring_radio: RadioConfig | None = None)

  Executor to use TaskVine dynamic workflow system

  The TaskVineExecutor system utilizes the TaskVine framework to efficiently delegate Parsl apps to remote machines in clusters and grids using a fault-tolerant system. Users can run the vine_worker program on remote machines to connect to the TaskVineExecutor, and Parsl apps will then be sent out to these machines for execution and retrieval.

  This Executor sets up configurations for the TaskVine manager, TaskVine factory, and run both in separate processes. Sending tasks and receiving results are done through multiprocessing module native to Python.

  Parameters:

    - label (str) – A human readable label for the executor, unique with respect to other executors. Default is “TaskVineExecutor”.

    - worker_launch_method (Union[Literal['provider'], Literal['factory'], Literal['manual']]) – Choose to use Parsl provider, TaskVine factory, or manual user-provided workers to scale workers. Options are among {‘provider’, ‘factory’, ‘manual’}. Default is ‘factory’.

    - function_exec_mode (Union[Literal['regular'], Literal['serverless']]) – Choose to execute functions with a regular fresh python process or a pre-warmed forked python process. Default is ‘regular’.

    - manager_config (TaskVineManagerConfig) – Configuration for the TaskVine manager. Default

    - factory_config (TaskVineFactoryConfig) – Configuration for the TaskVine factory. Use of factory is disabled by default.

    - provider (ExecutionProvider) – The Parsl provider that will spawn worker processes. Default to spawning one local vine worker process.

    - storage_access (List[Staging]) – Define Parsl file staging providers for this executor. Default is None.

  __init__(label: str = 'TaskVineExecutor', worker_launch_method: Literal['provider', 'factory', 'manual'] = 'factory', function_exec_mode: Literal['regular', 'serverless'] = 'regular', manager_config: TaskVineManagerConfig = TaskVineManagerConfig(), factory_config: TaskVineFactoryConfig = TaskVineFactoryConfig(), provider: ExecutionProvider | None = None, storage_access: List[Staging] | None = None, remote_monitoring_radio: RadioConfig | None = None)

  Methods

  `__init__`([label, worker_launch_method, ...]) |
  `create_monitoring_info`(status) | Create a monitoring message for each block based on the poll status.
  `handle_errors`(status) | This method is called by the error management infrastructure after a status poll.
  `initialize_scaling`() | Compose the launch command and call scale out
  `monitor_resources`() | Should resource monitoring happen for tasks on running on this executor?
  `outstanding`() | Count the number of outstanding tasks.
  `poll_facade`() |
  `scale_in`(blocks) | Scale in method.
  `scale_in_facade`(n[, max_idletime]) |
  `scale_out_facade`(n) | Scales out the number of blocks by "blocks"
  `send_monitoring_info`(status) |
  `set_bad_state_and_fail_all`(exception) | Allows external error handlers to mark this executor as irrecoverably bad and cause all tasks submitted to it now and in the future to fail.
  `shutdown`(*args, **kwargs) | Shutdown the executor.
  `start`() | Create submit process and collector thread to create, send, and retrieve Parsl tasks within the TaskVine system.
  `status`() | Return the status of all jobs/blocks currently known to this executor.
  `submit`(func, resource_specification, *args, ...) | Processes the Parsl app by its arguments and submits the function information to the task queue, to be executed using the TaskVine system.

  Attributes

  `bad_state_is_set` | Returns true if this executor is in an irrecoverable error state.
  `executor_exception` | Returns an exception that indicates why this executor is in an irrecoverable state.
  `label` |
  `provider` |
  `run_dir` | Path to the run directory.
  `run_id` | UUID for the enclosing DFK.
  `status_facade` | Return the status of all jobs/blocks of the executor of this poller.
  `status_polling_interval` | Returns the interval, in seconds, at which the status method should be called.
  `tasks` |
  `workers_per_node` |

  initialize_scaling()

    Compose the launch command and call scale out

    Scales the workers to the appropriate nodes with provider

  outstanding() → int

    Count the number of outstanding tasks.

  shutdown(*args, **kwargs)

    Shutdown the executor. Sets flag to cancel the submit process and collector thread, which shuts down the TaskVine system submission.

  start()

    Create submit process and collector thread to create, send, and retrieve Parsl tasks within the TaskVine system.

  submit(func, resource_specification, *args, **kwargs)

    Processes the Parsl app by its arguments and submits the function information to the task queue, to be executed using the TaskVine system. The args and kwargs are processed for input and output files to the Parsl app, so that the files are appropriately specified for the TaskVine task.

    Parameters:

      - func (function) – Parsl app to be submitted to the TaskVine system

      - resource_specification (dict) – Dictionary containing relevant info about task. Include information about resources of task, execution mode of task (out of {regular, serverless}).

      - args (list) – Arguments to the Parsl app

      - kwargs (dict) – Keyword arguments to the Parsl app

  property workers_per_node: int | float