# parsl.executors.radical.RadicalPilotExecutor

class parsl.executors.radical.RadicalPilotExecutor(resource: str, label: str = RPEX, bulk_mode: bool = False, working_dir: str | None = None, rpex_cfg: ResourceConfig | None = None, **rpex_pilot_kwargs)
  Executor is designed for executing heterogeneous tasks

    in terms of type/resource.

  The RadicalPilotExecutor system has the following main components:

  > 1. “start”: Create and start the RADICAL-Pilot runtime components `rp.Session`,

  >      `rp.PilotManager` and `rp.TaskManager`.

  > 2. “translate”: Unwrap, identify, and parse Parsl `apps` into `rp.TaskDescription`.

  > 3. “submit”: Submit Parsl apps to `rp.TaskManager`.

  > 4. “shut_down”: Shut down the RADICAL-Pilot runtime and all associated components.

  Here is a diagram

      ----------------------------------------------------------------------------
               Parsl Data Flow Kernel        |   Task Translator |  rp.TaskManager
      ---------------------------------------|-------------------|----------------
                                             |                   |
      -> Dep. check ------> Parsl_tasks{} <--+--> Parsl Task     |
       Data management          +dfk.submit  |        |          |
                                             |        v          |
                                             |    RP Task(s) ->  | submit(task)
      ----------------------------------------------------------------------------

  The RadicalPilotExecutor creates a `rp.Session`, `rp.TaskManager`, and `rp.PilotManager`. The executor receives the Parsl apps from the DFK and translates these apps (in-memory) into `rp.TaskDescription` object to be passed to the `rp.TaskManager`. This executor has two submission mechanisms:

  1. Default_mode: where the executor submits the tasks directly to RADICAL-Pilot.

  2. Bulk_mode: where the executor accumulates N tasks (functions and executables) and submit them.

  Parameters:

    - rpex_cfg – a dataclass specifying resource configuration. Default is ResourceConfig instance.

    - label (str) – Label for this executor instance. Default is “RPEX”.

    - bulk_mode (bool) – Enable bulk mode submission and execution. Default is False (stream).

    - resource (Optional[str]) – The resource name of the targeted HPC machine or cluster. Default is local.localhost (user local machine).

    - runtime (int) – The maximum runtime for the entire job in minutes. Default is 30.

    - working_dir (str) – The working dir to be used by the executor.

    - rpex_pilot_kwargs (Dict of kwargs that are passed directly to the rp.PilotDescription object.) –

    - information (For more) –

  __init__(resource: str, label: str = RPEX, bulk_mode: bool = False, working_dir: str | None = None, rpex_cfg: ResourceConfig | None = None, **rpex_pilot_kwargs)

  Methods

  `__init__`(resource[, label, bulk_mode, ...]) |
  `create_monitoring_info`(status) | Create a monitoring message for each block based on the poll status.
  `monitor_resources`() | Should resource monitoring happen for tasks on running on this executor?
  `shutdown`([hub, targets, block]) | Shutdown the executor, including all RADICAL-Pilot components.
  `start`() | Create the Pilot component and pass it.
  `submit`(func, resource_specification, *args, ...) | Submits tasks in stream mode or bulks (bulk mode) to RADICAL-Pilot rp.TaskManager.
  `task_state_cb`(task, state) | Update the state of Parsl Future apps Based on RP task state callbacks.
  `task_translate`(tid, func, ...) | Convert Parsl function to RADICAL-Pilot rp.TaskDescription
  `unwrap`(func, args) | Unwrap a Parsl app and its args for further processing.

  Attributes

  `hub_address` | Address to the Hub for monitoring.
  `hub_port` | Port to the Hub for monitoring.
  `label` |
  `radio_mode` |
  `run_dir` | Path to the run directory.

  shutdown(hub=True, targets='all', block=False)

    Shutdown the executor, including all RADICAL-Pilot components.

  start()

    Create the Pilot component and pass it.

  submit(func, resource_specification, *args, **kwargs)

    Submits tasks in stream mode or bulks (bulk mode) to RADICAL-Pilot rp.TaskManager.

  task_state_cb(task, state)

    Update the state of Parsl Future apps Based on RP task state callbacks.

  task_translate(tid, func, parsl_resource_specification, args, kwargs)

    Convert Parsl function to RADICAL-Pilot rp.TaskDescription

  unwrap(func, args)

    Unwrap a Parsl app and its args for further processing.

    Parameters:

      - func (callable) – The function to be unwrapped.

      - args (tuple) – The arguments associated with the function.

    Returns:

      A tuple containing the unwrapped function, adjusted arguments, and task type information.

    Return type:

      tuple