# loki.batch.scheduler module

class ProcessingStrategy(*values)

  Bases: `Enum`

  List of available processing types for `Scheduler.process`

  Multiple options exist how the `Scheduler.process` method can apply a provided `Transformation` or `Pipeline` object to the items in a `Scheduler` graph. The permissible options and default values are provided by this class.

  SEQUENCE = 1

    Sequential processing of transformations

    For each transformation in a pipeline, the `Transformation.apply` method is called for every item in the graph, following the graph traversal mode specified in the transformation’s manifest, before repeating the same for the next transformation in the pipeline.

  PLAN = 2

    Planning mode using `ProcessingStrategy.SEQUENCE` strategy.

    This calls `Transformation.plan` (instead of `Transformation.apply`) for each transformation.

  DEFAULT = 1

    Default processing strategy, currently `ProcessingStrategy.SEQUENCE`

class Scheduler(paths, config=None, seed_routines=None, preprocess=False, includes=None, defines=None, definitions=None, xmods=None, omni_includes=None, full_parse=True, frontend=Frontend.FP, output_dir=None)

  Bases: `object`

  Work queue manager to discover and capture dependencies for a given call tree, and apply transformations for batch processing

  Using a given list of `paths` and `seed_routines` (can be inferred from `config`), all scopes and symbols defined in the source tree are discovered and a dependency graph is created. The nodes of the dependency graph are `Item` objects, each corresponding to a specific Loki IR node.

  The dependency graph is re-generated on-the-fly and can be filtered for specific dependency classes during traversals (see `SFilter`). All items, are stored in the cache of the associated `item_factory`.

  Under the hood, the Scheduler is initialised in a three-stage procedure:

  1. A discovery step, where the minimum top-level definitions (modules and procedures) are determined for every source file in the path list.

  2. A populate step, which instantiates a first `SGraph` dependency graph by chasing dependencies starting from the provided seed nodes.

  3. Optionally, a full parse is triggered for all `Sourcefile` that appear in an `Item` in the dependency graph.

  This first two stages rely on an incremental, incomplete parsing of the source files that extract only the minimum set of symbols in each file. This is driven by the `Item._parser_class` and `Item._depends_class` attributes, which declare the minimum `RegexParserClass` classes to use with the `REGEX` frontend.

  To discover dependencies, the item’s IR is concretized. This calls `Scope.make_complete` with the minimum set of additional parser classes (as defined in the `_parser_class` attribute) that are required to discover the dependencies (e.g., calls, imports). When creating the corresponding dependency’s item, the defining scope’s item (e.g., a module containing a derived type declaration) is queried for its `definitions`, which in turn may trigger also a concretize step with the `_parser_class` of all item types that are listed in the scope item’s `defines_items` attribute.

  A `Transformation` can be applied across all nodes in the dependency graph using the `process()` method. The class properties in the transformation implementation (such as `Transformation.reverse_traversal`, `Transformation.traverse_file_graph` or `Transformation.item_filter`) determine, what nodes should be processed.

  config

    The config object describing the Scheduler’s behaviour

    Type:

      `SchedulerConfig`

  full_parse

    Flag to indicate a full parse of scheduler items

    Type:

      bool

  paths

    List of paths where sourcefiles are searched

    Type:

      list of `pathlib.Path`

  seeds

    Names of seed routines that are the root of dependency graphs

    Type:

      list of str

  build_args

    List of frontend arguments that are given to `Sourcefile.from_file` when performing a full parse

    Type:

      dict

  item_factory

    Instance of the factory class for `Item` creation and caching

    Type:

      `ItemFactory`

  Parameters:

    - paths (str or list of str) – List of paths to search for automated source file detection.

    - config (dict or str, optional) – Configuration dict or path to scheduler configuration file

    - seed_routines (list of str, optional) – Names of routines from which to populate the callgraph initially. If not provided, these will be inferred from the given config.

    - preprocess (bool, optional) – Flag to trigger CPP preprocessing (by default False).

    - includes (list of str, optional) – Include paths to pass to the C-preprocessor.

    - defines (list of str, optional) – Symbol definitions to pass to the C-preprocessor.

    - definitions (list of `Module`, optional) – `Module` object(s) that may supply external type or procedure definitions.

    - xmods (str, optional) – Path to directory to find and store `.xmod` files when using the OMNI frontend.

    - omni_includes (list of str, optional) – Additional include paths to pass to the preprocessor run as part of the OMNI frontend parse. If set, this replaces (!) `includes`, otherwise `omni_includes` defaults to the value of `includes`.

    - full_parse (bool, optional) – Flag indicating whether a full parse of all sourcefiles is required. By default a full parse is executed, use this flag to suppress.

    - frontend (`Frontend`, optional) – Frontend to use for full parse of source files (default `FP`).

    - output_dir (str or path) – Directory for the output to be written to

  source_suffixes = ['.f90', '.F90', '.f', '.F']
  propagate_and_separate_modes(proc_strategy=ProcessingStrategy.SEQUENCE)

    If in “MULTIPIPELINE” mode, different driver items can have different transformation pipelines assigned to them, this utility:

    - propagates the transformation pipeline modes to all descendants, where some could end up with multiple being assigned to them

    - seaparates the call tree via duplication to make sure every item in the tree is only assigned a single transformation pipeline to

    Parameters:

      proc_strategy (`ProcessingStrategy`) – The processing strategy to use when applying the given `transformation` to the scheduler’s graph.

  property sgraph

    Create and return the `SGraph` constructed from the `seeds` of the Scheduler.

  property items

    All `Item` objects contained in the `Scheduler` dependency graph.

  property dependencies

    All individual pairs of `Item` that represent a dependency and form an edge in the :any`Scheduler` call graph.

  property definitions

    The list of definitions that the source files in the callgraph provide

  property file_graph

    Alternative dependency graph based on relations between source files

    Returns:

      A dependency graph containing only `FileItem` nodes

    Return type:

      `SGraph`

  rekey_item_cache()

    Rebuild the mapping of item names to items in the `item_factory`’s cache

    This is required when a `Transformation` renames items during processing, and is triggered automatically at the end of the `process()` method if the transformation has `Transformation.renames_items` specified.

    This update also updates `config` entries that are affected by the renaming.

  process(transformation, proc_strategy=ProcessingStrategy.SEQUENCE)

    Process all `items` in the scheduler’s graph with either a `Pipeline` or a single `Transformation`.

    A single `Transformation` pass invokes `process_transformation()` individually, while a `Pipeline` will apply each contained transformation in turn over the full dependency graph of the scheduler.

    Parameters:

      - transformation (`Transformation` or `Pipeline` or dict of :any:`Pipeline`s) – The transformation or transformation pipeline to apply

      - proc_strategy (`ProcessingStrategy`) – The processing strategy to use when applying the given `transformation` to the scheduler’s graph.

  process_pipeline(pipeline, proc_strategy=ProcessingStrategy.SEQUENCE, mode=None)

    Process a given `Pipeline` by applying its associated transformations in turn.

    Parameters:

      - transformation (`Pipeline`) – The transformation pipeline to apply

      - proc_strategy (`ProcessingStrategy`) – The processing strategy to use when applying the given `pipeline` to the scheduler’s graph.

      - mode (str, optional) – Transformation mode, selecting which code transformations/pipeline on which graph to apply. Default: None, thus mode agnostic.

  process_transformation(transformation, proc_strategy=ProcessingStrategy.SEQUENCE, mode=None)

    Process all `items` in the scheduler’s graph

    By default, the traversal is performed in topological order, which ensures that an item is processed before the items it depends upon (e.g., via a procedure call) This order can be reversed in the `Transformation` manifest by setting `Transformation.reverse_traversal` to `True`.

    The scheduler applies the transformation to the program unit scope corresponding to each item in the scheduler’s graph, determined by the `Item.transformation_ir` property. For example, for a `ProcedureItem`, the transformation is applied to the corresponding `Subroutine` object.

    Optionally, the traversal can be performed on a source file level only, if the transformation has set `Transformation.traverse_file_graph` to `True`. This uses the `filegraph` to process the dependency tree. If combined with a `Transformation.item_filter`, only source files with at least one object corresponding to an item of that type are processed.

    Parameters:

      - transformation (`Transformation`) – The transformation to apply over the dependency tree

      - proc_strategy (`ProcessingStrategy`) – The processing strategy to use when applying the given `transformation` to the scheduler’s graph.

      - mode (str, optional) – Transformation mode, selecting which code transformations on which graph to apply. Default: None, thus mode agnostic.

  callgraph(path, with_file_graph=False, with_legend=False)

    Generate a callgraph visualization and dump to file.

    Parameters:

      - path (str or pathlib.Path) – Path to write the callgraph figure to.

      - with_filegraph (bool or str or pathlib.Path) – Visualize file dependencies in an additional file. Can be set to True or a file path to write to.

      - with_legend (bool) – Add a key/legend to the plot. Can be enabled by setting the argument to True.

  write_cmake_plan(filepath, rootpath=None)

    Generate the “plan file” for CMake

    See `CMakePlanTransformation` for the specification of that file.

    Parameters:

      - filepath (str or Path) – The path of the CMake file to write.

      - rootpath (str or Path (optional)) – If given, all paths in the CMake file will be made relative to this root directory