# Ruffus Decorators

See also

Indicator objects

## Core

Decorator | Examples |
--- | --- | ---

@originate (Summary / Manual)

> - Creates (originates) a set of starting file without dependencies from scratch (ex nihilo!)
> - Only called to create files which do not exist.
> - Invoked onces (a job created) per item in the `output_files` list.

- @originate ( `output_files`, [`extra_parameters`,…] )

@split (Summary / Manual)

- Splits a single input into multiple output
- Globs in `output` can specify an indeterminate number of files.

- @split ( `tasks_or_file_names`, `output_files`, [`extra_parameters`,…] )

@transform (Summary / Manual)

- Applies the task function to transform input data to output.

- @transform ( `tasks_or_file_names`, suffix(`suffix_string`), `output_pattern`, [`extra_parameters`,…] )
- @transform ( `tasks_or_file_names`, regex(`regex_pattern`), `output_pattern`, [`extra_parameters`,…] )
- @transform ( `tasks_or_file_names`, formatter(`regex_pattern`), `output_pattern`, [`extra_parameters`,…] )

@merge (Summary / Manual)

- Merges multiple input files into a single output.

- @merge (`tasks_or_file_names`, `output`, [`extra_parameters`,…] )

## Combinatorics

Decorator | Examples |
--- | --- | ---

@product (Summary / Manual)

> - Generates the product, i.e. all vs all comparisons, between sets of input files.

- @product ( `tasks_or_file_names`, formatter ([ `regex_pattern` ]) ,*[* `tasks_or_file_names`, formatter ([ `regex_pattern` ]), ], `output_pattern`, [`extra_parameters`,…] )

@permutations (Summary / Manual)

> - Generates the permutations, between all the elements of a set of Input
> - Analogous to the python itertools.permutations
> - permutations(‘ABCD’, 2) –> AB AC AD BA BC BD CA CB CD DA DB DC

- @permutations ( `tasks_or_file_names`, formatter ([ `regex_pattern` ]), `tuple_size`, `output_pattern`, [`extra_parameters`,…] )

@combinations (Summary / Manual)

> - Generates the permutations, between all the elements of a set of Input
> - Analogous to the python itertools.combinations
> - combinations(‘ABCD’, 3) –> ABC ABD ACD BCD
> - Generates the combinations, between all the elements of a set of Input: i.e. r-length tuples of input elements with no repeated elements (A A) and where order of the tuples is irrelevant (either A B or B A, not both).

- @combinations ( `tasks_or_file_names`, formatter ([ `regex_pattern` ]), `tuple_size`, `output_pattern`, [`extra_parameters`,…] )

@combinations_with_replacement (Summary / Manual)

> - Generates the permutations, between all the elements of a set of Input
> - Analogous to the python itertools.permutations
> - combinations(‘ABCD’, 3) –> ABC ABD ACD BCD
> - Generates the combinations_with_replacement, between all the elements of a set of Input: i.e. r-length tuples of input elements with no repeated elements (A A) and where order of the tuples is irrelevant (either A B or B A, not both).

- @combinations_with_replacement ( `tasks_or_file_names`, formatter ([ `regex_pattern` ]), `tuple_size`, `output_pattern`, [`extra_parameters`,…] )

## Advanced

> Decorator | Examples |
> --- | --- | ---

> @subdivide (Summary / Manual) - Subdivides a set of Inputs each further into multiple Outputs. - The number of files in each Output can be set at runtime by the use of globs. - Many to Even More operator. - The use of split is a synonym for subdivide is deprecated.

> - @subdivide ( `tasks_or_file_names`, regex(`regex_pattern`), [ inputs | add_inputs(`input_pattern`), ] `output_pattern`, [`extra_parameters`,…] )
> - @subdivide ( `tasks_or_file_names`, formatter([`regex_pattern`] ), [ inputs | add_inputs(`input_pattern`), ] `output_pattern`, [`extra_parameters`,…] )

> @transform (Summary / Manual)
> - Infers input as well as output from regular expression substitutions
> - Useful for adding additional file dependencies

> - @transform ( `tasks_or_file_names`, regex(`regex_pattern`), [ inputs | add_inputs(`input_pattern`), ] `output_pattern`, [`extra_parameters`,…] )
> - @transform ( `tasks_or_file_names`, formatter(`regex_pattern`), [ inputs | add_inputs(`input_pattern`), ] `output_pattern`, [`extra_parameters`,…] )

> @collate (Summary / Manual)
> - Groups multiple input files using regular expression matching
> - Input resulting in the same output after substitution will be collated together.

> - @collate (`tasks_or_file_names`, regex(`regex_pattern`), `output_pattern`, [`extra_parameters`,…] )
> - @collate (`tasks_or_file_names`, regex(`regex_pattern`), inputs | add_inputs(`input_pattern`), `output_pattern`, [`extra_parameters`,…] )
> - @collate (`tasks_or_file_names`, formatter(`formatter_pattern`), `output_pattern`, [`extra_parameters`,…] )
> - @collate (`tasks_or_file_names`, formatter(`formatter_pattern`), inputs | add_inputs(`input_pattern`), `output_pattern`, [`extra_parameters`,…] )

> @follows (Summary / Manual)
> - Indicates task dependency
> - optional mkdir prerequisite (see Manual)

> - @follows ( `task1`, `'task2'` ))
> - @follows ( `task1`, mkdir( `'my/directory/'` ))

> @posttask (Summary / Manual)
> - Calls function after task completes
> - Optional touch_file indicator (Manual)

> - @posttask ( `signal_task_completion_function` )
> - @posttask (touch_file( `'task1.completed'` ))

> @active_if (Summary / Manual)

> > - Switches tasks on and off at run time depending on its parameters
> > - Evaluated each time pipeline_run(…), pipeline_printout(…) or pipeline_printout_graph(…) is called.
> > - Dormant tasks behave as if they are up to date and have no output.

> - @active_if ( `on_or_off1, [on_or_off2, ...]` )

> @jobs_limit (Summary / Manual)
> - Limits the amount of multiprocessing for the specified task
> - Ensures that fewer than N jobs for this task are run in parallel
> - Overrides `multiprocess` parameter in pipeline_run(…)

> - @jobs_limit ( `NUMBER_OF_JOBS_RUNNING_CONCURRENTLY` )

> @mkdir (Summary / Manual)
> - Generates paths for os.makedirs

> - @mkdir ( `tasks_or_file_names`, suffix(`suffix_string`), `output_pattern` )
> - @mkdir ( `tasks_or_file_names`, regex(`regex_pattern`), `output_pattern` )
> - @mkdir ( `tasks_or_file_names`, formatter(`regex_pattern`), `output_pattern`)

> @graphviz (Summary / Manual)
> - Customise the graphic for each task in printed flowcharts

> - @graphviz ( `graphviz_parameter = XXX`, `[graphviz_parameter2 = YYY ...]`)

## Esoteric!

> Decorator | Examples |
> --- | --- | ---

> @files (Summary / Manual)
> - I/O parameters
> - skips up-to-date jobs
> - Should use @transform etc instead

> - @files( `parameter_list` )
> - @files( `parameter_generating_function` )
> - @files ( `input_file`, `output_file`, `other_params`, … )

> @parallel (Summary / Manual)
> - By default, does not check if jobs are up to date
> - Best used in conjuction with @check_if_uptodate

> - @parallel ( `parameter_list` ) (see Manual)
> - @parallel ( `parameter_generating_function` ) (see Manual)

> @check_if_uptodate (Summary / Manual)
> - Custom function to determine if jobs need to be run

> - @check_if_uptodate ( `is_task_up_to_date_function` )

> Tip
> The use of this overly complicated function is discouraged.

>   @files_re (Summary)

>   - I/O file names via regular expressions
>   - start from lists of file names or glob results
>   - skips up-to-date jobs

> - @files_re ( `tasks_or_file_names`, `matching_regex`, [`input_pattern`,] `output_pattern`, `...` )
>     `input_pattern`/`output_pattern` are regex patterns used to create input/output file names from the starting list of either glob_str or file names