edu.isi.pegasus.planner.partitioner

## Class Horizontal

- java.lang.Object
- - edu.isi.pegasus.planner.partitioner.Partitioner
  - - edu.isi.pegasus.planner.partitioner.BFS
    - - edu.isi.pegasus.planner.partitioner.Horizontal

    public class Horizontal
    extends BFS

Horizontal based partitioning scheme, that allows the user to configure the number of partitions per transformation name per level. To set the size of the partition per transformation, the following properties need to be set

           pegasus.partitioner.horizontal.collapse.[txName]
           pegasus.partitioner.horizontal.bundle.[txName]

The bundle value designates the number of partitions per transformation per level. The collapse values designates the number of nodes in a partitioning referring to a particular transformation. If both are specified, then bundle value takes precedence.

Version:
  $Revision$
Author:
  Karan Vahi

### Nested Class Summary

Nested Classes
`private static class` `Horizontal.GraphNodeComparator` A GraphNode comparator, that allows us to compare nodes according to the transformation logical names.

### Field Summary

Fields
`static int` `DEFAULT_COLLAPSE_FACTOR` The default collapse factor for collapsing jobs with same logical name scheduled onto the same execution pool.
`static java.lang.String` `DESCRIPTION` A short description about the partitioner.
`private int` `mIDCounter` The global counter that is used to assign ID's to the partitions.
`private Horizontal.GraphNodeComparator` `mNodeComparator` A static instance of GraphNode comparator.
`private java.util.Map` `mPartitionMap` A map indexed by the partition ID.

### Fields inherited from class edu.isi.pegasus.planner.partitioner.Partitioner

`mGraph, mLogger, mProps, mRoot, PACKAGE_NAME, VERSION`

### Constructor Summary

Constructors
Constructor and Description
`Horizontal(GraphNode root, java.util.Map graph, PegasusProperties properties)` The overloaded constructor.

### Method Summary

All Methods Instance Methods Concrete Methods
`protected void` `constructLevelRelations(Callback c, int parent, int child)` Calls out to the callback with appropriate relations between the partitions constructed for the levels.
`protected void` `constructPartitions(Callback c, java.util.List nodes, int level)` Given a list of jobs, constructs (one or more) partitions out of it.
`protected void` `constructPartitions(Callback c, java.util.List nodes, int level, java.lang.String name)` Given a list of jobs, constructs (one or more) partitions out of it.
`protected Partition` `createPartition(java.util.List nodes)` Creates a partition out of a list of nodes.
`java.lang.String` `description()` Returns a textual description of the partitioner implementation.
`protected void` `done(Callback c)` Indicates that we are done with the traversal of the graph.
`protected int[]` `getCollapseFactor(java.lang.String txName, int size)` Returns the collapse factor, that is used to determine the number of nodes going in a partition.
`private java.lang.String` `getPartitionID(int id)` Constructs the id for the partition.
`private int` `idCounter()` Returns the current value of the ID counter.
`private void` `incrementIDCounter()` Increments the ID counter by 1.
`private java.util.Comparator` `nodeComparator()` Singleton access to the job comparator.

### Methods inherited from class edu.isi.pegasus.planner.partitioner.BFS

`determinePartitions`

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Field Detail

#### DESCRIPTION

    public static final java.lang.String DESCRIPTION

A short description about the partitioner.

See Also:
  Constant Field Values

#### DEFAULT_COLLAPSE_FACTOR

    public static final int DEFAULT_COLLAPSE_FACTOR

The default collapse factor for collapsing jobs with same logical name scheduled onto the same execution pool.

See Also:
  Constant Field Values

#### mPartitionMap

    private java.util.Map mPartitionMap

A map indexed by the partition ID. Each value is a partition object.

#### mNodeComparator

    private Horizontal.GraphNodeComparator mNodeComparator

A static instance of GraphNode comparator.

#### mIDCounter

    private int mIDCounter

The global counter that is used to assign ID's to the partitions.

### Constructor Detail

#### Horizontal

    public Horizontal(GraphNode root,
                      java.util.Map graph,
                      PegasusProperties properties)

The overloaded constructor.

Parameters:
  `root` - the dummy root node of the graph.
  `graph` - the map containing all the nodes of the graph keyed by the logical id of the nodes.
  `properties` - the properties passed to the planner.

### Method Detail

#### nodeComparator

    private java.util.Comparator nodeComparator()

Singleton access to the job comparator.

Returns:
  the job comparator.

#### description

    public java.lang.String description()

Returns a textual description of the partitioner implementation.

Overrides:
  `description` in class `BFS`
Returns:
  a short textual description

#### constructPartitions

    protected void constructPartitions(Callback c,
                                       java.util.List nodes,
                                       int level)

Given a list of jobs, constructs (one or more) partitions out of it. Calls out to the partitioner callback, for each of the partitions constructed.

Overrides:
  `constructPartitions` in class `BFS`
Parameters:
  `c` - the parititoner callback
  `nodes` - the list of `GraphNode` objects on a particular level.
  `level` - the level as determined from the root of the workflow.

#### constructPartitions

    protected void constructPartitions(Callback c,
                                       java.util.List nodes,
                                       int level,
                                       java.lang.String name)

Given a list of jobs, constructs (one or more) partitions out of it. Calls out to the partitioner callback, for each of the partitions constructed.

Parameters:
  `c` - the parititoner callback
  `nodes` - the list of `GraphNode` objects on a particular level, referring to the same transformation underneath.
  `level` - the level as determined from the root of the workflow.
  `name` - the transformation name

#### constructLevelRelations

    protected void constructLevelRelations(Callback c,
                                           int parent,
                                           int child)

Calls out to the callback with appropriate relations between the partitions constructed for the levels. This is an empty implementation, as we do our own book-keeping in this partitioner to determine the relations between the partitions.

Overrides:
  `constructLevelRelations` in class `BFS`
Parameters:
  `c` - the parititoner callback
  `parent` - the parent level
  `child` - the child level.
See Also:
  `done( Callback )`

#### done

    protected void done(Callback c)

Indicates that we are done with the traversal of the graph. Determines the relations between the partitions constructed and calls out to the appropriate callback function

Overrides:
  `done` in class `BFS`
Parameters:
  `c` - the partitioner callback

#### getCollapseFactor

    protected int[] getCollapseFactor(java.lang.String txName,
                                      int size)

Returns the collapse factor, that is used to determine the number of nodes going in a partition. The collapse factor is determined by getting the collapse and the bundle values specified for the transformations in the properties file. There are two orthogonal notions of bundling and collapsing. In case the bundle key is specified, it ends up overriding the collapse key, and the bundle value is used to generate the collapse values. If both are not specified or null, then collapseFactor is set to size.

Parameters:
  `txName` - the logical transformation name
  `size` - the number of jobs that refer to the same logical transformation and are scheduled on the same execution pool.
Returns:
  int array of size 2 where :- int[0] is the the collapse factor (number of nodes in a partition) int[1] is the number of parititons for whom collapsing is int[0] + 1.

#### createPartition

    protected Partition createPartition(java.util.List nodes)

Creates a partition out of a list of nodes. Also stores it in the internal partition map to track partitions later on. Associates the partition ID with each of the nodes making the partition also.

Parameters:
  `nodes` - the list of `GraphNodes` making the partition.
Returns:
  the partition out of those nodes.

#### incrementIDCounter

    private void incrementIDCounter()

Increments the ID counter by 1.

#### idCounter

    private int idCounter()

Returns the current value of the ID counter.

#### getPartitionID

    private java.lang.String getPartitionID(int id)

Constructs the id for the partition.

Parameters:
  `id` - an integer ID.
Returns:
  the ID for the Partition.