ndcctools.resource_monitor.Categories Class Reference

Class to encapsule all the categories in a workflow. More...

## Public Member Functions

| __init__ (self, all_categories_name='(all)')
| Create an empty set of categories.
| category_names (self)
| Returns a lists of the category categories.
| first_allocation (self, mode, category)
| Compute and return the first allocations for the given category.
| maximum_seen (self, category)
| Return the maximum resource values so far seen for the given category.
| accumulate_summary (self, summary)
| Add the summary (a dictionary) to the respective category.
| waste (self, category, field, allocation)
| Return the waste (unit x time) that would be produced if the accumulated summaries were run under the given allocation.
| wastepercentage (self, category, field, allocation)
| Return the percentage of wasted resources that would be produced if the accumulated summaries were run under the given allocation.
| throughput (self, category, field, allocation)
| Return the throughput that would be obtained if the accumulated summaries were run under the given allocation.
| retries (self, category, field, allocation)
| Return the number of tasks that would be retried if the accumulated summaries were run under the given allocation.
| count (self, category)
| Return the number of summaries in a particular category.

## Detailed Description

Class to encapsule all the categories in a workflow.

cs = Categories()
cs.accumulate_summary( { 'category': 'some_category', 'wall_time': 60, 'cores': 1, ... } )
print(cs.first_allocation(mode = 'throughput', category = 'some_category'))

## Constructor & Destructor Documentation

## __init__()

ndcctools.resource_monitor.Categories.__init__ | ( |  | self,
|  |  | all_categories_name = '(all)' )

Create an empty set of categories.

Parameters

  self | Reference to the current object.
  all_categories_name | Name of the general category that holds all of the summaries.

## Member Function Documentation

## category_names()

ndcctools.resource_monitor.Categories.category_names | ( |  | self | ) |

Returns a lists of the category categories.

List sorted lexicographicaly, with the exception of self.all_categories_name, which it is always the last entry.

Parameters

  self | Reference to the current object.

## first_allocation()

ndcctools.resource_monitor.Categories.first_allocation | ( |  | self,
|  |  | mode,
|  |  | category )

Compute and return the first allocations for the given category.

Note: wall_time needs to be defined in the resource summaries to be considered in this optimization.

Parameters

  self | Reference to the current object.
  mode | Optimization mode. One of 'throughput', 'waste', or 'fixed'.
  category | Name of the category

cs = Categories()
fa = cs.first_allocation(mode = 'throughput, category = 'some_category')
print(fa['cores'])
print(fa['memory'])
print(fa['disk'])

## maximum_seen()

ndcctools.resource_monitor.Categories.maximum_seen | ( |  | self,
|  |  | category )

Return the maximum resource values so far seen for the given category.

Parameters

  self | Reference to the current object.
  category | Name of the category

cs = Categories()
fa = cs.maximum_seen('some_category')
print(fa['cores'])
print(fa['memory'])
print(fa['disk'])

## accumulate_summary()

ndcctools.resource_monitor.Categories.accumulate_summary | ( |  | self,
|  |  | summary )

Add the summary (a dictionary) to the respective category.

At least both the 'category' and 'wall_time' keys should be defined.

cs = Categories()
cs.accumulate_summary( { 'category': 'some_category', 'wall_time': 50, 'cores': 1, ... } )

## waste()

ndcctools.resource_monitor.Categories.waste | ( |  | self,
|  |  | category,
|  |  | field,
|  |  | allocation )

Return the waste (unit x time) that would be produced if the accumulated summaries were run under the given allocation.

Parameters

  self | Reference to the current object.
  category | Name of the category
  field | Name of the resource (e.g., cores, memory, or disk)
  allocation | Value of allocation to test.

## wastepercentage()

ndcctools.resource_monitor.Categories.wastepercentage | ( |  | self,
|  |  | category,
|  |  | field,
|  |  | allocation )

Return the percentage of wasted resources that would be produced if the accumulated summaries were run under the given allocation.

Parameters

  self | Reference to the current object.
  category | Name of the category
  field | Name of the resource (e.g., cores, memory, or disk)
  allocation | Value of allocation to test.

## throughput()

ndcctools.resource_monitor.Categories.throughput | ( |  | self,
|  |  | category,
|  |  | field,
|  |  | allocation )

Return the throughput that would be obtained if the accumulated summaries were run under the given allocation.

Parameters

  self | Reference to the current object.
  category | Name of the category
  field | Name of the resource (e.g., cores, memory, or disk)
  allocation | Value of allocation to test.

## retries()

ndcctools.resource_monitor.Categories.retries | ( |  | self,
|  |  | category,
|  |  | field,
|  |  | allocation )

Return the number of tasks that would be retried if the accumulated summaries were run under the given allocation.

Parameters

  self | Reference to the current object.
  category | Name of the category
  field | Name of the resource (e.g., cores, memory, or disk)
  allocation | Value of allocation to test.

## count()

ndcctools.resource_monitor.Categories.count | ( |  | self,
|  |  | category )

Return the number of summaries in a particular category.

Parameters

  self | Reference to the current object.
  category | Name of the category

The documentation for this class was generated from the following file:

- resource_monitor.py