# Using TaskFlow

## Considerations

Things to consider before (and during) development and integration with TaskFlow into your project:

- Read over the paradigm shifts and engage the team in IRC (or via the openstack-dev mailing list) if these need more explanation (prefix `[Oslo][TaskFlow]` to your emails subject to get an even faster response).

- Follow (or at least attempt to follow) some of the established best practices (feel free to add your own suggested best practices).

- Keep in touch with the team (see above); we are all friendly and enjoy knowing your use cases and learning how we can help make your lives easier by adding or adjusting functionality in this library.

## User Guide

- Atoms, tasks and retries
  - Atom
  - Task
  - Retry
  - Interfaces
  - Hierarchy
- Arguments and results
  - Arguments specification
  - Results specification
  - Revert arguments
  - Retry arguments
- Inputs and outputs
  - Flow inputs and outputs
  - Engine and storage
- Patterns
  - `Flow`
  - Linear flow
  - Unordered flow
  - Graph flow
  - Hierarchy
- Engines
  - Overview
  - Creating
  - Usage
  - Types
  - How they run
  - Special cases
  - Scoping
  - Interfaces
  - Implementations
  - Hierarchy
- Overview
  - Terminology
  - Requirements
- Design
  - High level architecture
  - Executor and worker communication
  - Request state transitions
- Usage
  - Workers
  - Engines
- Limitations
- Implementations
  - `WorkerBasedActionEngine`
  - Components
- Notifications and listeners
  - Overview
  - Receiving notifications with callbacks
  - Listeners
  - Interfaces
  - Implementations
  - Hierarchy
- Persistence
  - Overview
  - How it is used
  - Usage
  - Types
  - Interfaces
  - Models
  - Implementations
  - Storage
  - Hierarchy
- Resumption
  - Overview
  - Factories
  - Names
  - Scenarios
- Jobs
  - Overview
  - Definitions
  - High level architecture
  - Features
  - Usage
  - Types
  - Considerations
  - Interfaces
  - Implementations
  - Hierarchy
- Conductors
  - Overview
  - Considerations
  - Interfaces
  - Implementations
  - Hierarchy
- Examples
  - Hello world
  - Passing values from and to tasks
  - Using listeners
  - Using listeners (to watch a phone call)
  - Dumping a in-memory backend
  - Making phone calls
  - Making phone calls (automatically reverting)
  - Building a car
  - Iterating over the alphabet (using processes)
  - Watching execution timing
  - Distance calculator
  - Table multiplier (in parallel)
  - Linear equation solver (explicit dependencies)
  - Linear equation solver (inferred dependencies)
  - Linear equation solver (in parallel)
  - Creating a volume (in parallel)
  - Summation mapper(s) and reducer (in parallel)
  - Sharing a thread pool executor (in parallel)
  - Storing & emitting a bill
  - Suspending a workflow & resuming
  - Creating a virtual machine (resumable)
  - Creating a volume (resumable)
  - Running engines via iteration
  - Controlling retries using a retry controller
  - Distributed execution (simple)
  - Distributed notification (simple)
  - Distributed mandelbrot (complex)
  - Jobboard producer/consumer (simple)
  - Conductor simulating a CI pipeline
  - Conductor running 99 bottles of beer song requests

## Miscellaneous

- Exceptions
  - `raise_with_cause()`
  - `TaskFlowException`
  - `StorageFailure`
  - `ConductorFailure`
  - `JobFailure`
  - `UnclaimableJob`
  - `ExecutionFailure`
  - `RequestTimeout`
  - `InvalidState`
  - `DependencyFailure`
  - `AmbiguousDependency`
  - `MissingDependencies`
  - `CompilationFailure`
  - `IncompatibleVersion`
  - `Duplicate`
  - `NotFound`
  - `Empty`
  - `MultipleChoices`
  - `InvalidFormat`
  - `DisallowedAccess`
  - `NotImplementedError`
  - `WrappedFailure`
- States
  - Engine
  - Flow
  - Task
  - Retry
  - Jobs
- Types
  - Entity
  - Failure
  - Graph
  - Notifier
  - Sets
  - Timing
  - Tree
- Utilities
  - Async
  - Banner
  - Eventlet
  - Iterators
  - Kazoo
  - Kombu
  - Miscellaneous
  - Persistence
  - Redis
  - Schema
  - Threading

## Bookshelf

A useful collection of links, documents, papers, similar projects, frameworks and libraries.

Note

Please feel free to submit your own additions and/or changes.

- Libraries & frameworks
- Languages
- Services
- Papers
- Related paradigms

## Release notes

- CHANGES
  - 5.11.0
  - 5.10.0
  - 5.9.1
  - 5.9.0
  - 5.8.0
  - 5.7.0
  - 5.5.0
  - 5.4.0
  - 5.3.0
  - 5.2.0
  - 5.1.0
  - 5.0.0
  - 4.7.0
  - 4.6.4
  - 4.6.3
  - 4.6.2
  - 4.6.1
  - 4.6.0
  - 4.5.0
  - 4.4.0
  - 4.3.1
  - 4.3.0
  - 4.2.0
  - 4.1.0
  - 4.0.0
  - 3.8.0
  - 3.7.1
  - 3.7.0
  - 3.6.0
  - 3.4.0
  - 3.3.1
  - 3.2.0
  - 3.1.0
  - 3.0.1
  - 3.0.0
  - 2.17.0
  - 2.16.0
  - 2.15.0
  - 2.14.0
  - 2.13.0
  - 2.12.0
  - 2.11.0
  - 2.10.0
  - 2.9.0
  - 2.8.0
  - 2.7.0
  - 2.6.0
  - 2.5.0
  - 2.4.0
  - 2.3.0
  - 2.2.0
  - 2.1.0
  - 2.0.0
  - 1.32.0
  - 1.31.0
  - 1.30.0
  - 1.29.0
  - 1.28.0
  - 1.27.0
  - 1.26.0
  - 1.25.0
  - 1.24.0
  - 1.23.0
  - 1.22.0
  - 1.21.0
  - 1.20.0
  - 1.19.0
  - 1.18.0
  - 1.17.0
  - 1.16.0
  - 1.15.0
  - 1.14.0
  - 0.13.0
  - 0.12.0
  - 0.11.0
  - 0.10.1
  - 0.10.0
  - 0.9.0
  - 0.8.1
  - 0.8.0
  - 0.7.1
  - 0.7.0
  - 0.6.1
  - 0.6.0
  - 0.5.0
  - 0.4.0
  - 0.3.21
  - 0.3
  - 0.2
  - 0.1.3
  - 0.1.2
  - 0.1.1
  - 0.1

this page last updated: 2025-02-27 12:56:39

Except where otherwise noted, this document is licensed under Creative Commons Attribution 3.0 License. See all OpenStack Legal Documents.

found an error? report a bug