tf::DataPipeline
taskflow/algorithm/data_pipeline.hpp
tf::DataPipeline::Line
tf::DataPipeline::PipeMeta
typename... Ps Ps
unique_variant_t< std::variant< std::conditional_t< std::is_void_v< typename Ps::output_t >, std::monostate, std::decay_t< typename Ps::output_t > >... > > using tf::DataPipeline< Ps >::data_t = unique_variant_t<std::variant<std::conditional_t< std::is_void_v<typename Ps::output_t>, std::monostate, std::decay_t<typename Ps::output_t>>... >> data_t tf::DataPipeline::data_t

internal storage type for each data token (default std::variant)

Graph Graph tf::DataPipeline< Ps >::_graph _graph tf::DataPipeline::_graph
size_t size_t tf::DataPipeline< Ps >::_num_tokens _num_tokens tf::DataPipeline::_num_tokens
std::tuple< Ps... > std::tuple<Ps...> tf::DataPipeline< Ps >::_pipes _pipes tf::DataPipeline::_pipes
std::array< PipeMeta, sizeof...(Ps)> std::array<PipeMeta, sizeof...(Ps)> tf::DataPipeline< Ps >::_meta _meta tf::DataPipeline::_meta
std::vector< std::array< Line, sizeof...(Ps)> > std::vector<std::array<Line, sizeof...(Ps)> > tf::DataPipeline< Ps >::_lines _lines tf::DataPipeline::_lines
std::vector< Task > std::vector<Task> tf::DataPipeline< Ps >::_tasks _tasks tf::DataPipeline::_tasks
std::vector< Pipeflow > std::vector<Pipeflow> tf::DataPipeline< Ps >::_pipeflows _pipeflows tf::DataPipeline::_pipeflows
std::vector< CachelineAligned< data_t > > std::vector<CachelineAligned<data_t> > tf::DataPipeline< Ps >::_buffer _buffer tf::DataPipeline::_buffer
tf::DataPipeline< Ps >::DataPipeline
(size_t num_lines, Ps &&... ps)
DataPipeline
tf::DataPipeline::DataPipeline
size_t num_lines
Ps &&... ps

constructs a data-parallel pipeline object

num_lines

the number of parallel lines

ps

a list of pipes

Constructs a data-parallel pipeline of up to `num_lines` parallel lines to schedule tokens through the given linear chain of pipes. The first pipe must define a serial direction (tf::PipeType::SERIAL) or an exception will be thrown.
tf::DataPipeline< Ps >::DataPipeline
(size_t num_lines, std::tuple< Ps... > &&ps)
DataPipeline
tf::DataPipeline::DataPipeline
size_t num_lines
std::tuple< Ps... > && ps

constructs a data-parallel pipeline object

num_lines

the number of parallel lines

ps

a tuple of pipes

Constructs a data-parallel pipeline of up to `num_lines` parallel lines to schedule tokens through the given linear chain of pipes stored in a std::tuple. The first pipe must define a serial direction (tf::PipeType::SERIAL) or an exception will be thrown.
size_t size_t tf::DataPipeline< Ps >::num_lines () const noexcept num_lines tf::DataPipeline::num_lines

queries the number of parallel lines

The function returns the number of parallel lines given by the user upon the construction of the pipeline. The number of lines represents the maximum parallelism this pipeline can achieve.

size_t size_t tf::DataPipeline< Ps >::num_pipes () const noexcept num_pipes tf::DataPipeline::num_pipes

queries the number of pipes

The Function returns the number of pipes given by the user upon the construction of the pipeline.

void void tf::DataPipeline< Ps >::reset () reset tf::DataPipeline::reset

resets the pipeline

Resetting the pipeline to the initial state. After resetting a pipeline, its token identifier will start from zero as if the pipeline was just constructed.

size_t size_t tf::DataPipeline< Ps >::num_tokens () const noexcept num_tokens tf::DataPipeline::num_tokens

queries the number of generated tokens in the pipeline

The number represents the total scheduling tokens that has been generated by the pipeline so far.

Graph & Graph & tf::DataPipeline< Ps >::graph () graph tf::DataPipeline::graph

obtains the graph object associated with the pipeline construct

This method is primarily used as an opaque data structure for creating a module task of this pipeline.

size_t... I I
auto auto tf::DataPipeline< Ps >::_gen_meta (std::tuple< Ps... > &&, std::index_sequence< I... >) _gen_meta tf::DataPipeline::_gen_meta std::tuple< Ps... > && ps std::index_sequence< I... >
void void tf::DataPipeline< Ps >::_on_pipe (Pipeflow &, Runtime &) _on_pipe tf::DataPipeline::_on_pipe Pipeflow & pf Runtime &
void void tf::DataPipeline< Ps >::_build () _build tf::DataPipeline::_build

class to create a data-parallel pipeline scheduling framework

Ps

data pipe types

Similar to tf::Pipeline, a tf::DataPipeline is a composable graph object for users to create a data-parallel pipeline scheduling framework using a module task in a taskflow. The only difference is that tf::DataPipeline provides a data abstraction for users to quickly express dataflow in a pipeline. The following example creates a data-parallel pipeline of three stages that generate dataflow from `void` to `int`, `std::string`, and `void`.

    #include<taskflow/taskflow.hpp>
    #include<taskflow/algorithm/data_pipeline.hpp>

    intmain(){

    //dataflow=>void->int->std::string->void
    tf::Taskflowtaskflow("pipeline");
    tf::Executorexecutor;

    constsize_tnum_lines=4;

    tf::DataPipelinepl(num_lines,
    tf::make_data_pipe<void, int>(tf::PipeType::SERIAL,[&](tf::Pipeflow&pf)->int{
    if(pf.token()==5){
    pf.stop();
    return0;
    }
    else{
    returnpf.token();
    }
    }),
    tf::make_data_pipe<int, std::string>(tf::PipeType::SERIAL,[](int&input){
    returnstd::to_string(input+100);
    }),
    tf::make_data_pipe<std::string, void>(tf::PipeType::SERIAL,[](std::string&input){
    std::cout<<input<<std::endl;
    })
    );

    //buildthepipelinegraphusingcomposition
    taskflow.composed_of(pl).name("pipeline");

    //dumpthepipelinegraphstructure(withcomposition)
    taskflow.dump(std::cout);

    //runthepipeline
    executor.run(taskflow).wait();

    return0;
    }

The pipeline schedules five tokens over four parallel lines in a circular fashion, as depicted below:

    {.shell-session}
    o->o->o
    |||
    vvv
    o->o->o
    |||
    vvv
    o->o->o
    |||
    vvv
    o->o->o

- tf::DataPipeline
  _buffer
- tf::DataPipeline
  _build
- tf::DataPipeline
  _gen_meta
- tf::DataPipeline
  _graph
- tf::DataPipeline
  _lines
- tf::DataPipeline
  _meta
- tf::DataPipeline
  _num_tokens
- tf::DataPipeline
  _on_pipe
- tf::DataPipeline
  _pipeflows
- tf::DataPipeline
  _pipes
- tf::DataPipeline
  _tasks
- tf::DataPipeline
  data_t
- tf::DataPipeline
  DataPipeline
- tf::DataPipeline
  DataPipeline
- tf::DataPipeline
  graph
- tf::DataPipeline
  num_lines
- tf::DataPipeline
  num_pipes
- tf::DataPipeline
  num_tokens
- tf::DataPipeline
  reset