TaskParallelPipelineWithTokenDependencies

# Task-parallel Pipeline with Token Dependencies

Understand Token Dependencies

Understand Token Dependencies

TaskParallelPipelineWithTokenDependencies_1DeferredPipelineTokenDependencies
Resolve Token Dependencies

Resolve Token Dependencies

TaskParallelPipelineWithTokenDependencies_1DeferredPipelineResolveTokenDependencies
Include the Header

Include the Header

TaskParallelPipelineWithTokenDependencies_1DeferredPipelineIncludeHeaderFile
Create a Deferred Pipeline Module Task

Create a Deferred Pipeline Module Task

TaskParallelPipelineWithTokenDependencies_1CreateADeferredPipelineModuleTask
Create a Deferred Scalable Pipeline Module Task

Create a Deferred Scalable Pipeline Module Task

TaskParallelPipelineWithTokenDependencies_1CreateADeferredScalablePipelineModuleTask
Learn More about Taskflow Pipeline

Learn More about Taskflow Pipeline

TaskParallelPipelineWithTokenDependencies_1ParalleliDeferredScalablePipelineLearnMore

Taskflow pipeline allows you to defer the execution of a token to future tokens. This deferral introduces a dependency from a future token to the current token, particularly suitable for many video encoding applications. We recommend reading Task-parallel Pipeline first before learning this interface.

# Understand Token Dependencies

Token dependencies establish the order in which data tokens should execute in a task-parallel pipeline. When token `t1` completes before `t2` starts, there is a dependency from `t1` to `t2`. We categorize token dependencies into two types:

- forward token dependencies (FTD): dependencies from earlier to future tokens

- backward token dependencies (BTD): dependencies from future to earlier tokens The following figure illustrates a sample token dependency diagram and its token execution sequence. Edge pointing from token 2 to 5 is FTD, and those from 8 to 2 and 5 and 9 to 5 are BTDs. Based on the dependencies, the tokens execute in the corresponding execution sequence.

# Resolve Token Dependencies

To resolve the token dependencies, the basic idea is to defer the execution of a token with unresolved dependencies and save the token in a data structure until its dependencies are resolved. To implement the idea, we leverage three data structures, deferred_tokens (DT), token_dependencies (TD), and ready_tokens (RT). DT and TD are associative containers and RT is a queue. DT stores deferred tokens and their dependents by which the deferred tokens are deferred. TD stores a dependent and its related deferred tokens. RT stores the tokens that were deferred tokens and now are ready because their dependencies are resolved. The following image illustrates the usages of the three data structures to resolve the token dependencies and get the corresponding serial execution sequence exemplified in Understand Token Dependencies.

The whole process has the following steps:

1. Token 1 is not a deferred token and then 1 is finished. Now the execution sequence is {1}.

2. Token 2 defers to 8. We insert DT[2]={8} and TD[8]={2}. The black circle 2 in the above image illustrates this step.

3. Token 3 is not a deferred token and then 3 is finished. Now the execution sequence is {1,3}.

4. Token 4 is not a deferred token and then 4 is finished. Now the execution sequence is {1,3,4}.

5. Token 5 defers to 2 and 7. We insert DT[5]={2,7}, TD[2]={5}, and TD[7]={5}. The black circle 5 in the above image illustrates this step.

6. Token 6 is not a deferred token and then 6 is finished. Now the execution sequence is {1,3,4,6}.

7. Token 7 is not a deferred token and then 7 is finished. Now the execution sequence is {1,3,4,6,7}. Since TD[7]={5}, we directly remove 7 from DT[5]. The black circle 7 in the above image illustrates this step.

8. Token 8 is not a deferred token and then 8 is finished. Now the execution sequence is {1,3,4,6,7,8}. Since TD[8]={2}, we directly remove 8 from DT[2] and find out DT[2] is empty. Now token 2 is no longer a deferred token and we move 2 to RT. The black circle 8 in the above image illustrates this step.

9. RT is not empty and has a token 2. Then we finish running 2. Now the execution sequence is {1,3,4,6,7,8,2}. Since TD[2]={5}, we directly remove 2 from DT[5] and find out DT[5] is empty. Now token 5 is no longer a deferred token and we move 5 to RT. The black circle 9 in the above image illustrates this step.

10. RT is not empty and has a token 5. Then we run 5 and find out token 5 defers the second time, defers to 9. We insert DT[5]={9} and TD[9]={5}. The black circle 20 in the above image illustrates this step.

11. Token 9 is not a deferred token and then 9 is finished. Now the execution sequence is {1,3,4,6,7,8,2,9}. Since TD[9]={5}, we directly remove 9 from DT[5] and find out DT[5] is empty. Now token 5 is no longer a deferred token and we move 5 to RT. The black circle 11 in the above image illustrates this step.

12. RT is not empty and has a token 5. Then we finish running 5. Now the execution sequence is {1,3,4,6,7,8,2,9,5}. The black circle 12 in the above image illustrates this step.

13. Token 10 is not a deferred token and then 10 is finished. Now the execution sequence is {1,3,4,6,7,8,2,9,5,10}.

# Include the Header

You need to include the header file, `taskflow/algorithm/pipeline.hpp`, for implementing deferred pipeline algorithms.

    #include<taskflow/algorithm/pipeline.hpp>

# Create a Deferred Pipeline Module Task

To create a deferred pipeline application, there are four steps, one more step than creating a task-parallel pipeline (tf::Pipeline):

1. Define the pipeline structure (e.g., pipe type, pipe callable, stopping rule, line count)

2. Define the token dependencies at the first pipe

3. Define the data storage and layout, if needed for the application

4. Define the pipeline taskflow graph using composition

The following example demonstrates the creation of a deferred pipeline application exemplified in the dependency diagram in Understand Token Dependencies. The example creates a deferred pipeline that generates a total of 10 data tokens. The pipeline structure consists of four lines and three stages (all serial types).

    1:tf::Taskflowtaskflow("deferred_pipeline");
    2:tf::Executorexecutor;
    3:
    4:constsize_tnum_lines=4;
    5:
    6://thepipelineconsistsofthreepipes(serial-serial-serial)
    7://anduptofourconcurrentschedulingtokens
    8:tf::Pipelinepl(num_lines,
    9:tf::Pipe{tf::PipeType::SERIAL,[](tf::Pipeflow&pf){
    10://stopat11schedulingtokens
    11:if(pf.token()==11){
    12:pf.stop();
    13:}
    14:else{
    15://Token5isdeferred
    16:if(pf.token()==5){
    17:switch(pf.num_deferrals()){
    18:case0:
    19:pf.defer(2);
    20:printf("1st-time:Token%zuisdeferredby2\n",pf.token());
    21:pf.defer(7);
    22:printf("1st-time:Token%zuisdeferredby7\n",pf.token());
    23:return;
    24:break;
    25:
    26:case1:
    27:pf.defer(9);
    28:printf("2nd-time:Token%zuisdeferredby9\n",pf.token());
    29:return;
    30:break;
    31:
    32:case2:
    33:printf("3rd-time:Tokens2,7and9resolveddependencies\
    fortoken%zu\n",pf.token());
    34:break;
    35:}
    36:}
    37://token2isdeferred
    38:elseif(pf.token()==2){
    39:switch(pf.num_deferrals()){
    40:case0:
    41:pf.defer(8);
    42:printf("1st-time:Token%zuisdeferredby8\n",pf.token());
    43:break;
    44:case1:
    45:printf("2nd-time:Token8resolveddependenciesfortoken%zu\n",
    pf.token());
    46:break;
    47:}
    48:}
    49:else{
    50:printf("stage1:Non-deferredtoken%zu\n",pf.token());
    51:}
    52:}
    53:}},
    54:
    55:tf::Pipe{tf::PipeType::SERIAL,[](tf::Pipeflow&pf){
    56:printf("stage2:inputtoken%zu(deferrals=%zu)\n",
    pf.token(),pf.num_deferrals());
    57:}},
    58:
    59:tf::Pipe{tf::PipeType::SERIAL,[](tf::Pipeflow&pf){
    60:printf("stage3:inputtoken%zu\n",pf.token());
    61:}}
    62:);
    63:
    64:tf::Tasktask=taskflow.composed_of(pl);
    65:
    66://runthepipeline
    67:executor.run(taskflow).wait();

Debrief:

- Line 8 defines the number of lines in the pipeline

- Lines 9-52 define the first serial pipe, which will stop the pipeline scheduling at the 11th token

- Lines 15-30 define the token dependencies for token 5

- Lines 37-48 define the token dependencies for token 2

- Lines 55-57 define the second serial pipe

- Lines 59-61 define the third serial pipe

- Line 64 defines the pipeline taskflow graph using composition

- Line 67 executes the taskflow

The following is one of the possible outcomes of the example.

    stage1:Non-deferredtoken0
    stage2:inputtoken0(deferrals=0)
    stage3:inputtoken0
    stage1:Non-deferredtoken1
    stage2:inputtoken1(deferrals=0)
    stage3:inputtoken1
    1st-time:Token2isdeferredby8
    stage1:Non-deferredtoken3
    stage2:inputtoken3(deferrals=0)
    stage3:inputtoken3
    stage1:Non-deferredtoken4
    stage2:inputtoken4(deferrals=0)
    stage3:inputtoken4
    1st-time:Token5isdeferredby2
    1st-time:Token5isdeferredby7
    stage1:Non-deferredtoken6
    stage2:inputtoken6(deferrals=0)
    stage1:Non-deferredtoken7
    stage2:inputtoken7(deferrals=0)
    stage1:Non-deferredtoken8
    stage3:inputtoken6
    stage3:inputtoken7
    stage2:inputtoken8(deferrals=0)
    stage3:inputtoken8
    2nd-time:Token8resolveddependenciesfortoken2
    stage2:inputtoken2(deferrals=1)
    stage3:inputtoken2
    2nd-time:Token5isdeferredby9
    stage1:Non-deferredtoken9
    stage2:inputtoken9(deferrals=0)
    stage3:inputtoken9
    3rd-time:Tokens2,7and9resolveddependenciesfortoken5
    stage2:inputtoken5(deferrals=2)
    stage3:inputtoken5
    stage1:Non-deferredtoken10
    stage2:inputtoken10(deferrals=0)
    stage3:inputtoken10

You can only specify the token dependencies at the first pipe to get the serial execution of tokens.

# Create a Deferred Scalable Pipeline Module Task

In addition to task-parallel pipeline (tf::Pipeline), you can specify token dependencies on top of a task-parallel scalable pipeline (tf::ScalablePipeline). We recommend reading Task-parallel Scalable Pipeline first before learning this interface.

To create a deferred scalable pipeline application, there are four steps, which are identical to the steps described in Create a Deferred Pipeline Module Task. They are:

1. Define the pipeline structure (e.g., pipe type, pipe callable, stopping rule, line count)

2. Define the token dependencies at the first pipe

3. Define the data storage and layout, if needed for the application

4. Define the pipeline taskflow graph using composition

The following code creates a deferred scalable pipeline that uses four parallel lines to schedule tokens through two serial pipes in the given vector, then resetting that pipeline to three serial pipes. The three pipe callables are identical to the pipe callables demonstrated in the code snippet in Create a Deferred Pipeline Module Task. The token dependencies are exemplified in Understand Token Dependencies.

    1://createavectorofthreepipes
    2:std::vector<tf::Pipe<std::function<void(tf::Pipeflow&)>>>pipes;
    3:
    4://definepipecallables
    5://first_pipe_callableissameaslines15-53intheabovecodesnippet
    6:autofirst_pipe_callable=[](tf::Pipeflow&pf){
    7://stopat11schedulingtokens
    8:if(pf.token()==11){
    9:pf.stop();
    10:}
    11:else{
    12://Token5isdeferred
    13:if(pf.token()==5){
    14:switch(pf.num_deferrals()){
    15:case0:
    16:pf.defer(2);
    17:printf("1st-time:Token%zuisdeferredby2\n",pf.token());
    18:pf.defer(7);
    19:printf("1st-time:Token%zuisdeferredby7\n",pf.token());
    20:return;
    21:break;
    22:
    23:case1:
    24:pf.defer(9);
    25:printf("2nd-time:Token%zuisdeferredby9\n",pf.token());
    26:return;
    27:break;
    28:
    29:case2:
    30:printf("3rd-time:Tokens2,7and9resolveddependenciesfortoken%zu\n",
    pf.token());
    31:break;
    32:}
    33:}
    34://token2isdeferred
    35:elseif(pf.token()==2){
    36:switch(pf.num_deferrals()){
    37:case0:
    38:pf.defer(8);
    39:printf("1st-time:Token%zuisdeferredby8\n",pf.token());
    40:break;
    41:case1:
    42:printf("2nd-time:Token8resolveddependenciesfortoken%zu\n",
    pf.token());
    43:break;
    44:}
    45:}
    46:else{
    47:printf("stage1:Non-deferredtoken%zu\n",pf.token());
    48:}
    49:};
    50:
    51://second_pipe_callableissameaslines55-57intheabovecodesnippet
    52:autosecond_pipe_callable=[](tf::Pipeflow&pf){
    53:printf("stage2:inputtoken%zu(deferrals=%zu)\n",
    pf.token(),pf.num_deferrals());
    54:};
    55:
    56://third_pipe_callableissameaslines59-61intheabovecodesnippet
    57:autothird_pipe_callable=[](tf::Pipeflow&pf){
    58:printf("stage3:inputtoken%zu\n",pf.token());
    59:};
    60:
    61:pipes.emplace_back(tf::PipeType::SERIAL,first_pipe_callable);
    62:pipes.emplace_back(tf::PipeType::SERIAL,second_pipe_callable);
    63:
    64://createapipelineoffourparallellinesbasedonthegivenvectorofpipes
    65:tf::ScalablePipelinepl(4,pipes.begin(),pipes.end());
    66:
    67:tf::Tasktask=taskflow.composed_of(pl);
    68:
    69://runthepipeline
    70:executor.run(taskflow).wait();
    71:
    72://resetthepipelinetoanewrangeofthreepipesandstartsfrom
    73://theinitialstate(i.e.,tokencountsfromzero)
    74:pipes.emplace_back(tf::PipeType::SERIAL,third_pipe_callable);
    75:pl.reset(pipes.begin(),pipes.end());
    76:
    77:executor.run(taskflow).wait();

Debrief:

- Lines 2 defines the vector of tf::Pipe type

- Lines 6-49 define the first pipe callable

- Lines 52-54 define the second pipe callable

- Lines 57-59 define the third pipe callable

- Lines 61-62 define the vector of two pipe callables

- Line 65 defines the scalable pipeline

- Line 67 defines the pipeline taskflow graph using composition

- Line 70 executes the taskflow for the first run

- Line 74 inserts the third pipe callable in the vector

- Line 75 resets the pipes to three pipe callables

- Line 77 executes the taskflow for the second run

# Learn More about Taskflow Pipeline

Visit the following pages to learn more about pipeline:

- Task-parallel Pipeline

- Data-parallel Pipeline

- Task-parallel Scalable Pipeline

- Text Processing Pipeline

- Graph Processing Pipeline

- Taskflow Processing Pipeline