# Chapter 9: Preparing directories for output with @mkdir()

See also

- Manual Table of Contents
- @follows(mkdir()) syntax in detail
- @mkdir syntax in detail

Note

Remember to look at the example code:

> - Chapter 9: Python Code for Preparing directories for output with @mkdir()

## Overview

> In Chapter 3, we saw that we could use @follows(mkdir()) to ensure that output directories exist:

>     #
>     #   create_new_files() @follows mkdir
>     #
>     @follows(mkdir("output/results/here"))
>     @originate(["output/results/here/a.start_file",
>                 "output/results/here/b.start_file"])
>     def create_new_files(output_file_pair):
>         pass

> This ensures that the decorated task follows (@follows) the making of the specified directory (`mkdir()`).

> Sometimes, however, the Output is intended not for any single directory but a group of destinations depending on the parsed contents of Input paths.

## Creating directories after string substitution in a zoo…

> You may remember this example from Chapter 8:

> We want to feed the denizens of a zoo. The original file names are spread over several directories and we group their food supply by the clade of the animal in the following manner:

> >     #   Put different animals in different directories depending on their clade
> >     @transform(create_initial_files,                                       # Input

> >                formatter(".+/(?P<clade>\w+).(?P<tame>\w+).animals"),       # Only animals: ignore plants!

> >                "{subpath[0][1]}/{clade[0]}/{tame[0]}.{subdir[0][0]}.food", # Replacement

> >                "{subpath[0][1]}/{clade[0]}",                               # new_directory
> >                "{subdir[0][0]}",                                           # animal_name
> >                "{tame[0]}")                                                # tameness
> >     def feed(input_file, output_file, new_directory, animal_name, tameness):
> >         print "%40s -> %90s" % (input_file, output_file)
> >         # this blows up
> >         # open(output_file, "w")

> The example code from Chapter 8 is, however, incomplete. If we were to actually create the specified files we would realise that we had forgotten to create the destination directories `reptiles`, `mammals` first!

### using formatter()

> We could of course create directories manually. However, apart from being tedious and error prone, we have already gone to some lengths to parse out the diretories for @transform. Why don’t we use the same logic to make the directories?

> Can you see the parallels between the syntax for @mkdir and @transform?

> >     # create directories for each clade
> >     @mkdir(    create_initial_files,                                       # Input

> >                formatter(".+/(?P<clade>\w+).(?P<tame>\w+).animals"),       # Only animals: ignore plants!
> >                "{subpath[0][1]}/{clade[0]})                                # new_directory

> >     #   Put animals of each clade in the same directory
> >     @transform(create_initial_files,                                       # Input

> >                formatter(".+/(?P<clade>\w+).(?P<tame>\w+).animals"),       # Only animals: ignore plants!

> >                "{subpath[0][1]}/{clade[0]}/{tame[0]}.{subdir[0][0]}.food", # Replacement

> >                "{subpath[0][1]}/{clade[0]}",                               # new_directory
> >                "{subdir[0][0]}",                                           # animal_name
> >                "{tame[0]}")                                                # tameness
> >     def feed(input_file, output_file, new_directory, animal_name, tameness):
> >         print "%40s -> %90s" % (input_file, output_file)
> >         # this works now
> >         open(output_file, "w")

> See the example code

### using regex()

> If you are particularly fond of using regular expression to parse file paths, you could also use regex():

> >     # create directories for each clade
> >     @mkdir(    create_initial_files,                                       # Input

> >                regex(r"(.*?)/?(\w+)/(?P<clade>\w+).(?P<tame>\w+).animals"), # Only animals: ignore plants!
> >                r"\1/\g<clade>")                                             # new_directory

> >     #   Put animals of each clade in the same directory
> >     @transform(create_initial_files,                                       # Input

> >                formatter(".+/(?P<clade>\w+).(?P<tame>\w+).animals"),       # Only animals: ignore plants!

> >                "{subpath[0][1]}/{clade[0]}/{tame[0]}.{subdir[0][0]}.food", # Replacement

> >                "{subpath[0][1]}/{clade[0]}",                               # new_directory
> >                "{subdir[0][0]}",                                           # animal_name
> >                "{tame[0]}")                                                # tameness
> >     def feed(input_file, output_file, new_directory, animal_name, tameness):
> >         print "%40s -> %90s" % (input_file, output_file)
> >         # this works now
> >         open(output_file, "w")