Chapter 6. Dependencies
Prev |  | Next

## Chapter 6. Dependencies

So far we've seen how SCons handles one-time builds. But one of the main functions of a build tool like SCons is to rebuild only what is necessary when source files change--or, put another way, SCons should not waste time rebuilding things that don't need to be rebuilt. You can see this at work simply by re-invoking SCons after building our simple `hello` example:

    % scons -Q
    cc -o hello.o -c hello.c
    cc -o hello hello.o
    % scons -Q
    scons: `.' is up to date.

The second time it is executed, SCons realizes that the `hello` program is up-to-date with respect to the current `hello.c` source file, and avoids rebuilding it. You can see this more clearly by naming the `hello` program explicitly on the command line:

    % scons -Q hello
    cc -o hello.o -c hello.c
    cc -o hello hello.o
    % scons -Q hello
    scons: `hello' is up to date.

Note that SCons reports `"...is up to date"` only for target files named explicitly on the command line, to avoid cluttering the output.

## 6.1. Deciding When an Input File Has Changed: the `Decider` Function

Another aspect of avoiding unnecessary rebuilds is the fundamental build tool behavior of rebuilding things when an input file changes, so that the built software is up to date. By default, SCons keeps track of this through a content signature, or hash, of the contents of each file, although you can easily configure SCons to use the modification times (or time stamps) instead. You can even write your own Python function for deciding if an input file should trigger a rebuild.

### 6.1.1. Using Content Signatures to Decide if a File Has Changed

By default, SCons uses a cryptographic hash of the file's contents, not the file's modification time, to decide whether a file has changed. This means that you may be surprised by the default SCons behavior if you are used to the Make convention of forcing a rebuild by updating the file's modification time (using the touch command, for example):

    % scons -Q hello
    cc -o hello.o -c hello.c
    cc -o hello hello.o
    % touch hello.c
    % scons -Q hello
    scons: `hello' is up to date.

Even though the file's modification time has changed, SCons realizes that the contents of the `hello.c` file have not changed, and therefore that the `hello` program need not be rebuilt. This avoids unnecessary rebuilds when, for example, someone rewrites the contents of a file without making a change. But if the contents of the file really do change, then SCons detects the change and rebuilds the program as required:

    % scons -Q hello
    cc -o hello.o -c hello.c
    cc -o hello hello.o
    %     [CHANGE THE CONTENTS OF hello.c]
    % scons -Q hello
    cc -o hello.o -c hello.c
    cc -o hello hello.o

Note that you can, if you wish, specify the default behavior of using content signatures explicitly, using the `Decider` function as follows:

    Program('hello.c')
    Decider('content')

You can also use the string `'MD5'` as a synonym for `'content'` when calling the `Decider` function - this older name is deprecated since SCons now supports a choice of hash functions, not just the MD5 function.

#### 6.1.1.1. Ramifications of Using Content Signatures

Using content signatures to decide if an input file has changed has one surprising benefit: if a source file has been changed in such a way that the contents of the rebuilt target file(s) will be exactly the same as the last time the file was built, then any "downstream" target files that depend on the rebuilt-but-not-changed target file actually need not be rebuilt.

So if, for example, a user were to only change a comment in a `hello.c` file, then the rebuilt `hello.o` file would be exactly the same as the one previously built (assuming the compiler doesn't put any build-specific information in the object file). SCons would then realize that it would not need to rebuild the `hello` program as follows:

    % scons -Q hello
    cc -o hello.o -c hello.c
    cc -o hello hello.o
    %   [CHANGE A COMMENT IN hello.c]
    % scons -Q hello
    cc -o hello.o -c hello.c
    scons: `hello' is up to date.

In essence, SCons "short-circuits" any dependent builds when it realizes that a target file has been rebuilt to exactly the same file as the last build. This does take some extra processing time to read the contents of the target (`hello.o`) file, but often saves time when the rebuild that was avoided would have been time-consuming and expensive.

### 6.1.2. Using Time Stamps to Decide If a File Has Changed

If you prefer, you can configure SCons to use the modification time of a file, not the file contents, when deciding if a target needs to be rebuilt. SCons gives you two ways to use time stamps to decide if an input file has changed since the last time a target has been built.

The most familiar way to use time stamps is the way Make does: that is, have SCons decide that a target must be rebuilt if a source file's modification time is newer than the target file. To do this, call the `Decider` function as follows:

    Object('hello.c')
    Decider('timestamp-newer')

This makes SCons act like Make when a file's modification time is updated (using the touch command, for example):

    % scons -Q hello.o
    cc -o hello.o -c hello.c
    % touch hello.c
    % scons -Q hello.o
    cc -o hello.o -c hello.c

And, in fact, because this behavior is the same as the behavior of Make, you can also use the string `'make'` as a synonym for `'timestamp-newer'` when calling the `Decider` function:

    Object('hello.c')
    Decider('make')

One drawback to using times stamps exactly like Make is that if an input file's modification time suddenly becomes older than a target file, the target file will not be rebuilt. This can happen if an old copy of a source file is restored from a backup archive, for example. The contents of the restored file will likely be different than they were the last time a dependent target was built, but the target won't be rebuilt because the modification time of the source file is not newer than the target.

Because SCons actually stores information about the source files' time stamps whenever a target is built, it can handle this situation by checking for an exact match of the source file time stamp, instead of just whether or not the source file is newer than the target file. To do this, specify the argument `'timestamp-match'` when calling the `Decider` function:

    Object('hello.c')
    Decider('timestamp-match')

When configured this way, SCons will rebuild a target whenever a source file's modification time has changed. So if we use the `touch -t` option to change the modification time of `hello.c` to an old date (January 1, 1989), SCons will still rebuild the target file:

    % scons -Q hello.o
    cc -o hello.o -c hello.c
    % touch -t 198901010000 hello.c
    % scons -Q hello.o
    cc -o hello.o -c hello.c

In general, the only reason to prefer `timestamp-newer` instead of `timestamp-match`, would be if you have some specific reason to require this Make-like behavior of not rebuilding a target when an otherwise-modified source file is older.

### 6.1.3. Deciding If a File Has Changed Using Both MD Signatures and Time Stamps

As a performance enhancement, SCons provides a way to use a file's content signature, but to read those contents only when the file's timestamp has changed. To do this, call the `Decider` function with `'content-timestamp'` argument as follows:

    Program('hello.c')
    Decider('content-timestamp')

So configured, SCons will still behave like it does when using `Decider('content')`:

    % scons -Q hello
    cc -o hello.o -c hello.c
    cc -o hello hello.o
    % touch hello.c
    % scons -Q hello
    scons: `hello' is up to date.
    % edit hello.c
        [CHANGE THE CONTENTS OF hello.c]
    % scons -Q hello
    cc -o hello.o -c hello.c
    cc -o hello hello.o

However, the second call to SCons in the above output, when the build is up-to-date, will have been performed by simply looking at the modification time of the `hello.c` file, not by opening it and performing a signature calcuation on its contents. This can significantly speed up many up-to-date builds.

The only drawback to using `Decider('content-timestamp')` is that SCons will not rebuild a target file if a source file was modified within one second of the last time SCons built the file. While most developers are programming, this isn't a problem in practice, since it's unlikely that someone will have built and then thought quickly enough to make a substantive change to a source file within one second. Certain build scripts or continuous integration tools may, however, rely on the ability to apply changes to files automatically and then rebuild as quickly as possible, in which case use of `Decider('content-timestamp')` may not be appropriate.

### 6.1.4. Extending SCons: Writing Your Own Custom `Decider` Function

The different string values that we've passed to the `Decider` function are essentially used by SCons to pick one of several specific internal functions that implement various ways of deciding if a dependency (usually a source file) has changed since a target file has been built. As it turns out, you can also supply your own function to decide if a dependency has changed.

For example, suppose we have an input file that contains a lot of data, in some specific regular format, that is used to rebuild a lot of different target files, but each target file really only depends on one particular section of the input file. We'd like to have each target file depend on only its section of the input file. However, since the input file may contain a lot of data, we want to open the input file only if its timestamp has changed. This could be done with a custom `Decider` function that might look something like this:

    Program('hello.c')
    def decide_if_changed(dependency, target, prev_ni, repo_node=None):
        if dependency.get_timestamp() != prev_ni.timestamp:
            dep = str(dependency)
            tgt = str(target)
            if specific_part_of_file_has_changed(dep, tgt):
                return True
        return False
    Decider(decide_if_changed)

Note that in the function definition, the `dependency` (input file) is the first argument, and then the `target`. Both of these are passed to the functions as SCons `Node` objects, which we convert to strings using the Python `str()`.

The third argument, `prev_ni`, is an object that holds the content signature and/or timestamp information that was recorded about the dependency the last time the target was built. A `prev_ni` object can hold different information, depending on the type of thing that the `dependency` argument represents. For normal files, the `prev_ni` object has the following attributes:

`csig`

  The content signature: a cryptgraphic hash, or checksum, of the file contents of the `dependency` file the last time the `target` was built.

`size`

  The size in bytes of the `dependency` file the last time the target was built.

`timestamp`

  The modification time of the `dependency` file the last time the `target` was built.

These attributes may not be present at the time of the first run. Without any prior build, no targets have been created and no `.sconsign` DB file exists yet. So you should always check whether the `prev_ni` attribute in question is available (use the Python `hasattr` method or a `try`-`except` block).

The fourth argument `repo_node` is the `Node` to use if it is not None when comparing `BuildInfo`. This is typically only set when the target node only exists in a `Repository`

Note that ignoring some of the arguments in your custom `Decider` function is a perfectly normal thing to do, if they don't impact the way you want to decide if the dependency file has changed.

We finally present a small example for a `csig`-based decider function. Note how the signature information for the `dependency` file has to get initialized via `get_csig` during each function call (this is mandatory!).

    env = Environment()

    def config_file_decider(dependency, target, prev_ni, repo_node=None):
        import os.path

        # We always have to init the .csig value...
        dep_csig = dependency.get_csig()
        # .csig may not exist, because no target was built yet...
        if not prev_ni.hasattr("csig"):
            return True
        # Target file may not exist yet
        if not os.path.exists(str(target.abspath)):
            return True
        if dep_csig != prev_ni.csig:
            # Some change on source file => update installed one
            return True
        return False

    def update_file():
        with open("test.txt", "a") as f:
            f.write("some line\n")

    update_file()

    # Activate our own decider function
    env.Decider(config_file_decider)

    env.Install("install", "test.txt")

### 6.1.5. Mixing Different Ways of Deciding If a File Has Changed

The previous examples have all demonstrated calling the global `Decider` function to configure all dependency decisions that SCons makes. Sometimes, however, you want to be able to configure different decision-making for different targets. When that's necessary, you can use the `env.Decider` method to affect only the configuration decisions for targets built with a specific construction environment.

For example, if we arbitrarily want to build one program using content signatures and another using file modification times from the same source we might configure it this way:

    env1 = Environment(CPPPATH = ['.'])
    env2 = env1.Clone()
    env2.Decider('timestamp-match')
    env1.Program('prog-content', 'program1.c')
    env2.Program('prog-timestamp', 'program2.c')

If both of the programs include the same `inc.h` file, then updating the modification time of `inc.h` (using the touch command) will cause only `prog-timestamp` to be rebuilt:

    % scons -Q
    cc -o program1.o -c -I. program1.c
    cc -o prog-content program1.o
    cc -o program2.o -c -I. program2.c
    cc -o prog-timestamp program2.o
    % touch inc.h
    % scons -Q
    cc -o program2.o -c -I. program2.c
    cc -o prog-timestamp program2.o

Prev |  | Next
5.5. `GetBuildPath`: Getting the Path From a `Node` or String | Home | 6.2. Implicit Dependencies: The `$CPPPATH` Construction Variable