# Ctemplate System Reference Guide

(as of
)

## Overview

The main class used by the template system is `TemplateDictionary`, which is used to expand a template file. It is used by the main functions for expanding a template, found in `template.h`.

`TemplateCache` is used to hold a collection of `Template` objects. `TemplateNamelist` provides various introspection routines on collections of `Template` objects.

`TemplateModifier` and `PerExpandData` are used to modify the values of a `TemplateDictionary` at expand time. `TemplateAnnotator` does too, but is intended for debugging purposes. `TemplateDictionaryPeer` is used for testing template code.

`ExpandEmitter` provides the ability to emit an expanded template to an arbitrary output store.

`TemplateString` is a string-like class that is built to be very efficient when used with the template system. For instance, tools are available to hash constant `TemplateString` objects at compile-time, speeding up their use at run-time.

The rest of this document describes these classes and functions in more detail, as well as build tools and other mechanisms for handling templates.

(Note: the code snippets below all assume the default configuration option is set, which puts template code in namespace `ctemplate`.)

## Expanding Templates

### The Template Language

Templates are strings that contain special formatting code called template markers. In the Ctemplate System, template strings are usually read from a file.

Anything found in a template of the form {{...}} is interpreted as a template marker. All other text is considered formatting text and is output verbatim at template expansion time. Outside of the template markers, templates may contain any text whatsoever, including (single) curly braces and NUL characters.

The template language has six types of markers:

1. Variable markers, which are replaced by text based on dictionary values. Variable markers look like this: {{FOO}}
2. Start section and end section markers, which delimit sections which may appear zero, one, or N times in the output. Section markers look like this: `{{#FOO}}...{{/FOO}}`
3. Template-include markers, which designate other templates to be expanded and inserted at the location where the marker appears. These are treated much like sections -- one may think of them as sections whose content is specified in a different file instead of inline -- and just like sections, can be expanded zero, one or N times in the output, each with a different dictionary. Template-include markers look like this: `{{>FOO}}`
4. Comment markers, which may annotate the template structure but drop completely out of the expanded output. Comment markers look like this: `{{! comment lives here -- cool, no?}}`
5. Set-delimiter markers, which change the marker delimiters from `{{` and `}}` to custom strings. (The only requirement is that these strings not contain whitespace or the equals sign.) This is useful for languages like TeX, where double-braces may occur in the text and are awkward to use for markup. Set-delimiter markers look like this: `{{=< >=}} <! Now markers are delimited by braces > <=| |=> |! And now markers are delimited by bars! |`
6. Pragma markers, which invoke additional built-in template features when processing the template. Pragma markers look like this: `{{%PRAGMA [name="value"...]}}`. Currently, AUTOESCAPE is the only pragma defined.

These marker types each have their own namespace. For readability, however, it is best to not overuse a single name.

### Modifiers

A variable and include-template can have one or more modifiers attached to them, like so:

       {{MYVAR:mod1:mod2:mod3=value:mod4=value with spaces:mod5}}

A modifier is a filter that's applied at template-expand time, that munges the value of the variable before it's output. See the `TemplateModifier` class for more details.

When expanding a variable (or template-include) with a modifier, the modifiers are applied in order, left to right. For a template-include, first the entire sub-template is expanded, as a single string, and then the modifiers are applied to that string.

In general, using explicit modifiers does not turn off auto-escaping on a variable. The explicit modifier `:none` does suppress auto-escaping.

### Special Markers

Some marker names may have a special meaning in the template system. Right now, there's one such name.

#### Separator Sections

If you have a section named FOO, you can define inside of it a section named FOO_separator, and the template system will automatically expand that section every time `FOO` is expanded, except for the last. Thus, the contents of `FOO_separator` can be used to separate repeated values of a section.

Here's an example:

       Here are the meeting attendees:
       {{#ATTENDEES}}
          {{NAME}}
          {{#ATTENDEES_separator}}, {{/ATTENDEES_separator}}
       {{/ATTENDEES}}
       .

Here is a more convoluted example, to show the date:

       {{#DATE}}{{DATE_COMPONENT}}{{#DATE_separator}}{{DATE_SEP}}{{/DATE_separator}}{{/DATE}}

You'd set up a template dictionary to repeat `DATE` three times, with `DATE_COMPONENT` set to the month, day, and year (or day, month, and year, depending on your locale...), and `DATE_SEP` set to `/` or `-` or whatever date-separator is called for.

`SEP_separator` is always evaluated with the current dictionary. Thus, in the date example, if you wanted a different separator each time, you could do so by setting `DATE_SEP` to a different value for each repetition of `DATE`.

While `SEP_separator` is automatically expanded by the template system, it is otherwise a perfectly normal section. You can even instantiate it yourself by calling `AddSectionDictionary("SEP_separator")`. In that case, the section will be expanded both via the automatic expansion as a separator, and as a normal section via the section dictionary you added. This is more confusing than helpful, and you should probably never do it.

There can be at most one "separator" sub-section per section. If there are more, only the last is automatically expanded.

### Specifying a template

In the template system -- in functions like `ExpandTemplate()` -- a template is specified by a pair: filename + strip-mode. The filename specifies the name of the template file on disk (but see below for string templates). The strip mode specifies how this file should be parsed as it's read from disk, in particular, how whitespace should be handled. It can take the following values:

- `ctemplate::DO_NOT_STRIP`: do nothing. This expands the template file verbatim.
- `ctemplate::STRIP_BLANK_LINES`: remove all blank lines. This ignores any blank lines found in the template file when parsing it. When the template is html, this reduces the size of the output text without requiring a sacrifice of readability for the input file.
- `ctemplate::STRIP_WHITESPACE`: remove not only blank lines when parsing, but also whitespace at the beginning and end of each line. It also removes any linefeed (possibly following whitespace) that follows a closing `}}` of any kind of template marker except a template variable. (This means a linefeed may be removed anywhere by simply placing a comment marker as the last element on the line.) When the template is html, this reduces the size of the output html without changing the way it renders (except in a few special cases.) When using this flag, the built-in template variables `BI_NEWLINE` and `BI_SPACE` can be useful to force a space or newline in a particular situation.

The filename is where the template file lives on disk. It can be either an absolute filename, or relative to a directory in the template search path.

In addition to reading a template from disk, it is possible to read a template from a string, using `StringToTemplateCache()`. The first argument of `StringToTemplateCache()` is a "key" to use to refer to this string-based template. This key takes the place of the filename, for any routine that asks for a filename + strip.

### ExpandTemplate()

This is the main workhorse function of the template system. It takes the filename of a template, a template dictionary, and a string to emit to, and emits an "expanded" version of the template using the given template dictionary to fill in the template markers.

If the template specified by the given filename is already found in an internal cache, the cached version of the template is used, otherwise the template is loaded from disk, parsed, and stored in the cache before expansion is performed. As always, the "filename" can also be a key to a string-based template, inserted directly into the cache via `StringToTemplateCache()`.

There is an overloaded version of `Expand()` that takes an `ExpandEmitter` rather than a string, as the source to expand the template into.

This function returns true if the template was successfully expanded into the output parameter. It returns false if expansion failed or was only partially successful. It might fail because the template file cannot be found on disk, or because the template has syntax errors and cannot be parsed, or because the template sub-includes another template, and that sub-template has errors. To minimize the risk of errors at expand-time, you can call `LoadTemplate()` (below) first to load and parse the template into the cache. This will catch all errors except for errors involving sub-templates.

In the case of partial failures -- typically, failures resulting in an error in a sub-inclued template -- there may be partial data emitted to the output before the error was detected. If `ExpandTemplate()` returns false, you should be careful to check for and remove this partial data, if desired.

### ExpandWithData()

`ExpandWithData()` is like `ExpandTemplate()`, with the addition that it allows you to pass in per-expand data. It's called like this:

       ctemplate::TemplateDictionary dict(...);
       ctemplate::PerExpandData per_expand_data;
       string output;
       ctemplate::ExpandWithData(filename, strip_mode, &output, &dict,
                                 &per_expand_data);

Per-expand data is applied to all templates that are seen while expanding: not only the template you called `Expand()` on, but also sub-templates that are brought in via template-includes (`{{>INCLUDE}}`). See the description of the `PerExpandData` class for more details about how expansion can be modified by per-expand data.

The return value has the same meaning as for `ExpandTemplate()` In fact, if you pass in `NULL` as the `per_expand_data` argument, this function is exactly equivalent to `ExpandTemplate()`.

### LoadTemplate()

This function takes a filename and a strip-mode, and loads the file into the default template cache. Future calls to `ExpandTemplate()` or `ExpandWithData()` will get the parsed template from the cache, without needing to go to disk.

This function returns true if the template was successfully read from disk, parsed, and inserted into the cache. It will also return true if the template is already in the cache (even if the file has changed on disk since the template was inserted into the cache). It will return false if the file cannot be found on disk, or cannot be successfully parsed. (Note that `LoadTemplate()` cannot detect errors in sub-included templates, since the identity of sub-included templates is specified in a `TemplateDictionary`, not in the template itself.)

### StringToTemplateCache()

In addition to reading a template file from disk, it is also possible to read a template file from a string, using `StringToTemplateCache()`. It takes the content as a string. It also takes in a key and a strip mode. The given key and strip mode can be used as the filename/strip pair in calls to `ExpandTemplate()` and similar functions. The key can also be used as the "filename" in calls to `ctemplate::TemplateDictionary::SetFilename()`, allowing this template to be included inside other templates.

`StringToTemplateCache()` returns true if the string is successfully inserted into the cache. It returns false otherwise, probably because there is already a string or filename in the cache with the same key.

### default_template_cache() and mutable_default_template_cache()

All the above routines -- `ExpandTemplate()`, `LoadTemplate()`, and the like -- read and write parsed templates to the default template cache. This is just a static instance of the `TemplateCache` class. It can be accessed via these two functions: `default_template_cache()` and, if you need a non-const version of the cache, `mutable_default_template_cache()`. These can be useful if you need the advanced features of template caches, such as `ReloadAllIfChanged()` or `ClearCache()`.

## The `TemplateDictionary` Class

The class `TemplateDictionary` is used for all template dictionary operations. In general, an application will need to call a `TemplateDictionary` method for every marker in the associated template (the major exception: markers that evaluate to the empty string can be ignored).

The appropriate function to call for a given template marker depends on its type.

### Data Dictionaries

A data dictionary is a map from keys to values. The keys are always strings, corresponding to the name of the associated template marker, so a section `{{#FOO}}` in the template text is matched to the key `FOO` in the dictionary, if it exists. Note the case must match as well.

The value associated with a key differs according to key type. The value associated with a variable is simple: it's the value for that variable. Both keys and values can be any 8-bit character-string, and may include internal NULs (`\0`).

The value associated with a section is a list of data dictionaries. At template-expansion time, the section is expanded once for each dictionary in the list. The first time, all variables/etc. in the section will be evaluated taking into account the first dictionary. The second time, all variables/etc. will be evaluated taking into account the second dictionary. (See below for a definition of "taking into account.")

The value associated with a template-include is also a list of data dictionaries. Each data dictionary in this list must also have one other, mandatory associated piece of associated information: the filename of the template to include. At expand-time, that filename is passed as-is to `ctemplate::ExpandTemplate()` (or, more exactly, the equivalent of `ExpandTemplate()` in the same `TemplateCache` that the parent template comes from).

### Details on Dictionary Lookup

The dictionary structure is a tree: there's a 'main' dictionary, and then a list of sub-dictionaries for each section or include-template. When looking up a marker -- be it a variable, section, or include-template marker -- the system looks in the currently applicable dictionary. If it's found there, that value is used. If not, and the parent dictionary is not an include-template, it continues the look in the parent dictionary, and possibly the grandparent, etc. That is, lookup has static scoping: you look in your dictionary and any parent dictionary that is associated with the same template-file. As soon as continuing the lookup would require you to jump to a new template-file (which is what include-template would do), we stop the lookup.

If lookup fails in all dictionaries, the template system does a final lookup in the global variable dictionary.

The system initializes the global dictionary with a few useful values for your convenience (the user can add more). All system variables are prefixed with `BI`, to emphasize they are "built in" variables.

- `BI_SPACE`, which has the value `<space>`. It is used to force a space at the beginning or end of a line in the template, where it would normally be suppressed. (See below.)
- `BI_NEWLINE`, which has the value `<newline>` It is used to force a newline at the end of a line, where it would normally be suppressed. (See below.)

Variable inheritance happens at expand time, not at dictionary-create time. So if you create a section dictionary, and then afterwards set a variable in its parent dictionary (or in the global dictionary), the section will inherit that variable value, if it doesn't define the value itself.

### SetValue() and variants

SetValue() is used to set the value for a variable marker in a dictionary. It takes a string as input -- nominally a `TemplateString`, though a C++ string or C char* will be auto-converted -- for the variable name and its value. The name is the same string as is used for the variable's template-marker inside the template this dictionary will be expanded with: if the template reads `Hello {{NAME}}` then the first argument to `SetValue()` must be `NAME`. Case matters.

`SetIntValue()` takes an integer as the value, rather than a string. On all platforms, the integer may be up to 64 bits.

`SetFormattedValue()` is a convenience routine. `SetFormattedValue(key, arg1, arg2, ...)` is logically equivalent to

       char buffer[A_BIG_ENOUGH_NUMBER];
       sprintf(buffer, arg1, arg2, ...);
       SetValue(key, buffer);

without having to worry about the size of `buffer`, or about stack overflow.

`SetValueWithoutCopy()` is provided to give an extra bit of efficiency when needed. `SetValue()` will copy the key and value into an internal data structure used by the `TemplateDictionary`; this avoids dangling pointers if the arguments to `SetValue()` are temporaries, or otherwise have shorter lifetime than the `TemplateDictionary`. But if you know that both the key and value strings have lifetime longer than the `TemplateDictionary` -- perhaps because they are global (static duration) `char*`'s -- you can call `SetValueWIthoutCopy()` to avoid the copy. This yields a small time savings at the cost of significant code fragility, so should only be used when absolutely necessary.

### SetGlobalValue() and SetTemplateGlobalValue()

`SetGlobalValue()` is like `SetValue()` but works on the global dictionary. Since the global dictionary is shared across all template dictionaries, this is a static method on `TemplateDictionary`. It is thread-safe. It is also relatively slow.

`SetTemplateGlobalValue()` is like `SetValue()`, but the values are preserved across template-includes.

This example shows all three forms:

    A.tpl:
       {{NAME}} has won {{>PRIZE}}.  It is worth {{AMOUNT}}.
    B.tpl:
       {{AMOUNT}} dollars!  And it's all yours, {{NAME}}
    C.tpl:
       To: {{NAME}}.  Amount: {{AMOUNT}}.
    code:
       ctemplate::TemplateDictionary dict("set_value_demo");
       ctemplate::TemplateDictionary* subdict = dict.AddIncludeDictionary("PRIZE");
       subdict->SetFilename("B.tpl");
       dict->SetValue("NAME", "Jane McJane");
       dict->SetTemplateGlobalValue("AMOUNT", "One Million");
       ctemplate::TemplateDictionary::SetGlobalValue("NAME", "John Doe");
       ctemplate::TemplateDictionary dict_c("set_value_demo, part 2");

       ctemplate::ExpandTemplate("A.tpl", ..., &dict);
       ctemplate::ExpandTemplate("C.tpl", ..., &dict_c);

The first expand yields this:

       Jane McJane has won One Million dollars!  And it's all yours, John Doe.  It is worth One Million.

The second expand yields this:

       To: John Doe.  Amount: .

`NAME` was set via `SetValue()`, so its value was not propagated across the include-dict boundary; instead, the subdict (B.tpl) got its value for `NAME` from the global dictionary. `AMOUNT`, on the other hand, was set via `SetTemplateGlobalValue()`, so its value was propagated across the boundary, and the subdict saw it. However, the totally unrelated template, C.tpl, did not see either value, and only sees the values in the global dictionary. (Of course, had we filled `dict_c` with some values, C.tpl would have had access to those.)

### AddSectionDictionary(), ShowSection(), and SetValueAndShowSection()

`AddSectionDictionary(section_name)` returns a sub-dictionary associated with the given section_name (for instance, `dict.AddSectionDictionary("MYSECT")` for a template like `{{#MYSECT}}...{{/MYSECT}}`). If called multiple times, it will return a new dictionary each time. During `Expand()`, the section will be expanded once for each time `AddSectionDictionary()` was called; that is, the text inside the section delimiters will be repeated once for each `AddSectionDictionary()` call, and within a single repetition, the dictionary returned by `AddSectionDictionary()` will be used to populate the text inside the section delimiters. (With the current dictionary as that dictionary's parent, for inheritance purposes.)

This suggests that if `AddSectionDictionary()` is never called on a section, the text inside the section will be omitted entirely by `Expand()`.

`ShowSection()` is a convenience method used to show the text inside the section exactly once. It is equivalent to calling `AddSectionDictionary()` once, and ignoring the returned sub-dictionary. All variables in the section will depend on dictionary inheritence to get their values.

`SetValueAndShowSection()` is another convenience method,, used to show a section if and only if a related variable is set to a non-empty value. `SetValueAndShowSection(name, value, section_name)` is equivalent to this: if value is empty do nothing, otherwise add a single dictionary to `section_name` and call `section_dict->AddValue(name, value)`.

Example:

       ctemplate::TemplateDictionary* dict = new ctemplate::TemplateDictionary("section example");
       const char* username = GetUsername();   // returns "" for no user
       if (username[0] != '\0') {
          ctemplate::TemplateDictionary* sub_dict = dict->AddSectionDictionary("CHANGE_USER");
          sub_dict->SetValue("USERNAME", username);
       } else {
          // don't need to do anything; we want a hidden section, which is the default
       }

       // Instead of the above 'if' statement, we could have done this:
       if (username[0] != '\0') {
          dict->ShowSection("CHANGE_USER");       // adds a single, empty dictionary
          dict->SetValue("USERNAME", username);   // take advantage of inheritance
       } else {
          // don't need to do anything; we want a hidden section, which is the default
       }

       // Or we could have done this:
       dict->SetValueAndShowSection("USERNAME", username, "CHANGE_USER");

       // Moving on...
       GetPrevSearches(prev_searches, &num_prev_searches);
       if (num_prev_searches > 0) {
          for (int i = 0; i < num_prev_searches; ++i) {
             TemplateDictionary* sub_dict = dict->AddSectionDictionary("PREV_SEARCHES");
             sub_dict->SetValue("PREV_SEARCH", prev_searches[i]);
          }
       }

### AddIncludeDictionary() and SetFilename()

`AddIncludeDictionary(section_name)` returns a sub-dictionary associated with the given include_name (for instance, `dict.AddIncludeDictionary("MYTPL")` for a template like `{{>MYTPL}}`). If called multiple times, it will return a new dictionary each time. It is the responsibility of the caller to then call `SetFilename()` on the sub-dictionary returned.

During `Expand()`, each dictionary returned by `AddIncludeDictionary` will be examined in turn. For each dictionary for which `SetFilename()` is called, that filename will be read as a template (via `LoadTemplate()`, with the same "strip" value as the current template), and expanded using the dictionary. (Note that the dictionary will not inherit `SetValue()` values from the parent dictionary, though it will inherit `SetTemplateGlobalValue()` values.) This expanded template will then be emitted to the output.

Note that if `AddIncludeDictionary()` is never called, the template-include will be a no-op during `Expand()`. Likewise, if it is called but `SetFilename()` is never called on the resulting sub-dictionary, the template-include will be a no-op. On the other hand, if it is called multiple times, multiple templates -- possibly all from the same file, possibly not -- will be inserted at the point of the template-include.

If a user has called `StringToTemplateCache(key, ...)`, then the user can call `SetFilename(key)` to include the contents of the string as the sub-template.

### Dump() and DumpToString()

These routines dump the contents of a dictionary and its sub-dictionaries. `Dump()` dumps to stdout, `DumpToString()` to a string. They are intended for debugging.

## The `TemplateCache` Class

This class holds a collection of templates. It can be used to give you a coherent view of the template system: you load all the templates you are going to use into the cache, and then reload them from disk in one atomic operation. You can have multiple `TemplateCache` objects in your executable, perhaps one for each service you provide, or perhaps one per thread (so as to avoid thread contention when loading templates).

One intended use of the template cache is to make it safe to reload template files while serving user requests from a webserver. The idea is that every user request, as it's created, is associated with the current template cache. When you wish to reload, you `Clone()` the current cache, and then reload inside the cloned copy. New requests will get the cloned cache, while old requests will continue to render using the old cache. `TemplateCache` is written to make this use-case efficient: templates are shared between caches when appropriate, with reference-counting to keep memory management easy.

### The default template cache

The template system creates a default template cache, which is available via the functions `default_template_cache()` and `mutable_default_template_cache()`.

For simple uses of the template system, using the default cache, via `ExpandTemplate()` and the like, may be perfectly adequate. If you want more sophisticated features, such as the ability to have different versions of the same template file active at one time, or to change the template root-directory, you will have to use an explicit `TemplateCache`.

### LoadTemplate() and StringToTemplateCache()

These two routines are how you explicitly insert data into the cache. `LoadTemplate()` reads a template from disk, while `StringToTemplateCache()` takes the data from a user-specified string. These work exactly the same as the global `LoadTemplate()` and `StringToTemplateCache()`, except they insert into the given `TemplateCache`, rather than the global cache.

`LoadTemplate()` is not strictly necessary: if the cache cannot find a template it needs at `Expand*()` time, it will automatically try to fetch it from disk. It is intended mostly for use with `Freeze()`, which disables this auto-fetch behavior.

Both of these routines take a strip mode specifying how the template system should treat whitespace while parsing.

If the template-cache is frozen, `LoadTemplate()` will only return true if the template is already stored in the cache, and will return false in every other case. For a frozen cache, `StringToTemplateCache()` will always return false.

### ExpandWithData() and ExpandFrozen()

These routines takes a string that represents either a template filename (for disk-based templates), or a key used in `StringToTemplateCache()` (for string-based templates), a strip mode saying how to parse the template's whitespace, and per-expand data (which can be NULL). Overloads are provided to output the expanded template to either a string or to an arbitrary `ExpandEmitter`.

Expand uses the filename + strip pair to fetch the template from the cache, if it's there. If not, `ExpandWithData()` will fetch the template from disk and insert it into the cache. `ExpandFrozen()`, on the other hand, will just fail to expand, and return false. This is the only difference in behavior between the two routines. To reinforce this behavior, `ExpandFrozen()` will return false immediately if `Freeze()` has not been called on the cache.

While expanding, the template system may come across template sub-includes (`{{>SUB_TEMPLATE}}`). It will attempt to fetch these templates from the cache as well; failing that, it will try to load the template from disk (for `ExpandWithData()`) or else fail the expand and return false (for `ExpandFrozen()`).

Because `ExpandFrozen()` never updates the cache, it is a const method, unlike `ExpandWithData()`.

Note that `TemplateCache` does not provide the convenience `ExpandTemplate()` routine, as the analogue of the global `ExpandTemplate()`. If you are not interested in the per-expand data, just call `ExpandWithData()` with the per-expand data set to NULL.

### ReloadAllIfChanged()

For every file-based template in the cache (this method ignores string-based templates), `ReloadAllIfChanged()` checks the filesystem to see if the file has changed since it was loaded into the cache. If so, the cache loads a new version of the file into the cache. Note that at this point both the old and new versions of the file exist in the cache! Only the new version is accessible via new calls to `Expand()`, but any expansions currently in flight during the `ReloadAllIfChanged()` call will continue to use the old version.

NOTE: `ReloadAllIfChanged()` never modifies existing items in the cache in any way. It only loads new entries into the cache.

`ReloadAllIfChanged()` comes in two flavors, controlled by the passed-in enum. In "immediate" mode, it synchronously iterates through the cache and reloads all files that need it. In "lazy" mode, it waits to reload a given template file until the next time it's actually used (in a call to `Expand*()`). "Immediate" mode is safer in the case where templates depend on each other and the files change a lot, since all files are reloaded at about the same time. "Lazy" mode avoids the latency spike of "immediate" mode, and is preferable in the (common) case that files on disk change only rarely.

### SetTemplateRootDirectory(), AddAlternateTemplateRootDirectory(), and template_root_directory()

By default, filenames passed in to `Expand*()` are taken to be relative to the current working directory. These functions change that behavior. `SetTemplateRootDirectory()` resets the search path to be the single path passed in to this function. Every time `AddAlternateTemplateRootDirectory()` is called, it adds another directory to the end of the search path. The template system will follow the search path, in order, when looking for a filename.

Note that the template search path is only meaningful when the filename passed to `Expand*()` (or specified for a sub-include) is a relative filename. If the filename is an absolute filename, starting with `/`, the search path is ignored.

`template_root_directory()` returns the first entry in the search path. There is currently no way to access other entries in the search path.

### FindTemplateFilename()

`FindTemplateFilename()` does the work of finding a template file in the filesystem, using the current template search path. It takes a relative pathname as input, and returns an absolute pathname as output, indicating where the template file lives on the filesystem. If the template file is not found, this method returns the empty string.

### Freeze()

`Freeze()` marks the template cache as immutable. After this method is called, the cache can no longer be modified by loading new templates or reloading existing templates. During expansion only cached included templates will be used; they won't be loaded on-demand.

Before calling `Freeze()`, you should make sure your cache has all the templates it might need, using `LoadTemplate()` and `StringToTemplateCache()`. Otherwise, `ExpandWithData()` and `ExpandFrozen()` may fail.

Once the cache is frozen, calls to `SetTemplateRootDirectory()`, `AddAlternateTemplateRootDirectory()`, `Delete()`, and `ReloadAllIfChanged()` will fail.

After the cache is frozen, the `TemplateCache` object is effectively const.

### Delete()

This method deletes an entry from the cache. If the entry is in the cache multiple times (each with a different "strip" mode), this method deletes all of them.

### ClearCache()

This method deletes all entries from the cache. It can be called even on a frozen cache.

Note: this method is not typically necessary unless you are testing for memory leaks. It is intended to be called just before exiting the program, and after all template expansions are completed. Using it in that way may prevent unnecessary reporting by the leak checker.

### Clone()

`Clone()` makes a shallow copy of the given `TemplateCache` object, and returns the copy. The new copy and the old share the same template objects; since it's a shallow copy, they actually share pointers to the exact same object. (Since the template cache never changes a template object once it's loaded into the cache, sharing the pointer isn't a risk.)

The intended use of `Clone()`, as described above, is to allow fine control over "epochal" changes in templates. One `TemplateCache` can hold all versions of the template files before the big update; after the update is done, you can clone the old cache and then call `ReloadAllIfChanged()` on the clone. This is a low-cost operation, since the two copies share most resources. Smart pointers take care of most memory management issues.

The caller is responsible for calling `delete` on the returned `TemplateCache` object.

## The `TemplateNamelist` Class

This class provides information about registered templates. Or more precisely, all templates corresponding to registered filenames.

All methods in this class are static.

### RegisterTemplate() and RegisterTemplateFilename()

`TemplateNamelist::RegisterTemplate()` takes a filename and adds it to the static namelist used by `TemplateNamelist`. All other methods of `TemplateNamelist` work only on registered filenames.

It's fairly rare to call this method directly. Instead, the more common use is to use the macro `RegisterTemplateFilename` somewhere in the global scope, like so:

    RegisterTemplateFilename(EXAMPLE_FN, "example.tpl");

The reason to prefer the macro is it defines a global variable that you can use instead of the hard-coded template name. This helps catch typos. The `RegisterTemplateFilename()` example is functionally equivalent to:

    #define EXAMPLE_FN "example.tpl"

It can be used like this:

       ctemplate::ExpandTemplate(EXAMPLE_FN, ctemplate::DO_NOT_STRIP, ...);

### GetMissingList()

`TemplateNamelist::GetMissingList()` returns a list of all registered templates (or rather, all filenames) where the file could not be found on disk.

### AllDoExist()

Returns true if and only if the missing-list is empty.

### GetBadSyntax()

Returns a list of all registered templates where the template contains a syntax error, and thus cannot be used.

### IsAllSyntaxOkay()

True if and only if the bad-syntax list is empty.

### GetLastmodTime()

Returns the latest last-modified time for any registered template-file.

## The TemplateModifier Class and Associated Functions

A modifier is a filter that's applied at template-expand time, that munges the value of the variable before it's output. For instance, `<html><body>{{NAME:html_escape}}</body></html>` asks the template system to apply the built-in `html_escape` modifier when expanding `{{NAME}}`. If you set `NAME` in your dictionary to be `Jim & Bob`, what will actually be emitted in the template is `Jim &amp; Bob`.

You can chain modifiers together. This template first html-escapes `NAME`, and then javascript-escapes that result:

       <html><body>{{NAME:html_escape:javascript_escape}}</body></html>

Modifiers typically have a long, descriptive name and also a one-letter abbreviation. So this example is equivalent to the previous one:

       <html><body>{{NAME:h:j}}</body></html>

Some modifiers take an argument, specified after an equals sign:

       <html><body>{{NAME:html_escape_with_arg=pre}}</body></html>

### Built-in Modifiers

Here are the modifiers that are built in to the template system. They are all defined in template_modifiers.cc:

long name | short name | description
--- | --- | ---
`:cleanse_css` | `:c` | Removes characters not safe for a CSS value. Safe characters are alphanumeric, space, underscore, period, coma, exclamation mark, pound, percent, and dash.
`:html_escape` | `:h` | html-escapes the variable before output (eg `&` -> `&amp;`)
`:html_escape_with_arg` | `:H` | special purpose html escaping. See :H Arguments below for details.
`:img_src_url_escape_with_arg` | `:I` | special purpose image url escaping. See :I and :U Arguments below for details.
`:javascript_escape` | `:j` | javascript-escapes the variable before output (eg `"` -> `\x27` and `&` -> `\x26`)
`:javascript_escape_with_arg` | `:J` | special purpose javascript escaping. See :J Arguments below for details.
`:json_escape` | `:o` | json-escapes a variable before output as a string in json; HTML characters are escaped using Unicode escape sequences (e.g `&` -> `\u0026`) to comply with RFC 4627.
`:none` |  | leaves the variable as is (used to disable auto-escaping)
`:pre_escape` | `:p` | pre-escapes the variable before output (same as html_escape but whitespace is preserved; useful for <pre>...</pre>)
`:url_escape_with_arg` | `:U` | special purpose url escaping. See :I and :U Arguments below for details.

`:url_query_escape`

`:u`

performs URL escaping on the variable before output. space is turned into +, and everything other than [0-9a-zA-Z.,_:*/~!()-], is transformed into %-style escapes. Use this when you are building URLs with variables as parameters:

    <a href="http://google.com/search?q={{QUERY:u}}">{{QUERY:h}}</a>

`:xml_escape` |  | xml-escapes the variable before output (the five characters `<>&"'` become `&lt&gt;&amp;&quot;&#39;`) suitable for content returned in raw XML. It is not intended for escaping content within CDATA blocks.

The `*_with_arg` modifiers require an argument to specify the type of escaping to use. The following sections list the supported arguments for each of these modifiers.

#### :H Arguments

Here are the values that are supported by the `html_escape_with_arg` modifier:

value | description
--- | ---
`=snippet` | like `html_escape`, but allows HTML entities and some tags to pass through unchanged. The allowed tags are `<br>`, `<wbr>`, `<b>`, and `</b>`.
`=pre` | same as `pre_escape`
`=url` | same as `:U=html` below. For backwards compatibility.
`=attribute` | replaces characters not safe for an use in an unquoted attribute with underscore. Safe characters are alphanumeric, underscore, dash, period, and colon.

#### :I and :U Arguments

Here are the values that are supported by the `img_src_url_escape_with_arg` and `url_escape_with_arg` modifiers:

value | description
--- | ---

`=html`

Ensures that a variable contains a safe URL. Safe means that it is either a http or https URL, or else it has no protocol specified.

- If the URL is safe, the modifier HTML-escapes the URL.
- Otherwise, the modifier replaces the unsafe URL with one of the following values:
  - `/images/cleardot.gif` (the `:I` modifier)
  - `#` (the `:U` modifier)

Do not use `:U` for image URLs. Use `:I` instead. `#` is not a safe replacement for an image URL. `<img src=#>` can cause each browser to request the entire page again.

`=javascript` | Same as `=html`, but using javascript escaping instead of html escaping.
`=css` | Same as `=html` but using CSS escaping instead of html escaping so that the variable can be safely inserted within a CSS `@import` statement or a CSS property. The characters in [\r\n()'"<>*\] are transformed into %-style escapes.
`=query` | (Supported for `:U` only) Same as `url_query_escape`.

#### :J Arguments

Here are the values that are supported by the `javascript_escape_with_arg` modifier:

value | description
--- | ---

`=number`

Ensures that the variable is a valid number or boolean javascript literal. This includes booleans `true` and `false`, decimal numbers (e.g. `4.10` or `-5.01e+10`) as well as hex numbers (e.g. `0x5FF`). This modifier is intended to ensure the variable not enclosed in quotes cannot contain javascript code that may execute. It does not guarantee that the variable is syntactically well-formed. If the variable is safe, it is returned as-is, otherwise it is replaced with `null`. In the future we may add more logic to support objects and arrays.

### Custom Modifiers

In addition to the built-in modifiers, you can write your own modifier. Custom modifiers must have a name starting with "x-", and the name can contain alphanumeric characters plus dashes and underscores. Custom modifiers can also accept values with any character except for : and `}`. For example this template could be a valid use of a custom modifier:

    {{VAR:x-my_modifier:value1,value2,value3 has spaces,etc}}

See `<template_modifiers.h>` for details on how to write a modifier and how to register it. In short, you write a modifier by subclassing `ctemplate::TemplateModifier` and overriding the `Modify` method, and you register it by calling `ctemplate::AddModifier()` Here is an example of the code for a custom modifier:

       class StarEscape : public ctemplate::TemplateModifier {
         void Modify(const char* in, size_t inlen,
                     const ctemplate::PerExpandData* per_expand_data,
                     ctemplate::ExpandEmitter* outbuf, const string& arg) const {
           outbuf->Emit(string("*") + string(in, inlen) + string("*"));
         }
       };

### Subclassing TemplateModifier

The minimum work to create a custom modifier is to subclass `TemplateModifier` and override the `Modify()` method. `Modify()` takes as input the value of the text to be modified, as well as the per expand data associated with this `Expand()` call. The method may use this input, plus any other data it may have, to generate output, which it should write into the given `ExpandEmitter`.

The subclass can also override `MightModify()`. This is useful for modifiers that are typically a no-op (in which case the modifier is just doing busy-work, copying its input to the output `ExpandEmitter`). If `MightModify()` returns false, the template system will avoid calling `Modify()` at all on that variable, avoiding the busy-work copy.

### AddModifier()

AddModifier() is used to register a custom modifier, so the modifier can be used in templates. The name of the modifier must start with `x-`.

### AddXssSafeModifier()

`AddXssSafeModifier()` is used to register a custom modifier that can work well with the auto-escape system. It is used when the modifier produces output that is "safe" from cross-site scripting attacks in all contexts in which it might be used. For instance, a modifier that only emits numbers is xss-safe if it's only used in html or javascript contexts.

### Auto-escaping and Manual Escaping

If the auto-escape pragma is used in a document, then all variables will be auto-escaped, even if explicit modifiers are used on the variable. The rules are a little complicated:

- If the explicit modifier is a built-in modifier, and that modifier is "compatible" with the modifier that auto-escape would perform ("compatible" means it safely escapes in at least as many contexts as the auto-escape modifier), then only the explicit modifier is applied, and no further auto-escaping is done.
- If the explicit modifier is a custom modifier registered using AddXssSafeModifier(), no further auto-escaping is done.
- If the explicit modifier is the :none modifier, no further auto-escaping is done. This is the mechanism for manually turning off auto-escaping.
- In all other situations, auto-escaping will be performed after the explicit modifiers have all run.

## The `PerExpandData` Class

`ExpandWithData()` and `TemplateModifier::Modify()` both take a `PerExpandData` object. Per-expand data is applied to all templates that are seen while expanding: not only the template you called `ExpandWithData()` on, but also sub-templates that are brought in via template-includes (`{{>INCLUDE}}`).

There are several types of per-expand data you can set, by calling the appropriate method on a `PerExpandData` object.

### SetAnnotateOutput()

This is a debugging function. When expanding this template, it adds marker-strings to the output to indicate what template-substitutions the system made. This takes a string argument which can be used to shorten the filenames printed in the annotations: if the filename contains the string you give, everything before that string is elided from the filename before printing. (It's safe to just always pass in the empty string.)

### SetAnnotator()

This overrides the default text-based annotation used by `SetAnnotateOutput()`. If this is set and `SetAnnotateOutput()` is called, the per-expand data will use this `TemplateAnnotator` instance to do the annotation, rather than the default instance.

### InsertForModifiers() and LookupForModifiers()

`InsertForModifiers()` stores an arbitrary key/value pair in the `PerExpandData` structure. This is used with template modifiers: the `PerExpandData` object passed to `ExpandWithData()` is made available to every template modifier that is called during expand time (including any custom modifiers). The intended use of this functionality is to allow a modifier to work one way when expanding a template with dictionary A, and another way when expanding a template with dictionary B. For instance, a modifier might encrypt part of a webpage using a user's secret-key, which is of course different for every expansion of the webpage.

`LookupForModifiers()` can be used by a template-modifier to read the key/value pair inserted by `InsertForModifiers()`. `LookupForModifiersAsString()` is the same, but returns the value as a char* rather than a void*, for convenience.

### SetTemplateExpansionModifier()

This is an advanced feature for those who need a custom hook into template expansion. It will not be used by most programmers. It takes a template-modifier as its argument, but this template modifier is treated specially: instead of applying when an appropriate magic string appears in the text of a template, it applies every time a template is expanded during a call to `ExpandWithData()`. Note this means that it's called not only after the top-level template, that `ExpandWithData()` is called on, is expanded, but also on every sub-template that is expanded due to a template-include.

This unusual functionality has a few unusual properties. Since the template expansion modifier is not called in normal fashion, the normal `arg` argument to the template modifier does not make sense. Instead, the `arg` is set to the path of the template which is being expanded. Also, template expansion modifiers can be expensive, since they are applied pretty indiscriminately, so it can be worth implementing the `MightModifiy()` predicate for the passed-in `TemplateModifier` to avoid unnecessary work.

## The `TemplateAnnotator` Class

The `TemplateAnnotator` class is used to pass in to `PerExpandData::SetAnnotator()`, to control how expand-time annotation is done. This is meant to be used as a debugging routine.

This class is an abstract base class; `SetAnnotator()` takes a subclass that implements the various methods of the base class. These methods control the action taken when various template markers are seen during template expansion.

`template_annotator.h` defines the abstract base class, and also a concrete subclass that is used by default for annotation if `SetAnnotator()` is not called.

## The `TemplateDictionaryPeer` Class

By design, it is not possible to view the contents of a `TemplateDictionary`; we want to make sure the interface between the logic layer of the application and the presentation layer of the template goes in only one direction. While generally this keeps programs as clean as possible, it presents problems in testing code, which may want to verify that a given `TemplateDictionary` has been filled properly. The `TemplateDictionaryPeer` addresses this need. It lives in `template_test_util.h`.

Some of the methods of `TemplateDictionaryPeer` are useful for internal tests only. Below are some of the methods that are most useful for user-level tests.

### STS_INIT_FOR_TEST

This macro allows use of `STS_INIT` for testing, even when the input pointer is not guaranteed to be allocated for the entire length of the test (it must, of course, be allocated for the entire lifetime of the template being tested). Since normally `STS_INIT` requires inputs to have static duration, this allows for more flexibility in tests, at the cost of worse memory management and decreased code safety (in that it's possible to accidentally violate the lifetime requirements).

### GetSectionValue()

This returns the value for the named variable. It uses normal template scoping rules to resolve the name.

### IsHiddenSection() and IsHiddenTemplate()

Checks if a section or sub-template is "hidden" (that is, won't be displayed at all).

### GetSectionDictionaries() and GetIncludeDictionaries()

Returns all the sub-dictionaries for a given section or template-include, in a vector.

### Other Testing Functions

template_test_util.h has other useful routines for testing, such as ExpandIs(), which tests whether a template + dictionary pair expands into an expected string. See the header file for details.

## The `ExpandEmitter` Class

There are two overloads of `ExpandTemplate()`: the first emits the expanded template to a C++ string, and the second emits to an `ExpandEmitter`: an abstract base class that serves as a data source. It supports just one overloaded method, `Emit()`, that can take a char, a char*, or a C++ string as input.

Using this class requires subclassing `ExpandEmitter` and providing the various definitions for `Emit()`. For instance, a subclass might provide definitions of `Emit()` that send the input bytes out to a network socket.

In addition to defining the abstract base class, `template_emitter.h` provides a sample concrete subclass implementation, for emitting to a string.

## The `TemplateString` and `StaticTemplateString` Classes

`TemplateString` is a string-like implementation. Ctemplate uses `TemplateString` almost exclusively, internally. Many of the public API methods, such as `TemplateDictionary::SetValue()`, also take `TemplateString` as input. `TemplateString` has implicit constructors for both C++ strings and char*'s, so every method that takes a `TemplateString` will also work if given a C++ string or a char*. If you have a char* and know its length, you can save a bit of work by explicitly constructing a `TemplateString` with a char* + length, for instance:

       dict->SetValue(ctemplate::TemplateString("MYVAR", 5), value);

Some compile-time tools work with `TemplateString` to offload computation from runtime to compile-time. This is possible because the Ctemplate code often stores a hash of a string rather than a string directly. For static, immutable strings, `TemplateString` can store a pre-computed hash value. This functionality is used by `make_tpl_varnames_h`. Thus, using this tool to create constants to use for `SetValue()` keys provides not only protection against typos, but a speed improvement as well.

For immutable strings in your code, you can create efficient compile-time template-string objects of your own -- in this case of type `StaticTemplateString` -- by using the `STS_INIT` macro, like so:

    static const StaticTemplateString kSectName =
        STS_INIT(kSectName, "test_SetAddSectionDictionary");

The string variable's name, `kSectName` is repeated twice. The variable's value is specified inside the macro. Note that this macro should be used at the top level of a file, not inside functions (even when the variables are made static), and you should define the `StaticTemplateString` exactly as above: `static const StaticTemplateString`. Otherwise, the undefined constructor/destructor order of C++ may result in surprising behavior, wherein the `StaticTemplateString` is not initialized when it ought to be.

## The `Template` Class

In older version of ctemplate, the `Template` class, which holds parsed templates, was a major part of the template workflow: the common template use-case would be:

       Template* tpl = Template::GetTemplate(filename, strip_mode);
       TemplateDictionary dict(name);
       tpl->Expand(&dict, &outstring);

In current use, this model is deprecated in favor of the single `ExpandTemplate()` call; support for `Template` methods may be removed entirely in future versions of ctemplate. However, you may still find older code using this old formulation.

## Auto-Escaping

The Guide to using Auto Escape has an overview of Auto Escape as well as discussion of its limitations. To use it, put the following text at the top of the template:

      {{%AUTOESCAPE context="CONTEXT" [state="STATE"]}}

Description of the arguments:

- The context attribute is one of `HTML`, `JAVASCRIPT`, `CSS`, `JSON` or `XML`. It must correspond to the context in which the browser will interpret this template. Warning: Setting the wrong context will result in wrong escaping applied to all variables in the given template. In particular, if the template starts with a `<script>` tag, its context is `HTML` and not `JAVASCRIPT`. Auto-Escape will recognize the `<script>` tag and hence properly Javascript-escape the variables within it. Similarly, if the template starts with a `<style>` tag, its context is `HTML` and not `CSS`.
- The state attribute is commonly omitted. It accepts the value `IN_TAG` in the `HTML` context to indicate that the template only contains (one or more) HTML attribute name and value pairs that are part of an HTML tag formed in a parent template.

This will auto-escape every variable in the template. To turn off auto-escaping for a particular variable, you can apply the `none` modifier, like so: `{{MYVAR:none}}`. Here is an example of an autoescaped document:

      {{%AUTOESCAPE context="HTML"}}

      <body>
        <p>Hello {{USER}}</p>
        <p><a href="{{URL}}">Your Account</a></p>
        <p>Your identifier: {{ID:none}}{{! This is dangerous! }}</p>
      </body>

## Development Tools

This package includes several tools to make it easier to use write and use templates.

### make_tpl_varnames_h: Template Syntax Checker and Header File Generator

`make_tpl_varnames_h` is a "lint" style syntax checker and header file generator. It takes the names of template files as command line arguments and loads each file into a Template object by retrieving the file via the Template factory method. The loading of the file does pure syntax checking and reports such errors as mis-matched section start/end markers, mis-matched open/close double-curly braces, such as `"{{VAR}"`, or invalid characters in template variables/names/comments.

If the template passes the syntax check, by default the utility then creates a header file for use in the executable code that fills the dictionary for the template. If the developer includes this header file, then constants in the header file may be referenced in the dictionary building function, rather than hard-coding strings as variable and section names. By using these constants, the compiler can notify the developer of spelling errors and mismatched names. Here's an example of how this is used, and how it helps prevent errors:

       const char * const kosr_RESULT_NUMBER = "RESULT_NUMBER";  // script output
       dict.SetValue("RESSULT_NUMBER", "4");    // typo is silently missed
       dict.SetValue(kosr_RESSULT_NUMBER, "4");   // compiler catches typo

Each constant is named as follows:

- The initial letter 'k', indicating a defined constant.
- One or more prefix letters which are derived from the template file name. These prefix letters consist of the first letter of the file name, followed by the first letter following each underscore in the name, with the exception of the letter 'p' when it is followed by the letters "ost", as is a recommended convention for template versioning. For example, the prefix letters for the file `one_search_result_post20020815.tpl` are `osr`.
- An underscore.
- The variable or section name itself, same casing.

As an example, the section name "RESULT_NUMBER" in the file one_search_result_post20020815.tpl would be given the constant name `kosr_RESULT_NUMBER` and would appear in the header file as `const char * const kosr_RESULT_NUMBER = "RESULT_NUMBER";` -- as in the example above. (The variable is actually a `StaticTemplateString`, not a char*, but the basic idea holds.)

An alternate output directory may be specified by the command line flag `--header_dir`.

The name of the generated header file is the same as the name of the template file with an extension added to the name. By default, that extension is `.varnames.h`. In the above example, the header file containing the constant declarations would be named `one_search_result_post20020815.tpl.varnames.h`. An alternate extension may be provided via the command line flag `--outputfile_suffix`.

Important command line flags:

- `--noheader` -- Indicates that a header file should not be generated; only syntax checking should be done.
- `--header_dir` -- sets the directory where the header is written. Default: "./"
- `--template_dir` -- sets the template root directory. Default: `./` which is the correct specification when it is run from the directory where the templates are located. This is only used if the input template filenames are specified as relative paths rather than absolute paths.
- `--outputfile_suffix` -- the extension added to the name of the template file to create the name of the generated header file. Default: `.varnames.h`.

For a full list of command line flags, run `make_tpl_varnames_h --help`.

### template-converter: convert a template to a C++ string

`StringToTemplateCache()` lets you load a template from a string instead of a file. Applications may prefer this option to reduce the dependencies of the executable, or use it in environments where data files are not practical. In such cases, `template-converter` can be used as a template "compiler", letting the developer write a template file as a data file in the normal way, and then "compiling" it to a C++ string to be included in the executable.

Usage is `template-converter <template filename>`. C++ code is output is to stdout; it can be stored in a .h file or included directly into a C++ file. Perl must be installed to use this script.

- User's Guide
- Auto Escape
- Tips
- Example