# Extending the parser

Modules such as `page3` extend the CSS 2.1 parser to add support for CSS 3 syntax. They do so by sub-classing `css21.CSS21Parser` and overriding/extending some of its methods. If fact, the parser is made of methods in a class (rather than a set of functions) solely to enable this kind of sub-classing.

tinycss is designed to enable you to have parser subclasses outside of tinycss, without monkey-patching. If however the syntax you added is for a W3C specification, consider including your subclass in a new tinycss module and send a pull request: see Hacking tinycss.

## Example: star hack

The star hack uses invalid declarations that are only parsed by some versions of Internet Explorer. By default, tinycss ignores invalid declarations and logs an error.

    >>> from tinycss.css21 import CSS21Parser
    >>> css = '#elem { width: [W3C Model Width]; *width: [BorderBox Model]; }'
    >>> stylesheet = CSS21Parser().parse_stylesheet(css)
    >>> stylesheet.errors
    [ParseError('Parse error at 1:35, expected a property name, got DELIM',)]
    >>> [decl.name for decl in stylesheet.rules[0].declarations]
    ['width']

If for example a minifier based on tinycss wants to support the star hack, it can by extending the parser:

    >>> class CSSStarHackParser(CSS21Parser):
    ...     def parse_declaration(self, tokens):
    ...         has_star_hack = (tokens[0].type == 'DELIM' and tokens[0].value == '*')
    ...         if has_star_hack:
    ...             tokens = tokens[1:]
    ...         declaration = super(CSSStarHackParser, self).parse_declaration(tokens)
    ...         declaration.has_star_hack = has_star_hack
    ...         return declaration
    ...
    >>> stylesheet = CSSStarHackParser().parse_stylesheet(css)
    >>> stylesheet.errors
    []
    >>> [(d.name, d.has_star_hack) for d in stylesheet.rules[0].declarations]
    [('width', False), ('width', True)]

This class extends the `parse_declaration()` method. It removes any `*` delimeter `Token` at the start of a declaration, and adds a `has_star_hack` boolean attribute on parsed `Declaration` objects: `True` if a `*` was removed, `False` for “normal” declarations.

## Parser methods

In addition to methods of the user API (see Parsing a stylesheet), here are the methods of the CSS 2.1 parser that can be overriden or extended:

CSS21Parser.parse_rules(tokens, context)

  Parse a sequence of rules (rulesets and at-rules).

  Parameters

    - tokens – An iterable of tokens.

    - context – Either `'stylesheet'` or an at-keyword such as `'@media'`. (Most at-rules are only allowed in some contexts.)

  Returns

    A tuple of a list of parsed rules and a list of `ParseError`.

CSS21Parser.read_at_rule(at_keyword_token, tokens)

  Read an at-rule from a token stream.

  Parameters

    - at_keyword_token – The ATKEYWORD token that starts this at-rule You may have read it already to distinguish the rule from a ruleset.

    - tokens – An iterator of subsequent tokens. Will be consumed just enough for one at-rule.

  Returns

    An unparsed `AtRule`.

  Raises

    `ParseError` if the head is invalid for the core grammar. The body is not validated. See `AtRule`.

CSS21Parser.parse_at_rule(rule, previous_rules, errors, context)

  Parse an at-rule.

  Subclasses that override this method must use `super()` and pass its return value for at-rules they do not know.

  In CSS 2.1, this method handles @charset, @import, @media and @page rules.

  Parameters

    - rule – An unparsed `AtRule`.

    - previous_rules – The list of at-rules and rulesets that have been parsed so far in this context. This list can be used to decide if the current rule is valid. (For example, @import rules are only allowed before anything but a @charset rule.)

    - context – Either `'stylesheet'` or an at-keyword such as `'@media'`. (Most at-rules are only allowed in some contexts.)

  Raises

    `ParseError` if the rule is invalid.

  Returns

    A parsed at-rule

CSS21Parser.parse_media(tokens)

  For CSS 2.1, parse a list of media types.

  Media Queries are expected to override this.

  Parameters

    tokens – A list of tokens

  Raises

    `ParseError` on invalid media types/queries

  Returns

    For CSS 2.1, a list of media types as strings

CSS21Parser.parse_page_selector(tokens)

  Parse an @page selector.

  Parameters

    tokens – An iterable of token, typically from the `head` attribute of an unparsed `AtRule`.

  Returns

    A page selector. For CSS 2.1, this is `'first'`, `'left'`, `'right'` or `None`.

  Raises

    `ParseError` on invalid selectors

CSS21Parser.parse_declarations_and_at_rules(tokens, context)

  Parse a mixed list of declarations and at rules, as found eg. in the body of an @page rule.

  Note that to add supported at-rules inside @page, `CSSPage3Parser` extends `parse_at_rule()`, not this method.

  Parameters

    - tokens – An iterable of token, typically from the `body` attribute of an unparsed `AtRule`.

    - context – An at-keyword such as `'@page'`. (Most at-rules are only allowed in some contexts.)

  Returns

    A tuple of:

    - A list of `Declaration`

    - A list of parsed at-rules (empty for CSS 2.1)

    - A list of `ParseError`

CSS21Parser.parse_ruleset(first_token, tokens)

  Parse a ruleset: a selector followed by declaration block.

  Parameters

    - first_token – The first token of the ruleset (probably of the selector). You may have read it already to distinguish the rule from an at-rule.

    - tokens – an iterator of subsequent tokens. Will be consumed just enough for one ruleset.

  Returns

    a tuple of a `RuleSet` and an error list. The errors are recovered `ParseError` in declarations. (Parsing continues from the next declaration on such errors.)

  Raises

    `ParseError` if the selector is invalid for the core grammar. Note a that a selector can be valid for the core grammar but not for CSS 2.1 or another level.

CSS21Parser.parse_declaration_list(tokens)

  Parse a `;` separated declaration list.

  You may want to use `parse_declarations_and_at_rules()` (or some other method that uses `parse_declaration()` directly) instead if you have not just declarations in the same context.

  Parameters

    tokens – an iterable of tokens. Should stop at (before) the end of the block, as marked by `}`.

  Returns

    a tuple of the list of valid `Declaration` and a list of `ParseError`

CSS21Parser.parse_declaration(tokens)

  Parse a single declaration.

  Parameters

    tokens – an iterable of at least one token. Should stop at (before) the end of the declaration, as marked by a `;` or `}`. Empty declarations (ie. consecutive `;` with only white space in-between) should be skipped earlier and not passed to this method.

  Returns

    a `Declaration`

  Raises

    `ParseError` if the tokens do not match the ‘declaration’ production of the core grammar.

CSS21Parser.parse_value_priority(tokens)

  Separate any `!important` marker at the end of a property value.

  Parameters

    tokens – A list of tokens for the property value.

  Returns

    A tuple of the actual property value (a list of tokens) and the `priority`.

## Unparsed at-rules

class tinycss.css21.AtRule(at_keyword, head, body, line, column)

  An unparsed at-rule.

  at_keyword

    The normalized (lower-case) at-keyword as a string. Eg: `'@page'`

  head

    The part of the at-rule between the at-keyword and the `{` marking the body, or the `;` marking the end of an at-rule without a body. A `TokenList`.

  body

    The content of the body between `{` and `}` as a `TokenList`, or `None` if there is no body (ie. if the rule ends with `;`).

  The head was validated against the core grammar but not the body, as the body might contain declarations. In case of an error in a declaration, parsing should continue from the next declaration. The whole rule should not be ignored as it would be for an error in the head.

  These at-rules are expected to be parsed further before reaching the user API.

## Parsing helper functions

The `tinycss.parsing` module contains helper functions for parsing tokens into a more structured form:

tinycss.parsing.strip_whitespace(tokens)

  Remove whitespace at the beggining and end of a token list.

  Whitespace tokens in-between other tokens in the list are preserved.

  Parameters

    tokens – A list of `Token` or `ContainerToken`.

  Returns

    A new sub-sequence of the list.

tinycss.parsing.split_on_comma(tokens)

  Split a list of tokens on commas, ie `,` DELIM tokens.

  Only “top-level” comma tokens are splitting points, not commas inside a function or other `ContainerToken`.

  Parameters

    tokens – An iterable of `Token` or `ContainerToken`.

  Returns

    A list of lists of tokens

tinycss.parsing.validate_value(tokens)

  Validate a property value.

  Parameters

    tokens – an iterable of tokens

  Raises

    `ParseError` if there is any invalid token for the ‘value’ production of the core grammar.

tinycss.parsing.validate_block(tokens, context)
  Raises

    `ParseError` if there is any invalid token for the ‘block’ production of the core grammar.

  Parameters

    - tokens – an iterable of tokens

    - context – a string for the ‘unexpected in …’ message

tinycss.parsing.validate_any(token, context)
  Raises

    `ParseError` if this is an invalid token for the ‘any’ production of the core grammar.

  Parameters

    - token – a single token

    - context – a string for the ‘unexpected in …’ message