### Navigation

- index
- modules |
- next |
- previous |
- patsy 0.2.1 documentation »

# Coding categorical data

Patsy allows great flexibility in how categorical data is coded, via the function `C()`. `C()` marks some data as being categorical (including data which would not automatically be treated as categorical, such as a column of integers), while also optionally setting the preferred coding scheme and level ordering.

Let’s get some categorical data to work with:

    In [1]: from patsy import dmatrix, demo_data, ContrastMatrix, Poly

    In [2]: data = demo_data("a", nlevels=3)

    In [3]: data
    Out[3]: {'a': ['a1', 'a2', 'a3', 'a1', 'a2', 'a3']}

As you know, simply giving Patsy a categorical variable causes it to be coded using the default `Treatment` coding scheme. (Strings and booleans are treated as categorical by default.)

    In [1]: dmatrix("a", data)
    Out[1]:
    DesignMatrix with shape (6, 3)
      Intercept  a[T.a2]  a[T.a3]
              1        0        0
              1        1        0
              1        0        1
              1        0        0
              1        1        0
              1        0        1
      Terms:
        'Intercept' (column 0)
        'a' (columns 1:3)

We can also alter the level ordering, which is useful for, e.g., `Diff` coding:

    In [1]: l = ["a3", "a2", "a1"]

    In [2]: dmatrix("C(a, levels=l)", data)
    Out[2]:
    DesignMatrix with shape (6, 3)
      Intercept  C(a, levels=l)[T.a2]  C(a, levels=l)[T.a1]
              1                     0                     1
              1                     1                     0
              1                     0                     0
              1                     0                     1
              1                     1                     0
              1                     0                     0
      Terms:
        'Intercept' (column 0)
        'C(a, levels=l)' (columns 1:3)

But the default coding is just that – a default. The easiest alternative is to use one of the other built-in coding schemes, like orthogonal polynomial coding:

    In [3]: dmatrix("C(a, Poly)", data)
    Out[3]:
    DesignMatrix with shape (6, 3)
      Intercept  C(a, Poly).Linear  C(a, Poly).Quadratic
              1           -0.70711               0.40825
              1           -0.00000              -0.81650
              1            0.70711               0.40825
              1           -0.70711               0.40825
              1           -0.00000              -0.81650
              1            0.70711               0.40825
      Terms:
        'Intercept' (column 0)
        'C(a, Poly)' (columns 1:3)

There are a number of built-in coding schemes; for details you can check the API reference. But we aren’t restricted to those. We can also provide a custom contrast matrix, which allows us to produce all kinds of strange designs:

    In [4]: contrast = [[1, 2], [3, 4], [5, 6]]

    In [5]: dmatrix("C(a, contrast)", data)
    Out[5]:
    DesignMatrix with shape (6, 3)
      Intercept  C(a, contrast)[custom0]  C(a, contrast)[custom1]
              1                        1                        2
              1                        3                        4
              1                        5                        6
              1                        1                        2
              1                        3                        4
              1                        5                        6
      Terms:
        'Intercept' (column 0)
        'C(a, contrast)' (columns 1:3)

    In [6]: dmatrix("C(a, [[1], [2], [-4]])", data)
    Out[6]:
    DesignMatrix with shape (6, 2)
      Intercept  C(a, [[1], [2], [-4]])[custom0]
              1                                1
              1                                2
              1                               -4
              1                                1
              1                                2
              1                               -4
      Terms:
        'Intercept' (column 0)
        'C(a, [[1], [2], [-4]])' (column 1)

Hmm, those `[custom0]`, `[custom1]` names that Patsy auto-generated for us are a bit ugly looking. We can attach names to our contrast matrix by creating a `ContrastMatrix` object, and make things prettier:

    In [7]: contrast_mat = ContrastMatrix(contrast, ["[pretty0]", "[pretty1]"])

    In [8]: dmatrix("C(a, contrast_mat)", data)
    Out[8]:
    DesignMatrix with shape (6, 3)
      Intercept  C(a, contrast_mat)[pretty0]  C(a, contrast_mat)[pretty1]
              1                            1                            2
              1                            3                            4
              1                            5                            6
              1                            1                            2
              1                            3                            4
              1                            5                            6
      Terms:
        'Intercept' (column 0)
        'C(a, contrast_mat)' (columns 1:3)

And, finally, if we want to get really fancy, we can also define our own “smart” coding schemes like `Poly`. Just define a class that has two methods, `code_with_intercept()` and `code_without_intercept()`. They have identical signatures, taking a list of levels as their argument and returning a `ContrastMatrix`. Patsy will automatically choose the appropriate method to call to produce a full-rank design matrix without redundancy; see Redundancy and categorical factors for the full details on how Patsy makes this decision.

As an example, here’s a simplified version of the built-in `Treatment` coding object:

    import numpy as np

    class MyTreat(object):
        def __init__(self, reference=0):
            self.reference = reference

        def code_with_intercept(self, levels):
            return ContrastMatrix(np.eye(len(levels)),
                                  ["[My.%s]" % (level,) for level in levels])

        def code_without_intercept(self, levels):
            eye = np.eye(len(levels) - 1)
            contrasts = np.vstack((eye[:self.reference, :],
                                   np.zeros((1, len(levels) - 1)),
                                   eye[self.reference:, :]))
            suffixes = ["[MyT.%s]" % (level,) for level in
                        levels[:self.reference] + levels[self.reference + 1:]]
            return ContrastMatrix(contrasts, suffixes)

And it can now be used just like the built-in methods:

    # Full rank:
    In [10]: dmatrix("0 + C(a, MyTreat)", data)
    Out[10]:
    DesignMatrix with shape (6, 3)
      C(a, MyTreat)[My.a1]  C(a, MyTreat)[My.a2]  C(a, MyTreat)[My.a3]
                         1                     0                     0
                         0                     1                     0
                         0                     0                     1
                         1                     0                     0
                         0                     1                     0
                         0                     0                     1
      Terms:
        'C(a, MyTreat)' (columns 0:3)

    # Reduced rank:
    In [11]: dmatrix("C(a, MyTreat)", data)
    Out[11]:
    DesignMatrix with shape (6, 3)
      Intercept  C(a, MyTreat)[MyT.a2]  C(a, MyTreat)[MyT.a3]
              1                      0                      0
              1                      1                      0
              1                      0                      1
              1                      0                      0
              1                      1                      0
              1                      0                      1
      Terms:
        'Intercept' (column 0)
        'C(a, MyTreat)' (columns 1:3)

    # With argument:
    In [12]: dmatrix("C(a, MyTreat(2))", data)
    Out[12]:
    DesignMatrix with shape (6, 3)
      Intercept  C(a, MyTreat(2))[MyT.a1]  C(a, MyTreat(2))[MyT.a2]
              1                         1                         0
              1                         0                         1
              1                         0                         0
              1                         1                         0
              1                         0                         1
              1                         0                         0
      Terms:
        'Intercept' (column 0)
        'C(a, MyTreat(2))' (columns 1:3)

### Navigation

- index
- modules |
- next |
- previous |
- patsy 0.2.1 documentation »