Skip to content

# Fields

## The `id` field

The `ObjectId` data type is the default primary key type used by MongoDB. An `ObjectId` comes with many information embedded into it (timestamp, machine identifier, ...). Since by default, MongoDB will create a field `_id` containing an `ObjectId` primary key, ODMantic will bind it automatically to an implicit field named `id`.

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12

    from odmantic import Model

    class Player(Model):
        name: str

    leeroy = Player(name="Leeroy Jenkins")
    print(leeroy.id)
    #> ObjectId('5ed50fcad11d1975aa3d7a28')
    print(repr(leeroy))
    #> Player(id=ObjectId('5ed50fcad11d1975aa3d7a28'), name="Leeroy Jenkins")

ObjectId creation

This `id` field will be generated on instance creation, before saving the instance to the database. This helps to keep consistency between the instances persisted to the database and the ones only created locally.

Even if this behavior is convenient, it is still possible to define custom primary keys.

## Field types

### Optional fields

By default, every single field will be required. To specify a field as non-required, the easiest way is to use the `typing.Optional` generic type that will allow the field to take the `None` value as well (it will be stored as `null` in the database) and to give it a default value of `None`.

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12

    from typing import Optional

    from odmantic import Model

    class Person(Model):
        name: str
        age: Optional[int] = None

    john = Person(name="John")
    print(john.age)
    #> None

### Union fields

As explained in the Python Typing documentation, `Optional[X]` is equivalent to `Union[X, None]`. That implies that the field type will be either `X` or `None`.

It's possible to combine any kind of type using the `typîng.Union` type constructor. For example if we want to allow both `string` and `int` in a field:

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16

    from typing import Union

    from odmantic import Model

    class Thing(Model):
        ref_id: Union[int, str]

    thing_1 = Thing(ref_id=42)
    print(thing_1.ref_id)
    #> 42

    thing_2 = Thing(ref_id="i am a string")
    print(thing_2.ref_id)
    #> i am a string

NoneType

Internally python describes the type of the `None` object as `NoneType` but in practice, `None` is used directly in type annotations (more details).

### Enum fields

To define choices, it's possible to use the standard `enum` classes:

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29

    from enum import Enum

    from odmantic import AIOEngine, Model

    class TreeKind(str, Enum):
        BIG = "big"
        SMALL = "small"

    class Tree(Model):
        name: str
        kind: TreeKind

    sequoia = Tree(name="Sequoia", kind=TreeKind.BIG)
    print(sequoia.kind)
    #> TreeKind.BIG
    print(sequoia.kind == "big")
    #> True

    spruce = Tree(name="Spruce", kind="small")
    print(spruce.kind)
    #> TreeKind.SMALL
    print(spruce.kind == TreeKind.SMALL)
    #> True

    engine = AIOEngine()
    await engine.save_all([sequoia, spruce])

Resulting documents in the collection `tree` after execution

    { "_id" : ObjectId("5f818f2dd5708527282c49b6"), "kind" : "big", "name" : "Sequoia" }
    { "_id" : ObjectId("5f818f2dd5708527282c49b7"), "kind" : "small", "name" : "Spruce" }

If you try to use a value not present in the allowed choices, a ValidationError exception will be raised.

Usage of `enum.auto`

If you might add some values to an `Enum`, it's strongly recommended not to use the `enum.auto` value generator. Depending on the order you add choices, it could completely break the consistency with documents stored in the database.

Unwanted behavior example

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12

    from enum import Enum, auto

    class Color(Enum):
        RED = auto()
        BLUE = auto()

    print(Color.RED.value)
    #> 1
    print(Color.BLUE.value)
    #> 2

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15

    from enum import Enum, auto

    class Color(Enum):
        RED = auto()
        GREEN = auto()
        BLUE = auto()

    print(Color.RED.value)
    #> 1
    print(Color.GREEN.value)
    #> 2
    print(Color.BLUE.value)
    #> 3

### Container fields

#### List

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33

    from typing import List, Union

    from odmantic import Model

    class SimpleListModel(Model):
        field: list

    print(SimpleListModel(field=[1, "a", True]).field)
    #> [1, 'a', True]
    print(SimpleListModel(field=(1, "a", True)).field)
    #> [1, 'a', True]

    class IntListModel(Model):
        field: List[int]

    print(IntListModel(field=[1, 5]).field)
    #> [1, 5]
    print(IntListModel(field=(1, 5)).field)
    #> [1, 5]

    class IntStrListModel(Model):
        field: List[Union[int, str]]

    print(IntStrListModel(field=[1, "b"]).field)
    #> [1, 'b']
    print(IntStrListModel(field=(1, "b")).field)
    #> [1, 'b']

Tip

It's possible to define element count constraints for a list field using the Field descriptor.

#### Tuple

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33

    from typing import Tuple

    from odmantic import Model

    class SimpleTupleModel(Model):
        field: tuple

    print(SimpleTupleModel(field=[1, "a", True]).field)
    #> (1, 'a', True)
    print(SimpleTupleModel(field=(1, "a", True)).field)
    #> (1, 'a', True)

    class TwoIntTupleModel(Model):
        field: Tuple[int, int]

    print(SimpleTupleModel(field=(1, 10)).field)
    #> (1, 10)
    print(SimpleTupleModel(field=[1, 10]).field)
    #> (1, 10)

    class IntTupleModel(Model):
        field: Tuple[int, ...]

    print(IntTupleModel(field=(1,)).field)
    #> (1,)
    print(IntTupleModel(field=[1, 2, 3, 10]).field)
    #> (1, 2, 3, 10)

#### Dict

Tip

For mapping types with already known keys, you can see the embedded models section.

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27

    from typing import Dict, Union

    from odmantic import Model

    class SimpleDictModel(Model):
        field: dict

    print(SimpleDictModel(field={18: "a string", True: 42, 18.3: [1, 2, 3]}).field)
    #> {18: 'a string', True: 42, 18.3: [1, 2, 3]}

    class IntStrDictModel(Model):
        field: Dict[int, str]

    print(IntStrDictModel(field={1: "one", 2: "two"}).field)
    #> {1: 'one', 2: 'two'}

    class IntBoolStrDictModel(Model):
        field: Dict[int, Union[bool, str]]

    print(IntBoolStrDictModel(field={0: False, 1: True, 3: "three"}).field)
    #> {0: False, 1: True, 3: 'three'}

Performance tip

Whenever possible, try to avoid mutable container types (`List`, `Set`, ...) and prefer their Immutable alternatives (`Tuple`, `FrozenSet`, ...). This will allow ODMantic to speedup database writes by only saving the modified container fields.

### `BSON` types integration

ODMantic supports native python BSON types (`bson` package). Those types can be used directly as field types:

- `bson.ObjectId`

- `bson.Int64`

- `bson.Decimal128`

- `bson.Regex`

- `bson.Binary`

Generic python to BSON type map

Python type | BSON type | Comment
--- | --- | ---
`bson.ObjectId` | `objectId` |
`bool` | `bool` |
`int` | `int` | value between -2^31 and 2^31 - 1
`int` | `long` | value not between -2^31 and 2^31 - 1
`bson.Int64` | `long` |
`float` | `double` |
`bson.Decimal128` | `decimal` |
`decimal.Decimal` | `decimal` |
`str` | `string` |
`typing.Pattern` | `regex` |
`bson.Regex` | `regex` |
`bytes` | `binData` |
`bson.Binary` | `binData` |
`datetime.datetime` | `date` | microseconds are truncated, only naive datetimes are allowed
`typing.Dict` | `object` |
`typing.List` | `array` |
`typing.Sequence` | `array` |
`typing.Tuple[T, ...]` | `array` |

### Pydantic fields

Most of the types supported by pydantic are supported by ODMantic. See pydantic: Field Types for more field types.

Unsupported fields:

- `typing.Callable`

Fields with a specific behavior:

- `datetime.datetime`: Only naive datetime objects will be allowed as MongoDB doesn't store the timezone information. Also, the microsecond information will be truncated.

## Customization

The field customization can mainly be performed using the Field descriptor. This descriptor is here to define everything about the field except its type.

### Default values

The easiest way to set a default value to a field is by assigning this default value directly while defining the model.

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11

    from odmantic import Model

    class Player(Model):
        name: str
        level: int = 0

    p = Player(name="Dash")
    print(repr(p))
    #> Player(id=ObjectId('5f7cd4be16af832772f1615e'), name='Dash', level=0)

You can combine default values and an existing Field descriptor using the `default` keyword argument.

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11

    from odmantic import Field, Model

    class Player(Model):
        name: str
        level: int = Field(default=1, ge=1)

    p = Player(name="Dash")
    print(repr(p))
    #> Player(id=ObjectId('5f7cdbfbb54a94e9e8717c77'), name='Dash', level=1)

Default factory

You may as well define a factory function instead of a value using the `default_factory` argument of the Field descriptor.

By default, the default factories won't be used while parsing MongoDB documents. It's possible to enable this behavior with the `parse_doc_with_default_factories` Config option.

Default values validation

Currently the default values are not validated yet during the model creation.

An inconsistent default value might raise a ValidationError while building an instance.

### Document structure

By default, the MongoDB documents fields will be named after the field name. It is possible to override this naming policy by using the `key_name` argument in the Field descriptor.

Async Sync

    1
    2
    3
    4
    5
    6
    7
    8
    9

    from odmantic import AIOEngine, Field, Model

    class Player(Model):
        name: str = Field(key_name="username")

    engine = AIOEngine()
    await engine.save(Player(name="Jack"))

    1
    2
    3
    4
    5
    6
    7
    8
    9

    from odmantic import SyncEngine, Field, Model

    class Player(Model):
        name: str = Field(key_name="username")

    engine = SyncEngine()
    engine.save(Player(name="Jack"))

Resulting documents in the collection `player` after execution

    {
      "_id": ObjectId("5ed50fcad11d1975aa3d7a28"),
      "username": "Jack",
    }

See this section for more details about the `_id` field that has been added.

### Primary key

While ODMantic will by default populate the `id` field as a primary key, you can use any other field as the primary key.

Async Sync

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13

    from odmantic import AIOEngine, Field, Model

    class Player(Model):
        name: str = Field(primary_field=True)

    leeroy = Player(name="Leeroy Jenkins")
    print(repr(leeroy))
    #> Player(name="Leeroy Jenkins")

    engine = AIOEngine()
    await engine.save(leeroy)

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13

    from odmantic import SyncEngine, Field, Model

    class Player(Model):
        name: str = Field(primary_field=True)

    leeroy = Player(name="Leeroy Jenkins")
    print(repr(leeroy))
    #> Player(name="Leeroy Jenkins")

    engine = SyncEngine()
    engine.save(leeroy)

Resulting documents in the collection `player` after execution

    {
        "_id": "Leeroy Jenkins"
    }

Info

The Mongo name of the primary field will be enforced to `_id` and you will not be able to change it.

Warning

Using mutable types (Set, List, ...) as primary field might result in inconsistent behaviors.

### Indexed fields

You can define an index on a single field by using the `index` argument of the Field descriptor.

More details about index creation can be found in the Indexes section.

Async Sync

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10

    from odmantic import AIOEngine, Field, Model

    class Player(Model):
        name: str
        score: int = Field(index=True)

    engine = AIOEngine()
    await engine.configure_database([Player])

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10

    from odmantic import Field, Model, SyncEngine

    class Player(Model):
        name: str
        score: int = Field(index=True)

    engine = SyncEngine()
    engine.configure_database([Player])

Warning

When using indexes, make sure to call the `configure_database` method (AIOEngine.configure_database or SyncEngine.configure_database) to persist the indexes to the database.

### Unique fields

In the same way, you can define unique constrains on a single field by using the `unique` argument of the Field descriptor. This will ensure that values of this fields are unique among all the instances saved in the database.

More details about unique index creation can be found in the Indexes section.

Async Sync

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18

    from odmantic import AIOEngine, Field, Model

    class Player(Model):
        name: str = Field(unique=True)

    engine = AIOEngine()
    await engine.configure_database([Player])

    leeroy = Player(name="Leeroy")
    await engine.save(leeroy)

    another_leeroy = Player(name="Leeroy")
    await engine.save(another_leeroy)
    #> Raises odmantic.exceptions.DuplicateKeyError:
    #>    Duplicate key error for: Player.
    #>    Instance: id=ObjectId('6314b4c25a19444bfe0c0be5') name='Leeroy'

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18

    from odmantic import Field, Model, SyncEngine

    class Player(Model):
        name: str = Field(unique=True)

    engine = SyncEngine()
    engine.configure_database([Player])

    leeroy = Player(name="Leeroy")
    engine.save(leeroy)

    another_leeroy = Player(name="Leeroy")
    engine.save(another_leeroy)
    #> Raises odmantic.exceptions.DuplicateKeyError:
    #>    Duplicate key error for: Player.
    #>    Instance: id=ObjectId('6314b4c25a19444bfe0c0be5') name='Leeroy'

Warning

When using indexes, make sure to call the `configure_database` method (AIOEngine.configure_database or SyncEngine.configure_database) to persist the indexes to the database.

## Validation

As ODMantic strongly relies on pydantic when it comes to data validation, most of the validation features provided by pydantic are available:

- Add field validation constraints by using the Field descriptor

       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19

      from typing import List

      from odmantic import Field, Model

      class ExampleModel(Model):
          small_int: int = Field(le=10)
          big_int: int = Field(gt=1000)
          even_int: int = Field(multiple_of=2)

          small_float: float = Field(lt=10)
          big_float: float = Field(ge=1e10)

          short_string: str = Field(max_length=10)
          long_string: str = Field(min_length=100)
          string_starting_with_the: str = Field(regex=r"^The")

          short_str_list: List[str] = Field(max_items=5)
          long_str_list: List[str] = Field(min_items=15)

- Use strict types to prevent to coercion from compatible types (pydantic: Strict Types)

      1
      2
      3
      4
      5
      6
      7
      8
      9

      from pydantic import StrictBool, StrictFloat, StrictStr

      from odmantic import Model

      class ExampleModel(Model):
          strict_bool: StrictBool
          strict_float: StrictFloat
          strict_str: StrictStr

- Define custom field validators (pydantic: Validators)

       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      27
      28
      29
      30
      31
      32
      33
      34
      35
      36
      37
      38
      39
      40
      41
      42
      43
      44
      45
      46
      47
      48
      49
      50
      51
      52
      53
      54
      55
      56
      57
      58
      59

      from typing import ClassVar

      from pydantic import ValidationError, validator

      from odmantic import Model

      class SmallRectangle(Model):
          MAX_SIDE_SIZE: ClassVar[float] = 10

          length: float
          width: float

          @validator("width", "length")
          def check_small_sides(cls, v):
              if v > cls.MAX_SIDE_SIZE:
                  raise ValueError(f"side is greater than {cls.MAX_SIDE_SIZE}")
              return v

          @validator("width")
          def check_width_length(cls, width, values, **kwargs):
              length = values.get("length")
              if length is not None and width > length:
                  raise ValueError("width can't be greater than length")
              return width

      print(SmallRectangle(length=2, width=1))
      #> id=ObjectId('5f81e3c073103f509f97e374'), length=2.0, width=1.0

      try:
          SmallRectangle(length=2)
      except ValidationError as e:
          print(e)
          """
          1 validation error for SmallRectangle
          width
            field required (type=value_error.missing)
          """

      try:
          SmallRectangle(length=2, width=3)
      except ValidationError as e:
          print(e)
          """
          1 validation error for SmallRectangle
          width
            width can't be greater than length (type=value_error)
          """

      try:
          SmallRectangle(length=40, width=3)
      except ValidationError as e:
          print(e)
          """
          1 validation error for SmallRectangle
          length
            side is greater than 10 (type=value_error)
          """

- Define custom model validators: more details

## Custom field types

Exactly in the same way pydantic allows it, it's possible to define custom field types as well with ODMantic (Pydantic: Custom data types).

Sometimes, it might be required to customize as well the field BSON serialization. In order to do this, the field class will have to implement the `__bson__` class method.

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38

    from typing import Annotated
    from odmantic import AIOEngine, Model, WithBsonSerializer

    class ASCIISerializedAsBinaryBase(str):
        @classmethod
        def __get_validators__(cls):
            yield cls.validate

        @classmethod
        def validate(cls, v):
            if isinstance(v, bytes):  # Handle data coming from MongoDB
                return v.decode("ascii")
            if not isinstance(v, str):
                raise TypeError("string required")
            if not v.isascii():
                raise ValueError("Only ascii characters are allowed")
            return v

    def serialize_ascii_to_bytes(v: ASCIISerializedAsBinaryBase) -> bytes:
        # We can encode this string as ascii since it contains
        # only ascii characters
        bytes_ = v.encode("ascii")
        return bytes_

    ASCIISerializedAsBinary = Annotated[
        ASCIISerializedAsBinaryBase, WithBsonSerializer(serialize_ascii_to_bytes)
    ]

    class Example(Model):
        field: ASCIISerializedAsBinary

    engine = AIOEngine()
    await engine.save(Example(field="hello world"))
    fetched = await engine.find_one(Example)
    print(fetched.field)
    #> hello world

In this example, we decide to store string data manually encoded in the ASCII encoding. The encoding is handled in the `__bson__` class method. On top of this, we handle the decoding by attempting to decode `bytes` object in the `validate` method.

Resulting documents in the collection `example` after execution

    {
      "_id" : ObjectId("5f81fa5e8adaf4bf33f05035"),
      "field" : BinData(0,"aGVsbG8gd29ybGQ=")
    }

Warning

When using custom bson serialization, it's important to handle as well the data validation for data retrieved from Mongo. In the previous example it's done by handling `bytes` objects in the validate method.