Module kyotocabinet

[frames] | no frames]

# Module kyotocabinet

# Python 3.x Binding of Kyoto Cabinet

## Introduction

Kyoto Cabinet is a library of routines for managing a database. The database is a simple data file containing records, each is a pair of a key and a value. Every key and value is serial bytes with variable length. Both binary data and character string can be used as a key and a value. Each key must be unique within a database. There is neither concept of data tables nor data types. Records are organized in hash table or B+ tree.

The following access methods are provided to the database: storing a record with a key and a value, deleting a record by a key, retrieving a record by a key. Moreover, traversal access to every key are provided. These access methods are similar to ones of the original DBM (and its followers: NDBM and GDBM) library defined in the UNIX standard. Kyoto Cabinet is an alternative for the DBM because of its higher performance.

Each operation of the hash database has the time complexity of "O(1)". Therefore, in theory, the performance is constant regardless of the scale of the database. In practice, the performance is determined by the speed of the main memory or the storage device. If the size of the database is less than the capacity of the main memory, the performance will seem on-memory speed, which is faster than std::map of STL. Of course, the database size can be greater than the capacity of the main memory and the upper limit is 8 exabytes. Even in that case, each operation needs only one or two seeking of the storage device.

Each operation of the B+ tree database has the time complexity of "O(log N)". Therefore, in theory, the performance is logarithmic to the scale of the database. Although the performance of random access of the B+ tree database is slower than that of the hash database, the B+ tree database supports sequential access in order of the keys, which realizes forward matching search for strings and range search for integers. The performance of sequential access is much faster than that of random access.

This library wraps the polymorphic database of the C++ API. So, you can select the internal data structure by specifying the database name in runtime. This library works on Python 3.x (3.1 or later) only. Python 2.x requires another dedicated package.

## Installation

Install the latest version of Kyoto Cabinet beforehand and get the package of the Python binding of Kyoto Cabinet.

Enter the directory of the extracted package then perform installation. If your system has the another command except for the "python3" command, edit the Makefile beforehand.:

    make
    make check
    su
    make install

Symbols of the module `kyotocabinet' should be included in each source file of application programs.:

    import kyotocabinet

An instance of the class `DB' is used in order to handle a database. You can store, delete, and retrieve records with the instance.

## Example

The following code is a typical example to use a database.:

    from kyotocabinet import *
    import sys

    # create the database object
    db = DB()

    # open the database
    if not db.open("casket.kch", DB.OWRITER | DB.OCREATE):
        print("open error: " + str(db.error()), file=sys.stderr)

    # store records
    if not db.set("foo", "hop") or          not db.set("bar", "step") or          not db.set("baz", "jump"):
        print("set error: " + str(db.error()), file=sys.stderr)

    # retrieve records
    value = db.get_str("foo")
    if value:
        print(value)
    else:
        print("get error: " + str(db.error()), file=sys.stderr)

    # traverse records
    cur = db.cursor()
    cur.jump()
    while True:
        rec = cur.get_str(True)
        if not rec: break
        print(rec[0] + ":" + rec[1])
    cur.disable()

    # close the database
    if not db.close():
        print("close error: " + str(db.error()), file=sys.stderr)

The following code is a more complex example, which uses the Visitor pattern.:

    from kyotocabinet import *
    import sys

    # create the database object
    db = DB()

    # open the database
    if not db.open("casket.kch", DB.OREADER):
        print("open error: " + str(db.error()), file=sys.stderr)

    # define the visitor
    class VisitorImpl(Visitor):
        # call back function for an existing record
        def visit_full(self, key, value):
            print("{}:{}".format(key.decode(), value.decode()))
            return self.NOP
        # call back function for an empty record space
        def visit_empty(self, key):
            print("{} is missing".format(key.decode()), file=sys.stderr)
            return self.NOP
    visitor = VisitorImpl()

    # retrieve a record with visitor
    if not db.accept("foo", visitor, False) or          not db.accept("dummy", visitor, False):
        print("accept error: " + str(db.error()), file=sys.stderr)

    # traverse records with visitor
    if not db.iterate(visitor, False):
        print("iterate error: " + str(db.error()), file=sys.stderr)

    # close the database
    if not db.close():
        print("close error: " + str(db.error()), file=sys.stderr)

The following code is also a complex example, which is more suited to the Python style.:

    from kyotocabinet import *
    import sys

    # define the functor
    def dbproc(db):

      # store records
      db[b'foo'] = b'step';  # bytes is fundamental
      db['bar'] = 'hop';     # string is also ok
      db[3] = 'jump';        # number is also ok

      # retrieve a record value
      print("{}".format(db['foo'].decode()))

      # update records in transaction
      def tranproc():
          db['foo'] = 2.71828
          return True
      db.transaction(tranproc)

      # multiply a record value
      def mulproc(key, value):
          return float(value) * 2
      db.accept('foo', mulproc)

      # traverse records by iterator
      for key in db:
          print("{}:{}".format(key.decode(), db[key].decode()))

      # upcase values by iterator
      def upproc(key, value):
          return value.upper()
      db.iterate(upproc)

      # traverse records by cursor
      def curproc(cur):
          cur.jump()
          def printproc(key, value):
              print("{}:{}".format(key.decode(), value.decode()))
              return Visitor.NOP
          while cur.accept(printproc):
              cur.step()
      db.cursor_process(curproc)

    # process the database by the functor
    DB.process(dbproc, 'casket.kch')

## License

Copyright (C) 2009-2010 FAL Labs. All rights reserved.

Kyoto Cabinet is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or any later version.

Kyoto Cabinet is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

Classes
| Error Error data.
| Visitor Interface to access a record.
| FileProcessor Interface to process the database file.
| Cursor Interface of cursor to indicate a record.
| DB Interface of database abstraction.

Functions

conv_bytes(obj) Convert any object to a string. |

atoi(str) Convert a string to an integer. |

atoix(str) Convert a string with a metric prefix to an integer. |

atof(str) Convert a string to a real number. |

hash_murmur(str) Get the hash value of a string by MurMur hashing. |

hash_fnv(str) Get the hash value of a string by FNV hashing. |

levdist(a, b, utf) Calculate the levenshtein distance of two strings. |

Variables
| VERSION = ``'``x.y.z``'`` The version information.
| __package__ = `None`

Function Details

### conv_bytes(obj)

Convert any object to a string.

Parameters:
  - `obj` - the object.
Returns:
  the result string.

### atoi(str)

Convert a string to an integer.

Parameters:
  - `str` - specifies the string.
Returns:
  the integer. If the string does not contain numeric expression, 0 is returned.

### atoix(str)

Convert a string with a metric prefix to an integer.

Parameters:
  - `str` - the string, which can be trailed by a binary metric prefix. "K", "M", "G", "T", "P", and "E" are supported. They are case-insensitive.
Returns:
  the integer. If the string does not contain numeric expression, 0 is returned. If the integer overflows the domain, INT64_MAX or INT64_MIN is returned according to the sign.

### atof(str)

Convert a string to a real number.

Parameters:
  - `str` - specifies the string.
Returns:
  the real number. If the string does not contain numeric expression, 0.0 is returned.

### hash_murmur(str)

Get the hash value of a string by MurMur hashing.

Parameters:
  - `str` - the string.
Returns:
  the hash value.

### hash_fnv(str)

Get the hash value of a string by FNV hashing.

Parameters:
  - `str` - the string.
Returns:
  the hash value.

### levdist(a, b, utf)

Calculate the levenshtein distance of two strings.

Parameters:
  - `a` - one string.
  - `b` - the other string.
  - `utf` - flag to treat keys as UTF-8 strings.
Returns:
  the levenshtein distance.