# logilab.common package

## Subpackages

- logilab.common.ureports package
  - Submodules
  - logilab.common.ureports.docbook_writer module
  - logilab.common.ureports.html_writer module
  - logilab.common.ureports.nodes module
  - logilab.common.ureports.text_writer module
  - Module contents

## Submodules

## logilab.common.cache module

Cache module, with a least recently used algorithm for the management of the deletion of entries.

class logilab.common.cache.Cache(size: int = 100)

  Bases: `dict`

  A dictionary like cache.

  inv:

    len(self._usage) <= self.size len(self.data) <= self.size

  clear() → None. Remove all items from D.
  pop(k[, d]) → v, remove specified key and return the corresponding value.

    If key is not found, default is returned if given, otherwise KeyError is raised

  popitem()

    Remove and return a (key, value) pair as a 2-tuple.

    Pairs are returned in LIFO (last-in, first-out) order. Raises KeyError if the dict is empty.

  setdefault(key, default=None)

    Insert key with a value of default if key is not in the dictionary.

    Return the value for key if key is in the dictionary, else default.

  update([E, ]**F) → None. Update D from dict/iterable E and F.

    If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]

## logilab.common.changelog module

Manipulation of upstream change log files.

The upstream change log files format handled is simpler than the one often used such as those generated by the default Emacs changelog mode.

Sample ChangeLog format:

    Change log for project Yoo
    ==========================

     --
        * add a new functionality

    2002-02-01 -- 0.1.1
        * fix bug #435454
        * fix bug #434356

    2002-01-01 -- 0.1
        * initial release

There is 3 entries in this change log, one for each released version and one for the next version (i.e. the current entry). Each entry contains a set of messages corresponding to changes done in this release. All the non empty lines before the first entry are considered as the change log title.

class logilab.common.changelog.ChangeLog(changelog_file: str, title: str = '')

  Bases: `object`

  object representation of a whole ChangeLog file

  add(msg, create=None)

    add a new message to the latest opened entry

  add_entry(entry: logilab.common.changelog.ChangeLogEntry) → None

    add a new entry to the change log

  entry_class

    alias of `logilab.common.changelog.ChangeLogEntry`

  format_title() → str
  get_entry(version='', create=None)

    return a given changelog entry if version is omitted, return the current entry

  load() → None

    read a logilab’s ChangeLog from file

  save()

    write back change log

  write(stream: _io.StringIO = <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>) → None

    write changelog to stream

class logilab.common.changelog.ChangeLogEntry(date: Optional[str] = None, version: Optional[str] = None, **kwargs: Any)

  Bases: `object`

  a change log entry, i.e. a set of messages associated to a version and its release date

  add_message(msg: str) → None

    add a new message

  add_sub_message(sub_msg: str, key: Optional[Any] = None) → None
  complete_latest_message(msg_suite: str) → None

    complete the latest added message

  version_class

    alias of `logilab.common.changelog.Version`

  write(stream: _io.StringIO = <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>) → None

    write the entry to file

exception logilab.common.changelog.EntryNotFound

  Bases: `Exception`

  raised when we are unable to find a given entry

exception logilab.common.changelog.NoEntry

  Bases: `Exception`

  raised when we are unable to find an entry

class logilab.common.changelog.Version(versionstr)

  Bases: `tuple`

  simple class to handle soft version number has a tuple while correctly printing it as X.Y.Z

  classmethod parse(versionstr: str) → List[int]

## logilab.common.clcommands module

Helper functions to support command line tools providing more than one command.

e.g called as “tool command [options] args…” where <options> and <args> are command’specific

exception logilab.common.clcommands.BadCommandUsage

  Bases: `Exception`

  Raised when an unknown command is used or when a command is not correctly used (bad options, too much / missing arguments…).

  Trigger display of command usage.

class logilab.common.clcommands.Command(logger)

  Bases: `logilab.common.configuration.Configuration`

  Base class for command line commands.

  Class attributes:

  - name, the name of the command

  - min_args, minimum number of arguments, None if unspecified

  - max_args, maximum number of arguments, None if unspecified

  - arguments, string describing arguments, used in command usage

  - hidden, boolean flag telling if the command should be hidden, e.g. does not appear in help’s commands list

  - options, options list, as allowed by :mod:configuration

  arguments = ''
  check_args(args)

    check command’s arguments are provided

  classmethod description()
  hidden = False
  main_run(args, rcfile=None)

    Run the command and return status 0 if everything went fine.

    If `CommandError` is raised by the underlying command, simply log the error and return status 2.

    Any other exceptions, including `BadCommandUsage` will be propagated.

  max_args = None
  min_args = None
  name = ''
  run(args)

    run the command with its specific arguments

  classmethod short_description()

exception logilab.common.clcommands.CommandError

  Bases: `Exception`

  Raised when a command can’t be processed and we want to display it and exit, without traceback nor usage displayed.

class logilab.common.clcommands.CommandLine(pgm=None, doc=None, copyright=None, version=None, rcfile=None, logthreshold=40, check_duplicated_command=True)

  Bases: `dict`

  Usage:

      >>> LDI = cli.CommandLine('ldi', doc='Logilab debian installer',
                                version=version, rcfile=RCFILE)
      >>> LDI.register(MyCommandClass)
      >>> LDI.register(MyOtherCommandClass)
      >>> LDI.run(sys.argv[1:])

  Arguments:

  - pgm, the program name, default to basename(sys.argv[0])

  - doc, a short description of the command line tool

  - copyright, additional doc string that will be appended to the generated doc

  - version, version number of string of the tool. If specified, global –version option will be available.

  - rcfile, path to a configuration file. If specified, global –C/–rc-file option will be available? self.rcfile = rcfile

  - logger, logger to propagate to commands, default to logging.getLogger(self.pgm))

  create_logger(handler, logthreshold=None)
  get_command(cmd, logger=None)
  register(cls, force=False)

    register the given `Command` subclass

  run(args)

    main command line access point: * init logging * handle global options (-h/–help, –version, -C/–rc-file) * check command * run command

    Terminate by `SystemExit`

  usage()

    display usage for the main program (i.e. when no command supplied) and exit

  usage_and_exit(status)

class logilab.common.clcommands.ListCommandsCommand(logger)

  Bases: `logilab.common.clcommands.Command`

  list available commands, useful for bash completion.

  arguments = '[command]'
  hidden = True
  name = 'listcommands'
  option_groups: List[Tuple[Any, str]]
  options_providers: List[ConfigurationMixIn]
  run(args)

    run the command with its specific arguments

## logilab.common.compat module

Wrappers around some builtins introduced in python 2.3, 2.4 and 2.5, making them available in for earlier versions of python.

See another compatibility snippets from other projects:

> `lib2to3.fixes` `coverage.backward` `unittest2.compatibility`

logilab.common.compat.method_type(callable, instance, klass)

logilab.common.compat.str_encode(string: Union[int, str], encoding: str) → str

logilab.common.compat.str_to_bytes(string)

## logilab.common.configuration module

Classes to handle advanced configuration in simple to complex applications.

Allows to load the configuration from a file or from command line options, to generate a sample configuration file or to display program’s usage. Fills the gap between optik/optparse and ConfigParser by adding data types (which are also available as a standalone optik extension in the optik_ext module).

### Quick start: simplest usage

class logilab.common.configuration.Configuration(config_file=None, options=None, name=None, usage=None, doc=None, version=None)

  Bases: `logilab.common.configuration.ConfigurationMixIn`

  class for simple configurations which don’t need the manager / providers model and prefer delegation to inheritance

  configuration values are accessible through a dict like interface

class logilab.common.configuration.ConfigurationMixIn(*args: Any, **kwargs: Any)

  Bases: `logilab.common.configuration.OptionsManagerMixIn`, `logilab.common.configuration.OptionsProviderMixIn`

  basic mixin for simple configurations which don’t need the manager / providers model

  get(key, default=None)
  load_defaults()

    initialize the provider using default values

  register_options(options)

    add some options to the configuration

class logilab.common.configuration.OptionsManager2ConfigurationAdapter(provider)

  Bases: `object`

  Adapt an option manager to behave like a logilab.common.configuration.Configuration instance

  get(key, default=None)

class logilab.common.configuration.OptionsManagerMixIn(usage: Optional[str], config_file: Optional[Any] = None, version: Optional[Any] = None, quiet: int = 0)

  Bases: `object`

  MixIn to handle a configuration from both a configuration file and command line options

  add_help_section(title: str, description: str, level: int = 0) → None

    add a dummy option section for help purpose

  add_optik_option(provider: logilab.common.configuration.ConfigurationMixIn, optikcontainer: Union[logilab.common.types.OptionParser, optparse.OptionGroup], opt: str, optdict: Dict[str, Any]) → None
  add_option_group(group_name: str, doc: Optional[str], options: Union[List[Tuple[str, Dict[str, Any]]], List[Tuple[str, Dict[str, str]]]], provider: logilab.common.configuration.ConfigurationMixIn) → None

    add an option group including the listed options

  cb_set_provider_option(option: logilab.common.types.Option, opt: str, value: Union[List[str], int, str], parser: logilab.common.types.OptionParser) → None

    optik callback for option setting

  generate_config(stream: Optional[Union[_io.StringIO, _io.TextIOWrapper]] = None, skipsections: Tuple[()] = (), encoding: Optional[Any] = None) → None

    write a configuration file according to the current configuration into the given stream or stdout

  generate_manpage(pkginfo: logilab.common.types.attrdict, section: int = 1, stream: Optional[_io.StringIO] = None) → None

    write a man page for the current configuration into the given stream or stdout

  global_set_option(opt: str, value: Union[List[str], int, str]) → None

    set option on the correct option provider

  help(level: int = 0) → str

    return the usage string for available options

  input_config(onlysection=None, inputlevel=0, stream=None)

    interactively get configuration values by asking to the user and generate a configuration file

  load_command_line_configuration(args: Optional[List[str]] = None) → List[str]

    override configuration according to command line parameters

    return additional arguments

  load_config_file() → None

    dispatch values previously read from a configuration file to each options provider)

  load_configuration(**kwargs: Any) → None

    override configuration according to given parameters

  load_file_configuration(config_file: Optional[str] = None) → None

    load the configuration from file

  load_provider_defaults() → None

    initialize configuration using default values

  optik_option(provider: logilab.common.configuration.ConfigurationMixIn, opt: str, optdict: Dict[str, Any]) → Tuple[List[str], Dict[str, Any]]

    get our personal option definition and return a suitable form for use with optik/optparse

  read_config_file(config_file: Optional[str] = None) → None

    read the configuration file but do not load it (i.e. dispatching values to each options provider)

  register_options_provider(provider: logilab.common.configuration.ConfigurationMixIn, own_group: bool = True) → None

    register an options provider

  reset_parsers(usage: Optional[str] = '', version: Optional[Any] = None) → None

class logilab.common.configuration.OptionsProviderMixIn

  Bases: `object`

  Mixin to provide options to an OptionsManager

  all_options()

    return an iterator on available options for this provider option are actually described by a 3-uple: (section, option name, option dictionary)

  get_option_def(opt)

    return the dictionary defining an option given it’s name

  input_option(option, optdict, inputlevel=99)
  level = 0
  load_defaults() → None

    initialize the provider using default values

  name = 'default'
  option_attrname(opt, optdict=None)

    get the config attribute corresponding to opt

  option_default(opt, optdict=None)

    return the default value for an option

  option_name(opt, optdict=None) → Callable
  option_value(opt)

    get the current value for the given option

  options: Tuple = ()
  options_and_values(options=None)
  options_by_section() → Iterator[Any]

    return an iterator on options grouped by section

    (section, [list of (optname, optdict, optvalue)])

  priority = -1
  set_option(opt, value, action=None, optdict=None)

    method called to set an option (registered in the options list)

## logilab.common.daemon module

A daemonize function (for Unices)

logilab.common.daemon.daemonize(pidfile=None, uid=None, umask=63)

  daemonize a Unix process. Set paranoid umask by default.

  Return 1 in the original process, 2 in the first fork, and None for the second fork (eg daemon process).

logilab.common.daemon.setugid(user)

  Change process user and group ID

  Argument is a numeric user id or a user name

## logilab.common.date module

Date manipulation helper functions.

logilab.common.date.add_days_worked(start: datetime.date, days: int) → datetime.date

  adds date but try to only take days worked into account

logilab.common.date.date_range(begin: datetime.date, end: datetime.date, incday: Optional[Any] = None, incmonth: Optional[bool] = None) → Generator[datetime.date, Any, None]

  yields each date between begin and end

  Parameters

    - begin – the start date

    - end – the end date

    - incr – the step to use to iterate over dates. Default is one day.

    - include – None (means no exclusion) or a function taking a date as parameter, and returning True if the date should be included.

  When using mx datetime, you should NOT use incmonth argument, use instead oneDay, oneHour, oneMinute, oneSecond, oneWeek or endOfMonth (to enumerate months) as incday argument

logilab.common.date.datefactory(year: int, month: int, day: int, sampledate: Union[datetime.date, datetime.datetime]) → Union[datetime.date, datetime.datetime]

logilab.common.date.datetime2ticks(somedate: Union[datetime.date, datetime.datetime]) → int

logilab.common.date.datetime_to_seconds(date)

  return the number of seconds since the begining of the day for that date

logilab.common.date.days_between(start: Union[datetime.date, datetime.datetime], end: Union[datetime.date, datetime.datetime]) → int

logilab.common.date.days_in_month(somedate: datetime.date) → int

logilab.common.date.days_in_year(somedate)

logilab.common.date.first_day(somedate)

logilab.common.date.get_national_holidays(begin: Union[datetime.date, datetime.datetime], end: Union[datetime.date, datetime.datetime]) → Union[List[datetime.date], List[datetime.datetime]]

  return french national days off between begin and end

logilab.common.date.get_step(dateobj: Union[datetime.date, datetime.datetime], nbdays: int = 1) → datetime.timedelta

logilab.common.date.last_day(somedate: datetime.date) → datetime.date

logilab.common.date.nb_open_days(start: Union[datetime.date, datetime.datetime], end: Union[datetime.date, datetime.datetime]) → int

logilab.common.date.next_month(somedate: datetime.date, nbmonth: int = 1) → datetime.date

logilab.common.date.previous_month(somedate, nbmonth=1)

logilab.common.date.str2date(datestr: str, sampledate: Union[datetime.date, datetime.datetime]) → Union[datetime.date, datetime.datetime]

logilab.common.date.strptime_time(value, format='%H:%M')

logilab.common.date.ticks2datetime(ticks: int) → datetime.datetime

logilab.common.date.timedelta_to_days(delta)

  return the time delta as a number of seconds

logilab.common.date.timedelta_to_seconds(delta)

  return the time delta as a fraction of days

logilab.common.date.todate(somedate: datetime.date) → datetime.date

  return a date from a date (leaving unchanged) or a datetime

logilab.common.date.todatetime(somedate)

  return a date from a date (leaving unchanged) or a datetime

logilab.common.date.totime(somedate)

  return a time from a time (leaving unchanged), date or datetime

logilab.common.date.ustrftime(somedate: datetime.datetime, fmt: str = '%Y-%m-%d') → str

  like strftime, but returns a unicode string instead of an encoded string which may be problematic with localized date.

logilab.common.date.utcdatetime(dt: datetime.datetime) → datetime.datetime

logilab.common.date.utctime(dt)

logilab.common.date.weekday(dateobj: Union[datetime.date, datetime.datetime]) → int

## logilab.common.debugger module

Customized version of pdb’s default debugger.

- sets up a history file

- uses ipython if available to colorize lines of code

- overrides list command to search for current block instead of using 5 lines of context

class logilab.common.debugger.Debugger(tcbk=None)

  Bases: `pdb.Pdb`

  custom debugger

  - sets up a history file

  - uses ipython if available to colorize lines of code

  - overrides list command to search for current block instead of using 5 lines of context

  attr_matches(text, namespace)

    implementation coming from rlcompleter.Completer.attr_matches Compute matches when text contains a dot.

    Assuming the text is of the form NAME.NAME….[NAME], and is evaluatable in self.namespace, it will be evaluated and its attributes (as revealed by dir()) are used as possible completions. (For class instances, class members are also considered.)

    WARNING: this can still invoke arbitrary C code, if an object with a __getattr__ hook is evaluated.

  complete_p(text, line, begin_idx, end_idx)

    provide variable names completion for the `p` command

  do_l(arg)

    overrides default list command to display the surrounding block instead of 5 lines of context

  do_list(arg)

    overrides default list command to display the surrounding block instead of 5 lines of context

  do_o(arg)

    opens source file corresponding to the current stack level

  do_open(arg)

    opens source file corresponding to the current stack level

  get_class_members(klass)

    implementation coming from rlcompleter.get_class_members

  set_quit()

    quit hook: save commands in the history file

  setup(frame, tcbk)

    setup hook: set up history file

  setup_history_file()

    if readline is available, read pdb history file

  start()

    starts the interactive mode

logilab.common.debugger.colorize(source, start_lineno, curlineno)

  fallback colorize function

logilab.common.debugger.colorize_source(source)

logilab.common.debugger.getsource(obj)

  Return the text of the source code for an object.

  The argument may be a module, class, method, function, traceback, frame, or code object. The source code is returned as a single string. An IOError is raised if the source code cannot be retrieved.

logilab.common.debugger.pm()

  use our custom debugger

logilab.common.debugger.set_trace()

## logilab.common.decorators module

A few useful function/method decorators.

logilab.common.decorators.cached(callableobj: Optional[Callable] = None, keyarg: Optional[int] = None, **kwargs: Any) → Union[Callable, logilab.common.decorators.cached_decorator]

  Simple decorator to cache result of method call.

class logilab.common.decorators.cached_decorator(cacheattr: Optional[str] = None, keyarg: Optional[int] = None)

  Bases: `object`

class logilab.common.decorators.cachedproperty(wrapped)

  Bases: `object`

  Provides a cached property equivalent to the stacking of @cached and @property, but more efficient.

  After first usage, the <property_name> becomes part of the object’s __dict__. Doing:

  > del obj.<property_name> empties the cache.

  Idea taken from the pyramid framework and the mercurial project.

  wrapped

class logilab.common.decorators.classproperty(get)

  Bases: `object`

  this is a simple property-like class but for class attributes.

logilab.common.decorators.clear_cache(obj, funcname)

  Clear a cache handled by the `cached()` decorator. If ‘x’ class has @cached on its method foo, type

      >>> clear_cache(x, 'foo')

  to purge this method’s cache on the instance.

logilab.common.decorators.copy_cache(obj, funcname, cacheobj)

  Copy cache for <funcname> from cacheobj to obj.

logilab.common.decorators.get_cache_impl(obj, funcname)

class logilab.common.decorators.iclassmethod(func)

  Bases: `object`

  Descriptor for method which should be available as class method if called on the class or instance method if called on an instance.

logilab.common.decorators.locked(acquire, release)

  Decorator taking two methods to acquire/release a lock as argument, returning a decorator function which will call the inner method after having called acquire(self) et will call release(self) afterwards.

logilab.common.decorators.monkeypatch(klass: type, methodname: Optional[str] = None) → Callable

  Decorator extending class with the decorated callable. This is basically a syntactic sugar vs class assignment.

      >>> class A:
      ...     pass
      >>> @monkeypatch(A)
      ... def meth(self):
      ...     return 12
      ...
      >>> a = A()
      >>> a.meth()
      12
      >>> @monkeypatch(A, 'foo')
      ... def meth(self):
      ...     return 12
      ...
      >>> a.foo()
      12

logilab.common.decorators.timed(f)

class logilab.common.decorators.wproperty(setfunc)

  Bases: `object`

  Simple descriptor expecting to take a modifier function as first argument and looking for a _<function name> to retrieve the attribute.

## logilab.common.deprecation module

Deprecation utilities.

class logilab.common.deprecation.CallableDeprecatedCallable(*args, **kwargs)

  Bases: `Protocol`

class logilab.common.deprecation.DeprecationWarningKind(value)

  Bases: `enum.Enum`

  An enumeration.

  ARGUMENT = 'argument'
  ATTRIBUTE = 'attribute'
  CALLABLE = 'callable'
  CLASS = 'class'
  MODULE = 'module'

class logilab.common.deprecation.DeprecationWarningOperation(value)

  Bases: `enum.Enum`

  An enumeration.

  DEPRECATED = 'deprecated'
  MOVED = 'moved'
  REMOVED = 'removed'
  RENAMED = 'renamed'

class logilab.common.deprecation.DeprecationWrapper(proxied: Any, msg: Optional[str] = None, version: Optional[str] = None)

  Bases: `object`

  proxy to print a warning on access to any attribute of the wrapped object

exception logilab.common.deprecation.StructuredDeprecationWarning(reason: str)

  Bases: `DeprecationWarning`

  Base class for all structured DeprecationWarning Mostly used with isinstance

exception logilab.common.deprecation.TargetDeprecatedDeprecationWarning(reason: str, kind: logilab.common.deprecation.DeprecationWarningKind)

  Bases: `logilab.common.deprecation.StructuredDeprecationWarning`

exception logilab.common.deprecation.TargetMovedDeprecationWarning(reason: str, kind: logilab.common.deprecation.DeprecationWarningKind, old_name: str, new_name: str, old_module: str, new_module: str)

  Bases: `logilab.common.deprecation.StructuredDeprecationWarning`

exception logilab.common.deprecation.TargetRemovedDeprecationWarning(reason: str, kind: logilab.common.deprecation.DeprecationWarningKind, name: str)

  Bases: `logilab.common.deprecation.StructuredDeprecationWarning`

exception logilab.common.deprecation.TargetRenamedDeprecationWarning(reason: str, kind: logilab.common.deprecation.DeprecationWarningKind, old_name: str, new_name: str)

  Bases: `logilab.common.deprecation.StructuredDeprecationWarning`

logilab.common.deprecation.argument_removed(old_argument_name: str, version: Optional[str] = None) → Callable

  callable decorator to allow getting backward compatibility for renamed keyword arguments.

      >>> @argument_removed("old")
      ... def some_function(new):
      ...     return new
      >>> some_function(old=42)
      sample.py:15: DeprecationWarning: argument old of callable some_function has been renamed and
      is deprecated, use keyword argument new instead some_function(old=42)
      42

logilab.common.deprecation.argument_renamed(old_name: str, new_name: str, version: Optional[str] = None) → Callable

  callable decorator to allow getting backward compatibility for renamed keyword arguments.

      >>> @argument_renamed(old_name="old", new_name="new")
      ... def some_function(new):
      ...     return new
      >>> some_function(old=42)
      sample.py:15: DeprecationWarning: argument old of callable some_function has been renamed and
      is deprecated, use keyword argument new instead
        some_function(old=42)
      42

logilab.common.deprecation.attribute_renamed(old_name: str, new_name: str, version: Optional[str] = None) → Callable

  class decorator to allow getting backward compatibility for renamed attributes.

      >>> @attribute_renamed(old_name="old", new_name="new")
      ... class SomeClass:
      ...     def __init__(self):
      ...         self.new = 42

      >>> some_class = SomeClass()
      >>> print(some_class.old)
      sample.py:15: DeprecationWarning: SomeClass.old has been renamed and is deprecated, use
      SomeClass.new instead
        print(some_class.old)
      42
      >>> some_class.old = 43
      sample.py:16: DeprecationWarning: SomeClass.old has been renamed and is deprecated, use
      SomeClass.new instead
        some_class.old = 43
      >>> some_class.old == some_class.new
      True

logilab.common.deprecation.callable_deprecated(reason: Optional[str] = None, version: Optional[str] = None, stacklevel: int = 2) → Callable

logilab.common.deprecation.callable_moved(module_name: str, object_name: str, version: Optional[str] = None, stacklevel: int = 2, new_name: Optional[str] = None) → Callable

logilab.common.deprecation.callable_renamed(old_name: str, new_function: Callable, version: Optional[str] = None) → Callable

  use to tell that a callable has been renamed.

  It returns a callable wrapper, so that when its called a warning is printed telling what is the object new name.

      >>> old_function = renamed('old_function', new_function)
      >>> old_function()
      sample.py:57: DeprecationWarning: old_function has been renamed and is deprecated, uses
      new_function instead old_function()
      >>>

logilab.common.deprecation.class_deprecated

  alias of `logilab.common.deprecation._generate_class_deprecated.<locals>._class_deprecated`

logilab.common.deprecation.class_moved(new_class: type, old_name: Optional[str] = None, message: Optional[str] = None, version: Optional[str] = None) → type

  nice wrapper around class_renamed when a class has been moved into another module

logilab.common.deprecation.class_renamed(old_name: str, new_class: type, message: Optional[str] = None, version: Optional[str] = None, module_name: Optional[str] = None, deprecated_warning_class=<class 'logilab.common.deprecation.TargetRenamedDeprecationWarning'>, deprecated_warning_kwargs=None) → type

  automatically creates a class which fires a DeprecationWarning when instantiated.

      >>> Set = class_renamed('Set', set, 'Set is now replaced by set')
      >>> s = Set()
      sample.py:57: DeprecationWarning: Set is now replaced by set
      s = Set()
      >>>

logilab.common.deprecation.deprecated(reason: Optional[str] = None, version: Optional[str] = None, stacklevel: int = 2) → Callable

logilab.common.deprecation.get_real__name__(some_callable: Callable) → str

  This is another super edge magic case which is needed because we uses lazy_wraps because of logilab.common.modutils.LazyObject and because __name__ has special behavior and doesn’t work like a normal attribute and that __getattribute__ of lazy_wraps is bypassed.

  Therefor, to get the real callable name when several lazy_wrapped decorator are used we need to travers the __wrapped__ attributes chain.

logilab.common.deprecation.lazy_wraps(wrapped: Callable) → Callable

  This is the equivalent of the @wraps decorator of functools except it won’t try to grabs attributes of the targeted function on decoration but on access.

  This is needed because of logilab.common.modutils.LazyObject.

  Indeed: if you try to decorate a LazyObject with @wraps, wraps will try to access attributes of LazyObject and this will trigger the attempt to import the module decorated by LazyObject which you don’t want to do when you just want to mark this LazyObject has been a deprecated objet that you only wants to trigger if the user try to use it.

  Usage: like @wraps()

      >>> @lazy_wraps(function)
      >>> def wrapper(*args, **kwargs): ...

logilab.common.deprecation.moved(module_name: str, object_name: str, version: Optional[str] = None, stacklevel: int = 2, new_name: Optional[str] = None) → Callable

logilab.common.deprecation.renamed(old_name: str, new_function: Callable, version: Optional[str] = None) → Callable

logilab.common.deprecation.send_warning(reason: str, deprecation_class: Type[DeprecationWarning], deprecation_class_kwargs: Dict[str, Any], version: Optional[str] = None, stacklevel: int = 2, module_name: Optional[str] = None) → None

  Display a deprecation message only if the version is older than the compatible version.

## logilab.common.fileutils module

File and file-path manipulation utilities.

group path manipulation

  first_level_directory, relative_path, is_binary,get_by_ext, remove_dead_links

group file manipulation

  norm_read, norm_open, lines, stream_lines, lines,write_open_mode, ensure_fs_mode, export

sort

  path manipulation, file manipulation

class logilab.common.fileutils.ProtectedFile(filepath: str, mode: str)

  Bases: `_io.FileIO`

  A special file-object class that automatically does a ‘chmod +w’ when needed.

  XXX: for now, the way it is done allows ‘normal file-objects’ to be created during the ProtectedFile object lifetime. One way to circumvent this would be to chmod / unchmod on each write operation.

  One other way would be to :

  - catch the IOError in the __init__

  - if IOError, then create a StringIO object

  - each write operation writes in this StringIO object

  - on close()/del(), write/append the StringIO content to the file and do the chmod only once

  close() → None

    restore mode before closing

exception logilab.common.fileutils.UnresolvableError

  Bases: `Exception`

  Exception raised by relative path when it’s unable to compute relative path between two paths.

logilab.common.fileutils.abspath_listdir(path)

  Lists path’s content using absolute paths.

logilab.common.fileutils.ensure_fs_mode(filepath, desired_mode=128)

  Check that the given file has the given mode(s) set, else try to set it.

  Parameters

    - filepath (str) – path of the file

    - desired_mode (int) – ORed flags describing the desired mode. Use constants from the stat module for file permission’s modes

logilab.common.fileutils.export(from_dir: str, to_dir: str, blacklist: Tuple[str, str, str, str, str, str, str, str] = ('CVS', '.svn', '.hg', '.git', '.tox', 'debian', 'dist', 'build'), ignore_ext: Tuple[str, str, str, str, str, str] = ('.pyc', '.pyo', '.elc', '~', '.swp', '.orig'), verbose: int = 0) → None

  Make a mirror of from_dir in to_dir, omitting directories and files listed in the black list or ending with one of the given extensions.

  Parameters

    - from_dir (str) – directory to export

    - to_dir (str) – destination directory

    - blacklist (list or tuple) – list of files or directories to ignore, default to the content of BASE_BLACKLIST

    - ignore_ext (list or tuple) – list of extensions to ignore, default to the content of IGNORED_EXTENSIONS

    - verbose (bool) – flag indicating whether information about exported files should be printed to stderr, default to False

logilab.common.fileutils.first_level_directory(path: str) → str

  Return the first level directory of a path.

      >>> first_level_directory('home/syt/work')
      'home'
      >>> first_level_directory('/home/syt/work')
      '/'
      >>> first_level_directory('work')
      'work'
      >>>

  Parameters

    path (str) – the path for which we want the first level directory

  Return type

    str

  Returns

    the first level directory appearing in path

logilab.common.fileutils.is_binary(filename: str) → int

  Return true if filename may be a binary file, according to it’s extension.

  Parameters

    filename (str) – the name of the file

  Return type

    bool

  Returns

    true if the file is a binary file (actually if it’s mime type isn’t beginning by text/)

logilab.common.fileutils.lines(path: str, comments: Optional[str] = None) → List[str]

  Return a list of non empty lines in the file located at path.

  Parameters

    - path (str) – path to the file

    - comments (str or None) – optional string which can be used to comment a line in the file (i.e. lines starting with this string won’t be returned)

  Return type

    list

  Returns

    a list of stripped line in the file, without empty and commented lines

  Warning

    at some point this function will probably return an iterator

logilab.common.fileutils.relative_path(from_file, to_file)

  Try to get a relative path from from_file to to_file (path will be absolute if to_file is an absolute file). This function is useful to create link in from_file to to_file. This typical use case is used in this function description.

  If both files are relative, they’re expected to be relative to the same directory.

      >>> relative_path( from_file='toto/index.html', to_file='index.html')
      '../index.html'
      >>> relative_path( from_file='index.html', to_file='toto/index.html')
      'toto/index.html'
      >>> relative_path( from_file='tutu/index.html', to_file='toto/index.html')
      '../toto/index.html'
      >>> relative_path( from_file='toto/index.html', to_file='/index.html')
      '/index.html'
      >>> relative_path( from_file='/toto/index.html', to_file='/index.html')
      '../index.html'
      >>> relative_path( from_file='/toto/index.html', to_file='/toto/summary.html')
      'summary.html'
      >>> relative_path( from_file='index.html', to_file='index.html')
      ''
      >>> relative_path( from_file='/index.html', to_file='toto/index.html')
      Traceback (most recent call last):
        File "<string>", line 1, in ?
        File "<stdin>", line 37, in relative_path
      UnresolvableError
      >>> relative_path( from_file='/index.html', to_file='/index.html')
      ''
      >>>

  Parameters

    - from_file (str) – source file (where links will be inserted)

    - to_file (str) – target file (on which links point)

  Raises

    UnresolvableError – if it has been unable to guess a correct path

  Return type

    str

  Returns

    the relative path of to_file from from_file

logilab.common.fileutils.remove_dead_links(directory, verbose=0)

  Recursively traverse directory and remove all dead links.

  Parameters

    - directory (str) – directory to cleanup

    - verbose (bool) – flag indicating whether information about deleted links should be printed to stderr, default to False

logilab.common.fileutils.stream_lines(stream: _io.TextIOWrapper, comments: Optional[str] = None) → List[str]

  Return a list of non empty lines in the given stream.

  Parameters

    - stream (object implementing 'xreadlines' or 'readlines') – file like object

    - comments (str or None) – optional string which can be used to comment a line in the file (i.e. lines starting with this string won’t be returned)

  Return type

    list

  Returns

    a list of stripped line in the file, without empty and commented lines

  Warning

    at some point this function will probably return an iterator

logilab.common.fileutils.write_open_mode(filename: str) → str

  Return the write mode that should used to open file.

  Parameters

    filename (str) – the name of the file

  Return type

    str

  Returns

    the mode that should be use to open the file (‘w’ or ‘wb’)

## logilab.common.graph module

Graph manipulation utilities.

(dot generation adapted from pypy/translator/tool/make_dot.py)

class logilab.common.graph.DotBackend(graphname, rankdir=None, size=None, ratio=None, charset='utf-8', renderer='dot', additionnal_param={})

  Bases: `object`

  Dot File backend.

  emit(line)

    Adds <line> to final output.

  emit_edge(name1, name2, **props)

    emit an edge from <name1> to <name2>. edge properties: see http://www.graphviz.org/doc/info/attrs.html

  emit_node(name, **props)

    emit a node with given properties. node properties: see http://www.graphviz.org/doc/info/attrs.html

  generate(outputfile=None, dotfile=None, mapfile=None)

    Generates a graph file.

    Parameters

      - outputfile – filename and path [defaults to graphname.png]

      - dotfile – filename and path [defaults to graphname.dot]

    Return type

      str

    Returns

      a path to the generated file

  get_source()

    returns self._source

  property source

    returns self._source

class logilab.common.graph.GraphGenerator(backend)

  Bases: `object`

  generate(visitor, propshdlr, outputfile=None, mapfile=None)

exception logilab.common.graph.UnorderableGraph

  Bases: `Exception`

logilab.common.graph.escape(value)

  Make <value> usable in a dot file.

logilab.common.graph.get_cycles(graph_dict: Dict[logilab.common.graph.V, List[logilab.common.graph.V]], vertices: Optional[Iterable] = None) → List[List]

  given a dictionary representing an ordered graph (i.e. key are vertices and values is a list of destination vertices representing edges), return a list of detected cycles

logilab.common.graph.has_path(graph_dict: Dict[str, List[str]], fromnode: str, tonode: str, path: Optional[List[str]] = None) → Optional[List[str]]

  generic function taking a simple graph definition as a dictionary, with node has key associated to a list of nodes directly reachable from it.

  Return None if no path exists to go from fromnode to tonode, else the first path found (as a list including the destination node at last)

logilab.common.graph.normalize_node_id(nid)

  Returns a suitable DOT node id for nid.

logilab.common.graph.ordered_nodes(graph: Dict[logilab.common.graph.V, List[logilab.common.graph.V]]) → Tuple[logilab.common.graph.V, ...]

  takes a dependency graph dict as arguments and return an ordered tuple of nodes starting with nodes without dependencies and up to the outermost node.

  If there is some cycle in the graph, `UnorderableGraph` will be raised.

  Also the given graph dict will be emptied.

logilab.common.graph.target_info_from_filename(filename)

  Transforms /some/path/foo.png into (‘/some/path’, ‘foo.png’, ‘png’).

## logilab.common.interface module

Bases class for interfaces to provide ‘light’ interface handling.

> TODO:

>   _ implements a check method which check that an object implements the interface _ Attribute objects

This module requires at least python 2.2

class logilab.common.interface.Interface

  Bases: `object`

  Base class for interfaces.

  classmethod is_implemented_by(instance: type) → bool

logilab.common.interface.extend(klass: type, interface: type, _recurs: bool = False) → None

  Add interface to klass’__implements__ if not already implemented in.

  If klass is subclassed, ensure subclasses __implements__ it as well.

  NOTE: klass should be e new class.

logilab.common.interface.implements(obj: type, interface: type) → bool

  Return true if the give object (maybe an instance or class) implements the interface.

## logilab.common.logging_ext module

Extends the logging module from the standard library.

class logilab.common.logging_ext.ColorFormatter(fmt=None, datefmt=None, colors=None)

  Bases: `logging.Formatter`

  A color Formatter for the logging standard module.

  By default, colorize CRITICAL and ERROR in red, WARNING in orange, INFO in green and DEBUG in yellow.

  self.colors is customizable via the ‘color’ constructor argument (dictionary).

  self.colorfilters is a list of functions that get the LogRecord and return a color name or None.

  format(record)

    Format the specified record as text.

    The record’s attribute dictionary is used as the operand to a string formatting operation which yields the returned string. Before formatting the dictionary, a couple of preparatory steps are carried out. The message attribute of the record is computed using LogRecord.getMessage(). If the formatting string uses the time (as determined by a call to usesTime(), formatTime() is called to format the event time. If there is exception information, it is formatted using formatException() and appended to the message.

logilab.common.logging_ext.get_formatter(logformat='%(asctime)s - (%(name)s) %(levelname)s: %(message)s', logdateformat='%Y-%m-%d %H:%M:%S')

logilab.common.logging_ext.get_handler(debug=False, syslog=False, logfile=None, rotation_parameters=None)

  get an apropriate handler according to given parameters

logilab.common.logging_ext.get_threshold(debug=False, logthreshold=None)

logilab.common.logging_ext.init_log(debug=False, syslog=False, logthreshold=None, logfile=None, logformat='%(asctime)s - (%(name)s) %(levelname)s: %(message)s', logdateformat='%Y-%m-%d %H:%M:%S', fmt=None, rotation_parameters=None, handler=None)

  init the log service

logilab.common.logging_ext.set_color_formatter(logger=None, **kw)

  Install a color formatter on the ‘logger’. If not given, it will defaults to the default logger.

  Any additional keyword will be passed as-is to the ColorFormatter constructor.

logilab.common.logging_ext.set_log_methods(cls, logger)

  bind standard logger’s methods as methods on the class

logilab.common.logging_ext.xxx_cyan(record)

## logilab.common.modutils module

Python modules manipulation utility functions.

type PY_SOURCE_EXTS

  tuple(str)

var PY_SOURCE_EXTS

  list of possible python source file extension

type STD_LIB_DIR

  str

var STD_LIB_DIR

  directory where standard modules are located

type BUILTIN_MODULES

  dict

var BUILTIN_MODULES

  dictionary with builtin module names as key

class logilab.common.modutils.LazyObject(module, obj)

  Bases: `object`

  This class allows to lazyly declare a object (most likely only a callable according to the code) from a module without importing it.

  The import will be triggered when the user tries to access attributes of the object/callable or call it.

  Trying to set or delete attributes of the wrapped object/callable will not works as expected.

exception logilab.common.modutils.NoSourceFile

  Bases: `Exception`

  exception raised when we are not able to get a python source file for a precompiled file

logilab.common.modutils.clean_sys_modules(names)

  remove submodules starting with name from names from sys.modules

logilab.common.modutils.cleanup_sys_modules(directories)

  remove submodules of directories from sys.modules

logilab.common.modutils.file_from_modpath(modpath: List[str], path: Optional[Any] = None, context_file: Optional[str] = None) → Optional[str]

  given a mod path (i.e. splitted module / package name), return the corresponding file, giving priority to source file over precompiled file if it exists

  Parameters

    - modpath (list or tuple) – splitted module’s name (i.e name of a module or package splitted on ‘.’) (this means explicit relative imports that start with dots have empty strings in this list!)

    - path (list or None) – optional list of path where the module or package should be searched (use sys.path if nothing or None is given)

    - context_file (str or None) – context file to consider, necessary if the identifier has been introduced using a relative import unresolvable in the actual context (i.e. modutils)

  Raises

    ImportError – if there is no such module in the directory

  Return type

    str or None

  Returns

    the path to the module’s file or None if it’s an integrated builtin module such as ‘sys’

logilab.common.modutils.get_module_files(src_directory: str, blacklist: Sequence[str] = ('CVS', '.svn', '.hg', '.git', '.tox', 'debian', 'dist', 'build')) → List[str]

  given a package directory return a list of all available python module’s files in the package and its subpackages

  Parameters

    - src_directory (str) – path of the directory corresponding to the package

    - blacklist (list or tuple) – optional list of files or directory to ignore, default to the value of logilab.common.STD_BLACKLIST

  Return type

    list

  Returns

    the list of all available python module’s files in the package and its subpackages

logilab.common.modutils.get_module_part(dotted_name: str, context_file: Optional[str] = None) → str

  given a dotted name return the module part of the name :

      >>> get_module_part('logilab.common.modutils.get_module_part')
      'logilab.common.modutils'

  Parameters

    - dotted_name (str) – full name of the identifier we are interested in

    - context_file (str or None) – context file to consider, necessary if the identifier has been introduced using a relative import unresolvable in the actual context (i.e. modutils)

  Raises

    ImportError – if there is no such module in the directory

  Return type

    str or None

  Returns

    the module part of the name or None if we have not been able at all to import the given name

  XXX: deprecated, since it doesn’t handle package precedence over module (see #10066)

logilab.common.modutils.get_modules(package: str, src_directory: str, blacklist: Sequence[str] = ('CVS', '.svn', '.hg', '.git', '.tox', 'debian', 'dist', 'build')) → List[str]

  given a package directory return a list of all available python modules in the package and its subpackages

  Parameters

    - package (str) – the python name for the package

    - src_directory (str) – path of the directory corresponding to the package

    - blacklist (list or tuple) – optional list of files or directory to ignore, default to the value of logilab.common.STD_BLACKLIST

  Return type

    list

  Returns

    the list of all available python modules in the package and its subpackages

logilab.common.modutils.get_source_file(filename: str, include_no_ext: bool = False) → str

  given a python module’s file name return the matching source file name (the filename will be returned identically if it’s a already an absolute path to a python source file…)

  Parameters

    filename (str) – python module’s file name

  Raises

    NoSourceFile – if no source file exists on the file system

  Return type

    str

  Returns

    the absolute path of the source file if it exists

logilab.common.modutils.is_python_source(filename)

  rtype: bool return: True if the filename is a python source file

logilab.common.modutils.is_relative(modname: str, from_file: str) → bool

  return true if the given module name is relative to the given file name

  Parameters

    - modname (str) – name of the module we are interested in

    - from_file (str) – path of the module from which modname has been imported

  Return type

    bool

  Returns

    true if the module has been imported relatively to from_file

logilab.common.modutils.is_standard_module(modname: str, std_path: Union[List[str], Tuple[str]] = ('/usr/lib/python3.9',)) → bool

  try to guess if a module is a standard python module (by default, see std_path parameter’s description)

  Parameters

    - modname (str) – name of the module we are interested in

    - std_path (list(str) or tuple(str)) – list of path considered as standard

  Return type

    bool

  Returns

    true if the module: - is located on the path listed in one of the directory in std_path - is a built-in module

  Note: this function is known to return wrong values when inside virtualenv. See https://www.logilab.org/ticket/294756.

logilab.common.modutils.load_module_from_file(filepath, path=None, use_sys=True, extrapath=None)

  Load a Python module from it’s path.

  Parameters

    - filepath (str) – path to the python module or package

    - path (list or None) – optional list of path where the module or package should be searched (use sys.path if nothing or None is given)

    - use_sys (bool) – boolean indicating whether the sys.modules dictionary should be used or not

  Raises

    ImportError – if the module or package is not found

  Return type

    module

  Returns

    the loaded module

logilab.common.modutils.load_module_from_modpath(parts: List[str], path: Optional[Any] = None, use_sys: int = True) → Optional[module]

  Load a python module from its splitted name.

  Parameters

    - parts (list(str) or tuple(str)) – python name of a module or package splitted on ‘.’

    - path (list or None) – optional list of path where the module or package should be searched (use sys.path if nothing or None is given)

    - use_sys (bool) – boolean indicating whether the sys.modules dictionary should be used or not

  Raises

    ImportError – if the module or package is not found

  Return type

    module

  Returns

    the loaded module

logilab.common.modutils.load_module_from_name(dotted_name: str, path: Optional[Any] = None, use_sys: int = True) → module

  Load a Python module from its name.

  Parameters

    - dotted_name (str) – python name of a module or package

    - path (list or None) – optional list of path where the module or package should be searched (use sys.path if nothing or None is given)

    - use_sys (bool) – boolean indicating whether the sys.modules dictionary should be used or not

  Raises

    ImportError – if the module or package is not found

  Return type

    module

  Returns

    the loaded module

## logilab.common.optik_ext module

Add an abstraction level to transparently import optik classes from optparse (python >= 2.3) or the optik package.

It also defines three new types for optik/optparse command line parser :

> - regexp argument of this type will be converted using re.compile

> - csv argument of this type will be converted using split(‘,’)

> - yn argument of this type will be true if ‘y’ or ‘yes’, false if ‘n’ or ‘no’

> - named argument of this type are in the form <NAME>=<VALUE> or <NAME>:<VALUE>

> - password argument of this type wont be converted but this is used by other tools such as interactive prompt for configuration to double check value and use an invisible field

> - multiple_choice same as default “choice” type but multiple choices allowed

> - file argument of this type wont be converted but checked that the given file exists

> - color argument of this type wont be converted but checked its either a named color or a color specified using hexadecimal notation (preceded by a #)

> - time argument of this type will be converted to a float value in seconds according to time units (ms, s, min, h, d)

> - bytes argument of this type will be converted to a float value in bytes according to byte units (b, kb, mb, gb, tb)

class logilab.common.optik_ext.Option(*opts: str, **attrs: Any)

  Bases: `optparse.Option`

  override optik.Option to add some new option types

  ATTRS = ['action', 'type', 'dest', 'default', 'nargs', 'const', 'choices', 'callback', 'callback_args', 'callback_kwargs', 'help', 'metavar', 'hide', 'level']
  TYPES = ('string', 'int', 'long', 'float', 'complex', 'choice', 'regexp', 'csv', 'yn', 'named', 'password', 'multiple_choice', 'file', 'color', 'time', 'bytes')
  TYPE_CHECKER = {'bytes': <function check_bytes>, 'choice': <function check_choice>, 'color': <function check_color>, 'complex': <function check_builtin>, 'csv': <function check_csv>, 'file': <function check_file>, 'float': <function check_builtin>, 'int': <function check_builtin>, 'long': <function check_builtin>, 'multiple_choice': <function check_csv>, 'named': <function check_named>, 'password': <function check_password>, 'regexp': <function check_regexp>, 'time': <function check_time>, 'yn': <function check_yn>}
  process(opt: str, value: str, values: optparse.Values, parser: optparse.OptionParser) → int

class logilab.common.optik_ext.OptionGroup(parser, title, description=None)

  Bases: `optparse.OptionContainer`

  destroy()

    see OptionParser.destroy().

  format_help(formatter)
  level = 0
  set_title(title)

class logilab.common.optik_ext.OptionParser(option_class: type = <class 'logilab.common.optik_ext.Option'>, *args: Any, **kwargs: Any)

  Bases: `optparse.OptionParser`

  override optik.OptionParser to use our Option class

  format_option_help(formatter: Optional[optparse.HelpFormatter] = None) → str

exception logilab.common.optik_ext.OptionValueError(msg)

  Bases: `optparse.OptParseError`

  Raised if an invalid option value is encountered on the command line.

class logilab.common.optik_ext.Values(defaults=None)

  Bases: `object`

  ensure_value(attr, value)
  read_file(filename, mode='careful')
  read_module(modname, mode='careful')

## logilab.common.optparser module

Extend OptionParser with commands.

Example:

    >>> parser = OptionParser()
    >>> parser.usage = '%prog COMMAND [options] <arg> ...'
    >>> parser.add_command('build', 'mymod.build')
    >>> parser.add_command('clean', run_clean, add_opt_clean)
    >>> run, options, args = parser.parse_command(sys.argv[1:])
    >>> return run(options, args[1:])

With mymod.build that defines two functions run and add_options

class logilab.common.optparser.OptionParser(*args, **kwargs)

  Bases: `optparse.OptionParser`

  add_command(name, mod_or_funcs, help='')

    name of the command, name of module or tuple of functions (run, add_options)

  parse_command(args)
  print_main_help()

## logilab.common.proc module

module providing: * process information (linux specific: rely on /proc) * a class for resource control (memory / time / cpu time)

This module doesn’t work on windows platforms (only tested on linux)

organization

  Logilab

exception logilab.common.proc.LineageMemoryError

  Bases: `logilab.common.proc.ResourceError`

  Error raised when the total amount of memory used by a process and it’s child is reached

  limit = 'Lineage total Memory'

class logilab.common.proc.MemorySentinel(interval, memory_limit, gpid=None)

  Bases: `threading.Thread`

  A class checking a process don’t use too much memory in a separated daemonic thread

  stop()

    stop ap

exception logilab.common.proc.NoSuchProcess

  Bases: `Exception`

class logilab.common.proc.ProcInfo(pid)

  Bases: `logilab.common.tree.Node`

  provide access to process information found in /proc

  age()

    return the age of the process

  lineage_memory_usage()
  memory_usage()

    return the memory usage of the process in Ko

  name()

    return the process name found in /proc/<pid>/stat

  status()

    return the list of fields found in /proc/<pid>/stat

  time(children=0)

    return the number of jiffies that this process has been scheduled in user and kernel mode

class logilab.common.proc.ProcInfoLoader

  Bases: `object`

  manage process information

  list_pids()

    return a list of existent process ids

  load(pid)

    get a ProcInfo object for a given pid

  load_all()

    load all processes information

class logilab.common.proc.ResourceController(max_cpu_time=None, max_time=None, max_memory=None, max_reprieve=60)

  Bases: `object`

  clean_limit()

    reinstall the old process limit

  setup_limit()

    set up the process limit

exception logilab.common.proc.ResourceError

  Bases: `Exception`

  Error raise when resource limit is reached

  limit = 'Unknown Resource Limit'

exception logilab.common.proc.TimeoutError

  Bases: `logilab.common.proc.ResourceError`

  Error raised when the process is running for to much time

  limit = 'Real Time'

exception logilab.common.proc.XCPUError

  Bases: `logilab.common.proc.ResourceError`

  Error raised when CPU Time limit is reached

  limit = 'CPU Time'

logilab.common.proc.proc_exists(pid)

  check the a pid is registered in /proc raise NoSuchProcess exception if not

## logilab.common.pytest module

logilab-pytest is a tool that eases test running and debugging.

To be able to use logilab-pytest, you should either write tests using the logilab.common.testlib’s framework or the unittest module of the Python’s standard library.

You can customize logilab-pytest’s behaviour by defining a `pytestconf.py` file somewhere in your test directory. In this file, you can add options or change the way tests are run.

To add command line options, you must define a `update_parser` function in your `pytestconf.py` file. The function must accept a single parameter that will be the OptionParser’s instance to customize.

If you wish to customize the tester, you’ll have to define a class named `CustomPyTester`. This class should extend the default PyTester class defined in the logilab.common.pytest module. Take a look at the PyTester and DjangoTester classes for more information about what can be done.

For instance, if you wish to add a custom -l option to specify a loglevel, you could define the following `pytestconf.py` file

    import logging
    from logilab.common.pytest import PyTester

    def update_parser(parser):
        parser.add_option('-l', '--loglevel', dest='loglevel', action='store',
                          choices=('debug', 'info', 'warning', 'error', 'critical'),
                          default='critical', help="the default log level possible choices are "
                          "('debug', 'info', 'warning', 'error', 'critical')")
        return parser

    class CustomPyTester(PyTester):
        def __init__(self, cvg, options):
            super(CustomPyTester, self).__init__(cvg, options)
            loglevel = options.loglevel.upper()
            logger = logging.getLogger('erudi')
            logger.setLevel(logging.getLevelName(loglevel))

In your TestCase class you can then get the value of a specific option with the `optval` method:

    class MyTestCase(TestCase):
        def test_foo(self):
            loglevel = self.optval('loglevel')
            # ...

You can also tag your tag your test for fine filtering

With those tag:

    from logilab.common.testlib import tag, TestCase

    class Exemple(TestCase):

        @tag('rouge', 'carre')
        def toto(self):
            pass

        @tag('carre', 'vert')
        def tata(self):
            pass

        @tag('rouge')
        def titi(test):
            pass

you can filter the function with a simple python expression

> - `toto` and `titi` match `rouge`

> - `toto`, `tata` and `titi`, match `rouge or carre`

> - `tata` and `titi` match``rouge ^ carre``

> - `titi` match `rouge and not carre`

class logilab.common.pytest.DjangoTester(cvg, options)

  Bases: `logilab.common.pytest.PyTester`

  after_testfile()
  before_testfile()
  load_django_settings(dirname)

    try to find project’s setting and load it

  testall(exitfirst=False)

    walks through current working directory, finds something which can be considered as a testdir and runs every test there

  testfile(filename, batchmode=False)

    runs every test in filename

    Parameters

      filename – an absolute path pointing to a unittest file

  testonedir(testdir, exitfirst=False)

    finds each testfile in the testdir and runs it

    return true when all tests has been executed, false if exitfirst and some test has failed.

class logilab.common.pytest.GlobalTestReport

  Bases: `object`

  this class holds global test statistics

  failed_to_test_module(filename)

    called when the test module could not be imported by unittest

  feed(filename, testresult, ttime, ctime)

    integrates new test information into internal statistics

  skip_module(filename)

class logilab.common.pytest.NonStrictTestLoader

  Bases: `unittest.loader.TestLoader`

  Overrides default testloader to be able to omit classname when specifying tests to run on command line.

  For example, if the file test_foo.py contains

      class FooTC(TestCase):
          def test_foo1(self): # ...
          def test_foo2(self): # ...
          def test_bar1(self): # ...

      class BarTC(TestCase):
          def test_bar2(self): # ...

  ‘python test_foo.py’ will run the 3 tests in FooTC ‘python test_foo.py FooTC’ will run the 3 tests in FooTC ‘python test_foo.py test_foo’ will run test_foo1 and test_foo2 ‘python test_foo.py test_foo1’ will run test_foo1 ‘python test_foo.py test_bar’ will run FooTC.test_bar1 and BarTC.test_bar2

  getTestCaseNames(testCaseClass: type) → List[str]

    Return a sorted sequence of method names found within testCaseClass

  loadTestsFromName(name, module=None)

    Return a suite of all test cases given a string specifier.

    The name may resolve either to a module, a test case class, a test method within a test case class, or a callable object which returns a TestCase or TestSuite instance.

    The method optionally resolves the names relative to a given module.

  loadTestsFromNames(names: List[str], module: Optional[type] = None) → unittest.suite.TestSuite

    Return a suite of all test cases found using the given sequence of string specifiers. See ‘loadTestsFromName()’.

  loadTestsFromSuite(module, suitename)

class logilab.common.pytest.PyTester(cvg, options)

  Bases: `object`

  encapsulates testrun logic

  property errcode
  get_errcode()
  set_errcode(errcode)
  show_report()

    prints the report and returns appropriate exitcode

  testall(exitfirst=False)

    walks through current working directory, finds something which can be considered as a testdir and runs every test there

  testfile(filename, batchmode=False)

    runs every test in filename

    Parameters

      filename – an absolute path pointing to a unittest file

  testonedir(testdir, exitfirst=False)

    finds each testfile in the testdir and runs it

    return true when all tests has been executed, false if exitfirst and some test has failed.

class logilab.common.pytest.SkipAwareTestProgram(module='__main__', defaultTest=None, batchmode=False, cvg=None, options=None, outstream=<_io.TextIOWrapper name='<stderr>' mode='w' encoding='utf-8'>)

  Bases: `unittest.main.TestProgram`

  USAGE = "Usage: %(progName)s [options] [test] [...]\n\nOptions:\n -h, --help Show this message\n -v, --verbose Verbose output\n -i, --pdb Enable test failure inspection\n -x, --exitfirst Exit on first failure\n -s, --skip skip test matching this pattern (no regexp for now)\n -q, --quiet Minimal output\n --color colorize tracebacks\n\n -m, --match Run only test whose tag match this pattern\n\n -P, --profile FILE: Run the tests using cProfile and saving results\n in FILE\n\nExamples:\n %(progName)s - run default set of tests\n %(progName)s MyTestSuite - run suite 'MyTestSuite'\n %(progName)s MyTestCase.testSomething - run MyTestCase.testSomething\n %(progName)s MyTestCase - run all 'test*' test methods\n in MyTestCase\n"
  parseArgs(argv)
  runTests()

class logilab.common.pytest.SkipAwareTestResult(stream: unittest.runner._WritelnDecorator, descriptions: bool, verbosity: int, exitfirst: bool = False, pdbmode: bool = False, cvg: Optional[Any] = None, colorize: bool = False)

  Bases: `unittest.runner.TextTestResult`

  addError(test, err)

    err -> (exc_type, exc, tcbk)

  addFailure(test, err)

    Called when an error has occurred. ‘err’ is a tuple of values as returned by sys.exc_info().

  addSkip(test, reason)

    Called when a test is skipped.

  descrs_for(flavour: str) → List[Tuple[int, str]]
  printErrorList(flavour, errors)
  printErrors() → None

    Called by TestRunner after test run

  printSkippedList() → None

class logilab.common.pytest.SkipAwareTextTestRunner(stream=<_io.TextIOWrapper name='<stderr>' mode='w' encoding='utf-8'>, verbosity=1, exitfirst=False, pdbmode=False, cvg=None, test_pattern=None, skipped_patterns=(), colorize=False, batchmode=False, options=None)

  Bases: `unittest.runner.TextTestRunner`

  does_match_tags(test: Callable) → bool
  run(test)

    Run the given test case or test suite.

logilab.common.pytest.call(self: Any, result: Optional[logilab.common.pytest.SkipAwareTestResult] = None, runcondition: Optional[Callable] = None, options: Optional[Any] = None) → None

logilab.common.pytest.defaultTestResult(self)

  return a new instance of the defaultTestResult

logilab.common.pytest.enable_dbc(*args)

  Without arguments, return True if contracts can be enabled and should be enabled (see option -d), return False otherwise.

  With arguments, return False if contracts can’t or shouldn’t be enabled, otherwise weave ContractAspect with items passed as arguments.

logilab.common.pytest.load_pytest_conf(path, parser)

  loads a `pytestconf.py` file and update default parser and / or tester.

logilab.common.pytest.make_parser()

  creates the OptionParser instance

logilab.common.pytest.parseargs(parser)

  Parse the command line and return (options processed), (options to pass to unittest_main()), (explicitfile or None).

logilab.common.pytest.project_root(parser, projdir=None)

  try to find project’s root and add it to sys.path

logilab.common.pytest.remove_local_modules_from_sys(testdir)

  remove all modules from cache that come from testdir

  This is used to avoid strange side-effects when using the testall() mode of pytest. For instance, if we run pytest on this tree:

      A/test/test_utils.py
      B/test/test_utils.py

  we have to clean sys.modules to make sure the correct test_utils module is ran in B

logilab.common.pytest.this_is_a_testdir(dirpath: str) → Optional[re.Match]

  returns True if filename seems to be a test directory

logilab.common.pytest.this_is_a_testfile(filename: str) → Optional[re.Match]

  returns True if filename seems to be a test file

## logilab.common.registry module

This module provides bases for predicates dispatching (the pattern in use here is similar to what’s refered as multi-dispatch or predicate-dispatch in the literature, though a bit different since the idea is to select across different implementation ‘e.g. classes), not to dispatch a message to a function or method. It contains the following classes:

- `RegistryStore`, the top level object which loads implementation objects and stores them into registries. You’ll usually use it to access registries and their contained objects;

- `Registry`, the base class which contains objects semantically grouped (for instance, sharing a same API, hence the ‘implementation’ name). You’ll use it to select the proper implementation according to a context. Notice you may use registries on their own without using the store.

Note

implementation objects are usually designed to be accessed through the registry and not by direct instantiation, besides to use it as base classe.

The selection procedure is delegated to a selector, which is responsible for scoring the object according to some context. At the end of the selection, if an implementation has been found, an instance of this class is returned. A selector is built from one or more predicates combined together using AND, OR, NOT operators (actually &, | and ~). You’ll thus find some base classes to build predicates:

- `Predicate`, the abstract base predicate class

- `AndPredicate`, `OrPredicate`, `NotPredicate`, which you shouldn’t have to use directly. You’ll use &, | and ‘~’ operators between predicates directly

- `objectify_predicate()`

You’ll eventually find one concrete predicate: `yes`

class logilab.common.registry.RegistryStore(debugmode: bool = False)

  This class is responsible for loading objects and storing them in their registry which is created on the fly as needed.

  It handles dynamic registration of objects and provides a convenient api to access them. To be recognized as an object that should be stored into one of the store’s registry (`Registry`), an object must provide the following attributes, used control how they interact with the registry:

  `__registries__`

    list of registry names (string like ‘views’, ‘templates’…) into which the object should be registered

  `__regid__`

    object identifier in the registry (string like ‘main’, ‘primary’, ‘folder_box’)

  `__select__`

    the object predicate selectors

  Moreover, the `__abstract__` attribute may be set to True to indicate that an object is abstract and should not be registered (such inherited attributes not considered).

  Note

  When using the store to load objects dynamically, you always have to use super() to get the methods and attributes of the superclasses, and not use the class identifier. If not, you’ll get into trouble at reload time.

  For example, instead of writing:

      class Thing(Parent):
          __regid__ = 'athing'
          __select__ = yes()

          def f(self, arg1):
              Parent.f(self, arg1)

  You must write:

      class Thing(Parent):
          __regid__ = 'athing'
          __select__ = yes()

          def f(self, arg1):
              super(Thing, self).f(arg1)

  Dynamic loading is triggered by calling the `register_modnames()` method, given a list of modules names to inspect.

  register_modnames(modnames: List[str]) → None

    register all objects found in <modnames>

  For each module, by default, all compatible objects are registered automatically. However if some objects come as replacement of other objects, or have to be included only if some condition is met, you’ll have to define a registration_callback(vreg) function in the module and explicitly register all objects in this module, using the api defined below.

  register_all(objects: Iterable, modname: str, butclasses: Sequence = ()) → None

    register registrable objects into objects.

    Registrable objects are properly configured subclasses of `RegistrableObject`. Objects which are not defined in the module modname or which are in butclasses won’t be registered.

    Typical usage is:

        store.register_all(globals().values(), __name__, (ClassIWantToRegisterExplicitly,))

    So you get partially automatic registration, keeping manual registration for some object (to use `register_and_replace()` for instance).

  register_and_replace(obj, replaced, registryname=None)

    register obj object into registryname or obj.__registries__ if not specified. If found, the replaced object will be unregistered first (else a warning will be issued as it is generally unexpected).

  register(obj: Any, registryname: Optional[Any] = None, oid: Optional[Any] = None, clear: bool = False) → None

    register obj implementation into registryname or obj.__registries__ if not specified, with identifier oid or obj.__regid__ if not specified.

    If clear is true, all objects with the same identifier will be previously unregistered.

  unregister(obj, registryname=None)

    unregister obj object from the registry registryname or obj.__registries__ if not specified.

  Note

  Once the function registration_callback(vreg) is implemented in a module, all the objects from this module have to be explicitly registered as it disables the automatic object registration.

  Examples:

      def registration_callback(store):
         # register everything in the module except BabarClass
         store.register_all(globals().values(), __name__, (BabarClass,))

         # conditionally register BabarClass
         if 'babar_relation' in store.schema:
             store.register(BabarClass)

  In this example, we register all application object classes defined in the module except BabarClass. This class is then registered only if the ‘babar_relation’ relation type is defined in the instance schema.

      def registration_callback(store):
         store.register(Elephant)
         # replace Babar by Celeste
         store.register_and_replace(Celeste, Babar)

  In this example, we explicitly register classes one by one:

  - the Elephant class

  - the Celeste to replace Babar

  If at some point we register a new appobject class in this module, it won’t be registered at all without modification to the registration_callback implementation. The first example will register it though, thanks to the call to the register_all method.

  The REGISTRY_FACTORY class dictionary allows to specify which class should be instantiated for a given registry name. The class associated to None key will be the class used when there is no specific class for a name.

class logilab.common.registry.Registry(debugmode: bool)

  The registry store a set of implementations associated to identifier:

  - to each identifier are associated a list of implementations

  - to select an implementation of a given identifier, you should use one of the `select()` or `select_or_none()` method

  - to select a list of implementations for a context, you should use the `possible_objects()` method

  - dictionary like access to an identifier will return the bare list of implementations for this identifier.

  To be usable in a registry, the only requirement is to have a __select__ attribute.

  At the end of the registration process, the `__registered__()` method is called on each registered object which have them, given the registry in which it’s registered as argument.

  Registration methods:

  register(obj: Any, oid: Optional[Any] = None, clear: bool = False) → None

    base method to add an object in the registry

  unregister(obj)

    remove object <obj> from this registry

  Selection methods:

  select(_Registry__oid, *args, **kwargs)

    return the most specific object among those with the given oid according to the given context.

    raise `ObjectNotFound` if there are no object with id oid in this registry

    raise `NoSelectableObject` if no object can be selected

  select_or_none(_Registry__oid, *args, **kwargs)

    return the most specific object among those with the given oid according to the given context, or None if no object applies.

  possible_objects(*args, **kwargs)

    return an iterator on possible objects in this registry for the given context

  object_by_id(oid, *args, **kwargs)

    return object with the oid identifier. Only one object is expected to be found.

    raise `ObjectNotFound` if there are no object with id oid in this registry

    raise `AssertionError` if there is more than one object there

### Predicates

class logilab.common.registry.Predicate

  base class for selector classes providing implementation for operators `&`, `|` and `~`

  This class is only here to give access to binary operators, the selector logic itself should be implemented in the `__call__()` method. Notice it should usually accept any arbitrary arguments (the context), though that may vary depending on your usage of the registry.

  a selector is called to help choosing the correct object for a particular context by returning a score (int) telling how well the implementation given as first argument fit to the given context.

  0 score means that the class doesn’t apply.

logilab.common.registry.objectify_predicate(selector_func: Callable) → Any

  Most of the time, a simple score function is enough to build a selector. The `objectify_predicate()` decorator turn it into a proper selector class:

      @objectify_predicate
      def one(cls, req, rset=None, **kwargs):
          return 1

      class MyView(View):
          __select__ = View.__select__ & one()

class logilab.common.registry.yes(score: float = 0.5)

  Return the score given as parameter, with a default score of 0.5 so any other selector take precedence.

  Usually used for objects which can be selected whatever the context, or also sometimes to add arbitrary points to a score.

  Take care, yes(0) could be named ‘no’…

class logilab.common.registry.AndPredicate(*selectors: Any)

  and-chained selectors

class logilab.common.registry.OrPredicate(*selectors: Any)

  or-chained selectors

class logilab.common.registry.NotPredicate(selector)

  negation selector

### Debugging

class logilab.common.registry.traced_selection(traced='all')

  Typical usage is :

      >>> from logilab.common.registry import traced_selection
      >>> with traced_selection():
      ...     # some code in which you want to debug selectors
      ...     # for all objects

  This will yield lines like this in the logs:

      selector one_line_rset returned 0 for <class 'elephant.Babar'>

  You can also give to `traced_selection` the identifiers of objects on which you want to debug selection (‘oid1’ and ‘oid2’ in the example above).

      >>> with traced_selection( ('regid1', 'regid2') ):
      ...     # some code in which you want to debug selectors
      ...     # for objects with __regid__ 'regid1' and 'regid2'

  A potentially useful point to set up such a tracing function is the logilab.common.registry.Registry.select method body.

### Exceptions

class logilab.common.registry.RegistryException

  Base class for registry exception.

class logilab.common.registry.RegistryNotFound

  Raised when an unknown registry is requested.

  This is usually a programming/typo error.

class logilab.common.registry.ObjectNotFound

  Raised when an unregistered object is requested.

  This may be a programming/typo or a misconfiguration error.

class logilab.common.registry.NoSelectableObject(args, kwargs, objects)

  Raised when no object is selectable for a given context.

class logilab.common.registry.AndPredicate(*selectors: Any)

  Bases: `logilab.common.registry.MultiPredicate`

  and-chained selectors

class logilab.common.registry.MultiPredicate(*selectors: Any)

  Bases: `logilab.common.registry.Predicate`

  base class for compound selector classes

  classmethod merge_selectors(selectors: Sequence[logilab.common.registry.Predicate]) → List[logilab.common.registry.Predicate]

    deal with selector instanciation when necessary and merge multi-selectors if possible:

    AndPredicate(AndPredicate(sel1, sel2), AndPredicate(sel3, sel4)) ==> AndPredicate(sel1, sel2, sel3, sel4)

  search_selector(selector: logilab.common.registry.Predicate) → Optional[logilab.common.registry.Predicate]

    search for the given selector or selector instance (or tuple of selectors) in the selectors tree. Return None if not found

exception logilab.common.registry.NoSelectableObject(args, kwargs, objects)

  Bases: `logilab.common.registry.RegistryException`

  Raised when no object is selectable for a given context.

class logilab.common.registry.NotPredicate(selector)

  Bases: `logilab.common.registry.Predicate`

  negation selector

exception logilab.common.registry.ObjectNotFound

  Bases: `logilab.common.registry.RegistryException`

  Raised when an unregistered object is requested.

  This may be a programming/typo or a misconfiguration error.

class logilab.common.registry.OrPredicate(*selectors: Any)

  Bases: `logilab.common.registry.MultiPredicate`

  or-chained selectors

class logilab.common.registry.Predicate

  Bases: `object`

  base class for selector classes providing implementation for operators `&`, `|` and `~`

  This class is only here to give access to binary operators, the selector logic itself should be implemented in the `__call__()` method. Notice it should usually accept any arbitrary arguments (the context), though that may vary depending on your usage of the registry.

  a selector is called to help choosing the correct object for a particular context by returning a score (int) telling how well the implementation given as first argument fit to the given context.

  0 score means that the class doesn’t apply.

  property func_name
  search_selector(selector: logilab.common.registry.Predicate) → Optional[logilab.common.registry.Predicate]

    search for the given selector, selector instance or tuple of selectors in the selectors tree. Return None if not found.

class logilab.common.registry.PredicateMetaClass(*args, **kwargs)

  Bases: `type`

class logilab.common.registry.RegistrableInstance(*args, **kwargs)

  Bases: `logilab.common.registry.RegistrableObject`

  Inherit this class if you want instances of the classes to be automatically registered.

class logilab.common.registry.RegistrableObject

  Bases: `object`

  This is the base class for registrable objects which are selected according to a context.

  `__registry__`

    name of the registry for this object (string like ‘views’, ‘templates’…). You may want to define __registries__ directly if your object should be registered in several registries.

  `__regid__`

    object’s identifier in the registry (string like ‘main’, ‘primary’, ‘folder_box’)

  `__select__`

    class’selector

  Moreover, the __abstract__ attribute may be set to True to indicate that a class is abstract and should not be registered.

  You don’t have to inherit from this class to put it in a registry (having __regid__ and __select__ is enough), though this is needed for classes that should be automatically registered.

class logilab.common.registry.Registry(debugmode: bool)

  Bases: `dict`

  The registry store a set of implementations associated to identifier:

  - to each identifier are associated a list of implementations

  - to select an implementation of a given identifier, you should use one of the `select()` or `select_or_none()` method

  - to select a list of implementations for a context, you should use the `possible_objects()` method

  - dictionary like access to an identifier will return the bare list of implementations for this identifier.

  To be usable in a registry, the only requirement is to have a __select__ attribute.

  At the end of the registration process, the `__registered__()` method is called on each registered object which have them, given the registry in which it’s registered as argument.

  Registration methods:

  register(obj: Any, oid: Optional[Any] = None, clear: bool = False) → None

    base method to add an object in the registry

  unregister(obj)

    remove object <obj> from this registry

  Selection methods:

  select(_Registry__oid, *args, **kwargs)

    return the most specific object among those with the given oid according to the given context.

    raise `ObjectNotFound` if there are no object with id oid in this registry

    raise `NoSelectableObject` if no object can be selected

  select_or_none(_Registry__oid, *args, **kwargs)

    return the most specific object among those with the given oid according to the given context, or None if no object applies.

  possible_objects(*args, **kwargs)

    return an iterator on possible objects in this registry for the given context

  object_by_id(oid, *args, **kwargs)

    return object with the oid identifier. Only one object is expected to be found.

    raise `ObjectNotFound` if there are no object with id oid in this registry

    raise `AssertionError` if there is more than one object there

  add_select_best_listener(listener)

    Add a listener to the list of one parameters callables (function/method) that will be called everytime the selection Registry._select_best is done and they will recieve a dict of the following form:

        {"all_objects": [], "end_score": 0, "winners": [], "winner": None or winner,
        "self": self, "args": args, "kwargs": kwargs, }

  all_objects()

    return a list containing all objects in this registry.

  critical(msg, *args, **kwargs)

    Log ‘msg % args’ with severity ‘CRITICAL’.

    To pass exception information, use the keyword argument exc_info with a true value, e.g.

    logger.critical(“Houston, we have a %s”, “major disaster”, exc_info=1)

  debug(msg, *args, **kwargs)

    Log ‘msg % args’ with severity ‘DEBUG’.

    To pass exception information, use the keyword argument exc_info with a true value, e.g.

    logger.debug(“Houston, we have a %s”, “thorny problem”, exc_info=1)

  error(msg, *args, **kwargs)

    Log ‘msg % args’ with severity ‘ERROR’.

    To pass exception information, use the keyword argument exc_info with a true value, e.g.

    logger.error(“Houston, we have a %s”, “major problem”, exc_info=1)

  exception(msg, *args, exc_info=True, **kwargs)

    Convenience method for logging an ERROR with exception information.

  info(msg, *args, **kwargs)

    Log ‘msg % args’ with severity ‘INFO’.

    To pass exception information, use the keyword argument exc_info with a true value, e.g.

    logger.info(“Houston, we have a %s”, “interesting problem”, exc_info=1)

  initialization_completed() → None

    call method __registered__() on registered objects when the callback is defined

  object_by_id(oid, *args, **kwargs)

    return object with the oid identifier. Only one object is expected to be found.

    raise `ObjectNotFound` if there are no object with id oid in this registry

    raise `AssertionError` if there is more than one object there

  classmethod objid(obj: Any) → str

    returns a unique identifier for an object stored in the registry

  classmethod objname(obj: Any) → str

    returns a readable name for an object stored in the registry

  possible_objects(*args, **kwargs)

    return an iterator on possible objects in this registry for the given context

  register(obj: Any, oid: Optional[Any] = None, clear: bool = False) → None

    base method to add an object in the registry

  register_and_replace(obj, replaced)

    remove <replaced> and register <obj>

  select(_Registry__oid, *args, **kwargs)

    return the most specific object among those with the given oid according to the given context.

    raise `ObjectNotFound` if there are no object with id oid in this registry

    raise `NoSelectableObject` if no object can be selected

  select_or_none(_Registry__oid, *args, **kwargs)

    return the most specific object among those with the given oid according to the given context, or None if no object applies.

  selected(winner, args, kwargs)

    override here if for instance you don’t want “instanciation”

  unregister(obj)

    remove object <obj> from this registry

  warning(msg, *args, **kwargs)

    Log ‘msg % args’ with severity ‘WARNING’.

    To pass exception information, use the keyword argument exc_info with a true value, e.g.

    logger.warning(“Houston, we have a %s”, “bit of a problem”, exc_info=1)

exception logilab.common.registry.RegistryException

  Bases: `Exception`

  Base class for registry exception.

exception logilab.common.registry.RegistryNotFound

  Bases: `logilab.common.registry.RegistryException`

  Raised when an unknown registry is requested.

  This is usually a programming/typo error.

class logilab.common.registry.RegistryStore(debugmode: bool = False)

  Bases: `dict`

  This class is responsible for loading objects and storing them in their registry which is created on the fly as needed.

  It handles dynamic registration of objects and provides a convenient api to access them. To be recognized as an object that should be stored into one of the store’s registry (`Registry`), an object must provide the following attributes, used control how they interact with the registry:

  `__registries__`

    list of registry names (string like ‘views’, ‘templates’…) into which the object should be registered

  `__regid__`

    object identifier in the registry (string like ‘main’, ‘primary’, ‘folder_box’)

  `__select__`

    the object predicate selectors

  Moreover, the `__abstract__` attribute may be set to True to indicate that an object is abstract and should not be registered (such inherited attributes not considered).

  Note

  When using the store to load objects dynamically, you always have to use super() to get the methods and attributes of the superclasses, and not use the class identifier. If not, you’ll get into trouble at reload time.

  For example, instead of writing:

      class Thing(Parent):
          __regid__ = 'athing'
          __select__ = yes()

          def f(self, arg1):
              Parent.f(self, arg1)

  You must write:

      class Thing(Parent):
          __regid__ = 'athing'
          __select__ = yes()

          def f(self, arg1):
              super(Thing, self).f(arg1)

  Dynamic loading is triggered by calling the `register_modnames()` method, given a list of modules names to inspect.

  register_modnames(modnames: List[str]) → None

    register all objects found in <modnames>

  For each module, by default, all compatible objects are registered automatically. However if some objects come as replacement of other objects, or have to be included only if some condition is met, you’ll have to define a registration_callback(vreg) function in the module and explicitly register all objects in this module, using the api defined below.

  register_all(objects: Iterable, modname: str, butclasses: Sequence = ()) → None

    register registrable objects into objects.

    Registrable objects are properly configured subclasses of `RegistrableObject`. Objects which are not defined in the module modname or which are in butclasses won’t be registered.

    Typical usage is:

        store.register_all(globals().values(), __name__, (ClassIWantToRegisterExplicitly,))

    So you get partially automatic registration, keeping manual registration for some object (to use `register_and_replace()` for instance).

  register_and_replace(obj, replaced, registryname=None)

    register obj object into registryname or obj.__registries__ if not specified. If found, the replaced object will be unregistered first (else a warning will be issued as it is generally unexpected).

  register(obj: Any, registryname: Optional[Any] = None, oid: Optional[Any] = None, clear: bool = False) → None

    register obj implementation into registryname or obj.__registries__ if not specified, with identifier oid or obj.__regid__ if not specified.

    If clear is true, all objects with the same identifier will be previously unregistered.

  unregister(obj, registryname=None)

    unregister obj object from the registry registryname or obj.__registries__ if not specified.

  Note

  Once the function registration_callback(vreg) is implemented in a module, all the objects from this module have to be explicitly registered as it disables the automatic object registration.

  Examples:

      def registration_callback(store):
         # register everything in the module except BabarClass
         store.register_all(globals().values(), __name__, (BabarClass,))

         # conditionally register BabarClass
         if 'babar_relation' in store.schema:
             store.register(BabarClass)

  In this example, we register all application object classes defined in the module except BabarClass. This class is then registered only if the ‘babar_relation’ relation type is defined in the instance schema.

      def registration_callback(store):
         store.register(Elephant)
         # replace Babar by Celeste
         store.register_and_replace(Celeste, Babar)

  In this example, we explicitly register classes one by one:

  - the Elephant class

  - the Celeste to replace Babar

  If at some point we register a new appobject class in this module, it won’t be registered at all without modification to the registration_callback implementation. The first example will register it though, thanks to the call to the register_all method.

  The REGISTRY_FACTORY class dictionary allows to specify which class should be instantiated for a given registry name. The class associated to None key will be the class used when there is no specific class for a name.

  REGISTRY_FACTORY: Dict[Union[None, str], type] = {None: <class 'logilab.common.registry.Registry'>}
  critical(msg, *args, **kwargs)

    Log ‘msg % args’ with severity ‘CRITICAL’.

    To pass exception information, use the keyword argument exc_info with a true value, e.g.

    logger.critical(“Houston, we have a %s”, “major disaster”, exc_info=1)

  debug(msg, *args, **kwargs)

    Log ‘msg % args’ with severity ‘DEBUG’.

    To pass exception information, use the keyword argument exc_info with a true value, e.g.

    logger.debug(“Houston, we have a %s”, “thorny problem”, exc_info=1)

  error(msg, *args, **kwargs)

    Log ‘msg % args’ with severity ‘ERROR’.

    To pass exception information, use the keyword argument exc_info with a true value, e.g.

    logger.error(“Houston, we have a %s”, “major problem”, exc_info=1)

  exception(msg, *args, exc_info=True, **kwargs)

    Convenience method for logging an ERROR with exception information.

  info(msg, *args, **kwargs)

    Log ‘msg % args’ with severity ‘INFO’.

    To pass exception information, use the keyword argument exc_info with a true value, e.g.

    logger.info(“Houston, we have a %s”, “interesting problem”, exc_info=1)

  init_registration(path: List[str], extrapath: Optional[Any] = None) → List[Tuple[str, str]]

    reset registry and walk down path to return list of (path, name) file modules to be loaded

  initialization_completed() → None

    call initialization_completed() on all known registries

  classmethod is_registrable(obj: Any) → bool

    ensure obj should be registered

    as arbitrary stuff may be registered, do a lot of check and warn about weird cases (think to dumb proxy objects)

  is_reload_needed(path)

    return True if something module changed and the registry should be reloaded

  load_file(filepath: str, modname: str) → None

    load registrable objects (if any) from a python file

  load_module(module: module) → None

    Automatically handle module objects registration.

    Instances are registered as soon as they are hashable and have the following attributes:

    - __regid__ (a string)

    - __select__ (a callable)

    - __registries__ (a tuple/list of string)

    For classes this is a bit more complicated :

    - first ensure parent classes are already registered

    - class with __abstract__ == True in their local dictionary are skipped

    - object class needs to have registries and identifier properly set to a non empty string to be registered.

  register(obj: Any, registryname: Optional[Any] = None, oid: Optional[Any] = None, clear: bool = False) → None

    register obj implementation into registryname or obj.__registries__ if not specified, with identifier oid or obj.__regid__ if not specified.

    If clear is true, all objects with the same identifier will be previously unregistered.

  register_all(objects: Iterable, modname: str, butclasses: Sequence = ()) → None

    register registrable objects into objects.

    Registrable objects are properly configured subclasses of `RegistrableObject`. Objects which are not defined in the module modname or which are in butclasses won’t be registered.

    Typical usage is:

        store.register_all(globals().values(), __name__, (ClassIWantToRegisterExplicitly,))

    So you get partially automatic registration, keeping manual registration for some object (to use `register_and_replace()` for instance).

  register_and_replace(obj, replaced, registryname=None)

    register obj object into registryname or obj.__registries__ if not specified. If found, the replaced object will be unregistered first (else a warning will be issued as it is generally unexpected).

  register_modnames(modnames: List[str]) → None

    register all objects found in <modnames>

  register_objects(path: List[str], extrapath: Optional[Any] = None) → Callable
  registry_class(regid: str) → type

    return existing registry named regid or use factory to create one and return it

  reset() → None

    clear all registries managed by this store

  setdefault(regid: str) → logilab.common.registry.Registry

    Insert key with a value of default if key is not in the dictionary.

    Return the value for key if key is in the dictionary, else default.

  unregister(obj, registryname=None)

    unregister obj object from the registry registryname or obj.__registries__ if not specified.

  warning(msg, *args, **kwargs)

    Log ‘msg % args’ with severity ‘WARNING’.

    To pass exception information, use the keyword argument exc_info with a true value, e.g.

    logger.warning(“Houston, we have a %s”, “bit of a problem”, exc_info=1)

exception logilab.common.registry.SelectAmbiguity

  Bases: `logilab.common.registry.RegistryException`

  Raised when several objects compete at selection time with an equal score.

class logilab.common.registry.SelectBestReport

  Bases: `dict`

  all_objects: List
  args: List
  end_score: int
  kwargs: Dict
  registry: logilab.common.registry.Registry
  self: logilab.common.registry.Registry
  winner: Optional[Any]
  winners: List

logilab.common.registry.obj_registries(cls: Any, registryname: Optional[Any] = None) → Tuple[str]

  return a tuple of registry names (see __registries__)

logilab.common.registry.objectify_predicate(selector_func: Callable) → Any

  Most of the time, a simple score function is enough to build a selector. The `objectify_predicate()` decorator turn it into a proper selector class:

      @objectify_predicate
      def one(cls, req, rset=None, **kwargs):
          return 1

      class MyView(View):
          __select__ = View.__select__ & one()

class logilab.common.registry.traced_selection(traced='all')

  Bases: `object`

  Typical usage is :

      >>> from logilab.common.registry import traced_selection
      >>> with traced_selection():
      ...     # some code in which you want to debug selectors
      ...     # for all objects

  This will yield lines like this in the logs:

      selector one_line_rset returned 0 for <class 'elephant.Babar'>

  You can also give to `traced_selection` the identifiers of objects on which you want to debug selection (‘oid1’ and ‘oid2’ in the example above).

      >>> with traced_selection( ('regid1', 'regid2') ):
      ...     # some code in which you want to debug selectors
      ...     # for objects with __regid__ 'regid1' and 'regid2'

  A potentially useful point to set up such a tracing function is the logilab.common.registry.Registry.select method body.

logilab.common.registry.wrap_predicates(decorator: Callable) → None

class logilab.common.registry.yes(score: float = 0.5)

  Bases: `logilab.common.registry.Predicate`

  Return the score given as parameter, with a default score of 0.5 so any other selector take precedence.

  Usually used for objects which can be selected whatever the context, or also sometimes to add arbitrary points to a score.

  Take care, yes(0) could be named ‘no’…

## logilab.common.shellutils module

shell/term utilities, useful to write some python scripts instead of shell scripts.

class logilab.common.shellutils.DummyProgressBar

  Bases: `object`

  finish()
  refresh()
  text
  update()

class logilab.common.shellutils.ProgressBar(nbops: int, size: int = 20, stream: _io.StringIO = <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>, title: str = '')

  Bases: `object`

  A simple text progression bar.

  finish()
  refresh() → None

    Refresh the progression bar display.

  property text
  update(offset: int = 1, exact: bool = False) → None

    Move FORWARD to new cursor position (cursor will never go backward).

    Offset

      fraction of `size`

    Exact

      - False: offset relative to current cursor position if True

      - True: offset as an asbsolute position

class logilab.common.shellutils.RawInput(input_function: Optional[Callable] = None, printer: Optional[Callable] = None, **kwargs: Any)

  Bases: `object`

  ask(question: str, options: Tuple[str, ...], default: str) → str
  confirm(question: str, default_is_yes: bool = True) → bool

logilab.common.shellutils.chown(path, login=None, group=None)

  Same as os.chown function but accepting user login or group name as argument. If login or group is omitted, it’s left unchanged.

  Note: you must own the file to chown it (or be root). Otherwise OSError is raised.

logilab.common.shellutils.cp(source, destination)

  A shell-like cp, supporting wildcards.

logilab.common.shellutils.find(directory: str, exts: Union[Tuple[str, ...], str], exclude: bool = False, blacklist: Tuple[str, ...] = ('CVS', '.svn', '.hg', '.git', '.tox', 'debian', 'dist', 'build')) → List[str]

  Recursively find files ending with the given extensions from the directory.

  Parameters

    - directory (str) – directory where the search should start

    - exts (basestring or list or tuple) – extensions or lists or extensions to search

    - exts – if this argument is True, returning files NOT ending with the given extensions

    - blacklist (list or tuple) – optional list of files or directory to ignore, default to the value of logilab.common.STD_BLACKLIST

  Return type

    list

  Returns

    the list of all matching files

logilab.common.shellutils.generate_password(length=8, vocab='abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789')

  dumb password generation function

logilab.common.shellutils.getlogin()

  avoid using os.getlogin() because of strange tty / stdin problems (man 3 getlogin) Another solution would be to use $LOGNAME, $USER or $USERNAME

logilab.common.shellutils.globfind(directory: str, pattern: str, blacklist: Tuple[str, str, str, str, str, str, str, str] = ('CVS', '.svn', '.hg', '.git', '.tox', 'debian', 'dist', 'build')) → Iterator[str]

  Recursively finds files matching glob pattern under directory.

  This is an alternative to logilab.common.shellutils.find.

  Parameters

    - directory (str) – directory where the search should start

    - pattern (basestring) – the glob pattern (e.g .py, foo.py, etc.)

    - blacklist (list or tuple) – optional list of files or directory to ignore, default to the value of logilab.common.STD_BLACKLIST

  Return type

    iterator

  Returns

    iterator over the list of all matching files

logilab.common.shellutils.mv(source, destination, _action=<function move>)

  A shell-like mv, supporting wildcards.

class logilab.common.shellutils.progress(nbops=<object object>, size=<object object>, stream=<object object>, title=<object object>, enabled=True)

  Bases: `object`

class logilab.common.shellutils.pushd(directory)

  Bases: `object`

logilab.common.shellutils.rm(*files)

  A shell-like rm, supporting wildcards.

class logilab.common.shellutils.tempdir

  Bases: `object`

logilab.common.shellutils.unzip(archive, destdir)

## logilab.common.sphinx_ext module

## logilab.common.sphinxutils module

Sphinx utils

ModuleGenerator: Generate a file that lists all the modules of a list of packages in order to pull all the docstring. This should not be used in a makefile to systematically generate sphinx documentation!

Typical usage:

    >>> from logilab.common.sphinxutils import ModuleGenerator
    >>> mgen = ModuleGenerator('logilab common', '/home/adim/src/logilab/common')
    >>> mgen.generate('api_logilab_common.rst', exclude_dirs=('test',))

class logilab.common.sphinxutils.ModuleGenerator(project_title, code_dir)

  Bases: `object`

  class_def = '\n\n.. autoclass:: %s\n :members: %s\n\n'
  file_header = '.. -*- coding: utf-8 -*-\n\n%s\n'
  find_modules(exclude_dirs)
  gen_modules(exclude_dirs)

    generate all modules

  generate(dest_file, exclude_dirs=('CVS', '.svn', '.hg', '.git', '.tox', 'debian', 'dist', 'build'))

    make the module file

  module_def = '\n:mod:`%s`\n=======%s\n\n.. automodule:: %s\n :members: %s\n'

logilab.common.sphinxutils.class_members(klass)

logilab.common.sphinxutils.module_members(module)

## logilab.common.table module

Table management module.

class logilab.common.table.DocbookRenderer(**properties: Any)

  Bases: `logilab.common.table.TableCellRenderer`

  Defines how to render a cell for a docboook table

  define_col_header(col_index: int, table_style: logilab.common.table.TableStyle) → str

    Computes the colspec element according to the style

class logilab.common.table.DocbookTableWriter(stream: _io.StringIO, table: logilab.common.table.Table, style: Optional[Any], **properties: Any)

  Bases: `logilab.common.table.TableWriter`

  Defines an implementation of TableWriter to write a table in Docbook

  write_table(title: str = '') → None

    Writes the table

class logilab.common.table.Table(default_value: int = 0, col_names: Optional[List[str]] = None, row_names: Optional[Any] = None)

  Bases: `object`

  Table defines a data table with column and row names. inv:

      len(self.data) <= len(self.row_names)
      forall(self.data, lambda x: len(x) <= len(self.col_names))

  append_column(col_data: range, col_name: str) → None

    Appends the ‘col_index’ column pre:

        type(col_data) == types.ListType
        len(col_data) == len(self.row_names)

  append_row(row_data: Union[List[Union[float, str]], List[int]], row_name: Optional[str] = None) → int

    Appends a row to the table pre:

        type(row_data) == types.ListType
        len(row_data) == len(self.col_names)

  apply_stylesheet(stylesheet: logilab.common.table.TableStyleSheet) → None

    Applies the stylesheet to this table

  as_text()
  create_column(col_name: str) → None

    Creates a colname to the col_names list

  create_columns(col_names: List[str]) → None

    Appends col_names to the list of existing columns

  create_row(row_name: Optional[str] = None) → None

    Creates a rowname to the row_names list

  create_rows(row_names: List[str]) → None

    Appends row_names to the list of existing rows

  delete_column(index: int) → List[int]

    Deletes the ‘index’ column in the table, and returns it. Raises an IndexError if index is out of range

  delete_column_by_id(col_id: str) → None

    Deletes the ‘col_id’ col in the table. Raises a KeyError if col_id was not found.

  delete_row(index: int) → List[str]

    Deletes the ‘index’ row in the table, and returns it. Raises an IndexError if index is out of range

  delete_row_by_id(row_id: str) → None

    Deletes the ‘row_id’ row in the table. Raises a KeyError if row_id was not found.

  get_cell_by_ids(row_id, col_id)

    Returns the element at [row_id][col_id]

  get_column(col_index, distinct=False)

    get a column by index

  get_column_by_id(col_id, distinct=False)

    Returns the ‘col_id’ col

  get_columns() → List[List[int]]

    Returns all the columns in the table

  get_row_by_id(row_id)

    Returns the ‘row_id’ row

  get_shape() → Tuple[int, int]

    Returns a tuple which represents the table’s shape

  groupby(colname: str, *others: str) → Union[Dict[str, Dict[str, logilab.common.table.Table]], Dict[str, logilab.common.table.Table]]

    builds indexes of data :returns: nested dictionaries pointing to actual rows

  insert_column(index: int, col_data: range, col_name: str) → None

    Appends col_data before ‘index’ in the table. To make ‘insert’ behave like ‘list.insert’, inserting in an out of range index will insert col_data to the end of the list pre:

        type(col_data) == types.ListType
        len(col_data) == len(self.row_names)

  insert_row(index: int, row_data: List[str], row_name: Optional[str] = None) → None

    Appends row_data before ‘index’ in the table. To make ‘insert’ behave like ‘list.insert’, inserting in an out of range index will insert row_data to the end of the list pre:

        type(row_data) == types.ListType
        len(row_data) == len(self.col_names)

  pprint() → str

    returns a string representing the table in a pretty printed ‘text’ format.

  remove(colname, value)
  select(colname: str, value: str) → logilab.common.table.Table
  set_cell(row_index: int, col_index: int, data: int) → None

    sets value of cell ‘row_indew’, ‘col_index’ to data

  set_cell_by_ids(row_id: str, col_id: str, data: Union[int, str]) → None

    sets value of cell mapped by row_id and col_id to data Raises a KeyError if row_id or col_id are not found in the table

  set_column(col_index: int, col_data: Union[List[int], range]) → None

    sets the ‘col_index’ column pre:

        type(col_data) == types.ListType
        len(col_data) == len(self.row_names)

  set_column_by_id(col_id: str, col_data: Union[List[int], range]) → None

    sets the ‘col_id’ column pre:

        type(col_data) == types.ListType
        len(col_data) == len(self.col_names)

    Raises a KeyError if col_id is not found

  set_row(row_index: int, row_data: Union[List[float], List[int], List[str]]) → None

    sets the ‘row_index’ row pre:

        type(row_data) == types.ListType
        len(row_data) == len(self.col_names)

  set_row_by_id(row_id: str, row_data: List[str]) → None

    sets the ‘row_id’ column pre:

        type(row_data) == types.ListType
        len(row_data) == len(self.row_names)

    Raises a KeyError if row_id is not found

  property shape: Tuple[int, int]

    Returns a tuple which represents the table’s shape

  sort_by_column_id(col_id: str, method: str = 'asc') → None

    Sorts the table (in-place) according to data stored in col_id

  sort_by_column_index(col_index: int, method: str = 'asc') → None

    Sorts the table ‘in-place’ according to data stored in col_index

    method should be in (‘asc’, ‘desc’)

  transpose() → logilab.common.table.Table

    Keeps the self object intact, and returns the transposed (rotated) table.

class logilab.common.table.TableCellRenderer(**properties: Any)

  Bases: `object`

  Defines a simple text renderer

  render_cell(cell_coord: Tuple[int, int], table: logilab.common.table.Table, table_style: logilab.common.table.TableStyle) → Union[str, int]

    Renders the cell at ‘cell_coord’ in the table, using table_style

  render_col_cell(col_name: str, table: logilab.common.table.Table, table_style: logilab.common.table.TableStyle) → Union[str, int]

    Renders the cell for ‘col_id’ row

  render_row_cell(row_name: str, table: logilab.common.table.Table, table_style: logilab.common.table.TableStyle) → Union[str, int]

    Renders the cell for ‘row_id’ row

class logilab.common.table.TableStyle(table: logilab.common.table.Table)

  Bases: `object`

  Defines a table’s style

  get_alignment(col_id: str) → str

    Returns the alignment of the specified col_id

  get_alignment_by_index(col_index: int) → str

    Allors to get the alignment according to the column index rather than using the column’s id. BE CAREFUL : the ‘0’ column is the ‘__row_column__’ one !

  get_size(col_id: str) → str

    Returns the size of the specified col_id

  get_size_by_index(col_index: int) → str

    Allows to get the size according to the column index rather than using the column’s id. BE CAREFUL : the ‘0’ column is the ‘__row_column__’ one !

  get_unit(col_id: str) → str

    Returns the unit of the specified col_id

  get_unit_by_index(col_index: int) → str

    Allors to get the unit according to the column index rather than using the column’s id. BE CAREFUL : the ‘0’ column is the ‘__row_column__’ one !

  set_alignment(value: str, col_id: str) → None

    sets the alignment of the specified col_id to value

  set_alignment_by_index(value: str, col_index: int) → None

    Allows to set the alignment according to the column index rather than using the column’s id. BE CAREFUL : the ‘0’ column is the ‘__row_column__’ one !

  set_size(value: str, col_id: str) → None

    sets the size of the specified col_id to value

  set_size_by_index(value: str, col_index: int) → None

    Allows to set the size according to the column index rather than using the column’s id. BE CAREFUL : the ‘0’ column is the ‘__row_column__’ one !

  set_unit(value: str, col_id: str) → None

    sets the unit of the specified col_id to value

  set_unit_by_index(value: str, col_index: int) → None

    Allows to set the unit according to the column index rather than using the column’s id. BE CAREFUL : the ‘0’ column is the ‘__row_column__’ one ! (Note that in the ‘unit’ case, you shouldn’t have to set a unit for the 1st column (the __row__column__ one))

class logilab.common.table.TableStyleSheet(rules: Optional[List[str]] = None)

  Bases: `object`

  A simple Table stylesheet Rules are expressions where cells are defined by the row_index and col_index separated by an underscore (‘_’). For example, suppose you want to say that the (2,5) cell must be the sum of its two preceding cells in the row, you would create the following rule

      2_5 = 2_3 + 2_4

  You can also use all the math.* operations you want. For example:

      2_5 = sqrt(2_3**2 + 2_4**2)

  add_colavg_rule(dest_cell: Tuple[int, int], col_index: int, start_row: int, end_row: int) → None

    Creates and adds a rule to make the col average (from start_row to end_row) dest_cell is a tuple of two elements (x,y) of the destination cell No check is done for indexes ranges. pre:

        start_row >= 0
        end_row > start_row

  add_colsum_rule(dest_cell: Tuple[int, int], col_index: int, start_row: int, end_row: int) → None

    Creates and adds a rule to sum over the col at col_index from start_row to end_row. dest_cell is a tuple of two elements (x,y) of the destination cell No check is done for indexes ranges. pre:

        start_row >= 0
        end_row > start_row

  add_rowavg_rule(dest_cell: Tuple[int, int], row_index: int, start_col: int, end_col: int) → None

    Creates and adds a rule to make the row average (from start_col to end_col) dest_cell is a tuple of two elements (x,y) of the destination cell No check is done for indexes ranges. pre:

        start_col >= 0
        end_col > start_col

  add_rowsum_rule(dest_cell: Tuple[int, int], row_index: int, start_col: int, end_col: int) → None

    Creates and adds a rule to sum over the row at row_index from start_col to end_col. dest_cell is a tuple of two elements (x,y) of the destination cell No check is done for indexes ranges. pre:

        start_col >= 0
        end_col > start_col

  add_rule(rule: str) → None

    Adds a rule to the stylesheet rules

class logilab.common.table.TableWriter(stream: _io.StringIO, table: logilab.common.table.Table, style: Optional[Any], **properties: Any)

  Bases: `object`

  A class to write tables

  set_renderer(renderer: logilab.common.table.DocbookRenderer) → None

    sets the way to render cell

  set_style(style)

    sets the table’s associated style

  update_properties(**properties)

    Updates writer’s properties (for cell rendering)

  write_table(title: str = '') → None

    Writes the table

## logilab.common.tasksqueue module

Prioritized tasks queue

class logilab.common.tasksqueue.PrioritizedTasksQueue(maxsize=0)

  Bases: `queue.Queue`

  remove(tid: str) → None

    remove a specific task from the queue

class logilab.common.tasksqueue.Task(tid: str, priority: int = 0)

  Bases: `object`

  merge(other: logilab.common.tasksqueue.Task) → None

## logilab.common.testlib module

Run tests.

This will find all modules whose name match a given prefix in the test directory, and run them. Various command line options provide additional facilities.

Command line options:

> `-v`

>   verbose – run tests in verbose mode with output to stdout

> `-q`

>   quiet – don’t print anything except if a test fails

> `-t`

>   testdir – directory where the tests will be found

> `-x`

>   exclude – add a test to exclude

> `-p`

>   profile – profiled execution

> `-d`

>   dbc – enable design-by-contract

> `-m`

>   match – only run test matching the tag pattern which follow

If no non-option arguments are present, prefixes used are ‘test’, ‘regrtest’, ‘smoketest’ and ‘unittest’.

logilab.common.testlib.find_tests(testdir, prefixes=('test', 'regrtest', 'smoketest', 'unittest', 'func', 'validation'), suffix='.py', excludes=(), remove_suffix=True)

  Return a list of all applicable test modules.

logilab.common.testlib.nocoverage(func: Callable) → Callable

  Function decorator that pauses tracing functions

logilab.common.testlib.pause_trace(trace: Optional[Callable] = None) → Iterator

  A context manager that temporary replaces the trace function

logilab.common.testlib.unittest_main

  alias of `unittest.main.TestProgram`

## logilab.common.textutils module

Some text manipulation utility functions.

group text formatting

  normalize_text, normalize_paragraph, pretty_match,unquote, colorize_ansi

group text manipulation

  searchall, splitstrip

sort

  text formatting, text manipulation

type ANSI_STYLES

  dict(str)

var ANSI_STYLES

  dictionary mapping style identifier to ANSI terminal code

type ANSI_COLORS

  dict(str)

var ANSI_COLORS

  dictionary mapping color identifier to ANSI terminal code

type ANSI_PREFIX

  str

var ANSI_PREFIX

  ANSI terminal code notifying the start of an ANSI escape sequence

type ANSI_END

  str

var ANSI_END

  ANSI terminal code notifying the end of an ANSI escape sequence

type ANSI_RESET

  str

var ANSI_RESET

  ANSI terminal code resetting format defined by a previous ANSI escape sequence

logilab.common.textutils.apply_units(string: str, units: Dict[str, int], inter: Union[Callable, None, type] = None, final: type = <class 'float'>, blank_reg: re.Pattern = re.compile('(\\s|,)+'), value_reg: re.Pattern = re.compile('(?P<value>-?(([0-9]+\\.[0-9]*)|((0x?)?[0-9]+)))(?P<unit>[a-zA-Z]+)?')) → Union[float, int]

  Parse the string applying the units defined in units (e.g.: “1.5m”,{‘m’,60} -> 80).

  Parameters

    - string (str or unicode) – the string to parse

    - units (dict (or any object with __getitem__ using basestring key)) – a dict mapping a unit string repr to its value

    - inter (type) – used to parse every intermediate value (need __sum__)

    - blank_reg (regexp) – should match every blank char to ignore.

    - value_reg (regexp with "value" and optional "unit" group) – match a value and it’s unit into the

logilab.common.textutils.colorize_ansi(msg: str, color: Optional[str] = None, style: Optional[str] = None) → str

  colorize message by wrapping it with ansi escape codes

  Parameters

    - msg (str or unicode) – the message string to colorize

    - color (str or None) – the color identifier (see ANSI_COLORS for available values)

    - style (str or None) – style string (see ANSI_COLORS for available values). To get several style effects at the same time, use a coma as separator.

  Raises

    KeyError – if an unexistent color or style identifier is given

  Return type

    str or unicode

  Returns

    the ansi escaped string

logilab.common.textutils.diff_colorize_ansi(lines, out=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>, style={'add': 'green', 'remove': 'red', 'separator': 'cyan'})

logilab.common.textutils.normalize_paragraph(text: str, line_len: int = 80, indent: str = '') → str

  normalize a text to display it with a maximum line size and optionally arbitrary indentation. Line jumps are normalized. The indentation string may be used top insert a comment mark for instance.

  Parameters

    - text (str or unicode) – the input text to normalize

    - line_len (int) – expected maximum line’s length, default to 80

    - indent (str or unicode) – optional string to use as indentation

  Return type

    str or unicode

  Returns

    the input text normalized to fit on lines with a maximized size inferior to line_len, and optionally prefixed by an indentation string

logilab.common.textutils.normalize_rest_paragraph(text: str, line_len: int = 80, indent: str = '') → str

  normalize a ReST text to display it with a maximum line size and optionally arbitrary indentation. Line jumps are normalized. The indentation string may be used top insert a comment mark for instance.

  Parameters

    - text (str or unicode) – the input text to normalize

    - line_len (int) – expected maximum line’s length, default to 80

    - indent (str or unicode) – optional string to use as indentation

  Return type

    str or unicode

  Returns

    the input text normalized to fit on lines with a maximized size inferior to line_len, and optionally prefixed by an indentation string

logilab.common.textutils.normalize_text(text: str, line_len: int = 80, indent: str = '', rest: bool = False) → str

  normalize a text to display it with a maximum line size and optionally arbitrary indentation. Line jumps are normalized but blank lines are kept. The indentation string may be used to insert a comment (#) or a quoting (>) mark for instance.

  Parameters

    - text (str or unicode) – the input text to normalize

    - line_len (int) – expected maximum line’s length, default to 80

    - indent (str or unicode) – optional string to use as indentation

  Return type

    str or unicode

  Returns

    the input text normalized to fit on lines with a maximized size inferior to line_len, and optionally prefixed by an indentation string

logilab.common.textutils.pretty_match(match: re.Match, string: str, underline_char: str = '^') → str

  return a string with the match location underlined:

      >>> import re
      >>> print(pretty_match(re.search('mange', 'il mange du bacon'), 'il mange du bacon'))
      il mange du bacon
         ^^^^^
      >>>

  Parameters

    - match (_sre.SRE_match) – object returned by re.match, re.search or re.finditer

    - string (str or unicode) – the string on which the regular expression has been applied to obtain the match object

    - underline_char (str or unicode) – character to use to underline the matched section, default to the carret ‘^’

  Return type

    str or unicode

  Returns

    the original string with an inserted line to underline the match location

logilab.common.textutils.split_url_or_path(url_or_path)

  return the latest component of a string containing either an url of the form <scheme>://<path> or a local file system path

logilab.common.textutils.splitstrip(string: str, sep: str = ',') → List[str]

  return a list of stripped string by splitting the string given as argument on sep (‘,’ by default). Empty string are discarded.

      >>> splitstrip('a, b, c   ,  4,,')
      ['a', 'b', 'c', '4']
      >>> splitstrip('a')
      ['a']
      >>>

  Parameters

    - string (str or unicode) – a csv line

    - sep (str or unicode) – field separator, default to the comma (‘,’)

  Return type

    str or unicode

  Returns

    the unquoted string (or the input string if it wasn’t quoted)

logilab.common.textutils.splittext(text: str, line_len: int) → Tuple[str, str]

  split the given text on space according to the given max line size

  return a 2-uple: * a line <= line_len if possible * the rest of the text which has to be reported on another line

logilab.common.textutils.text_to_dict(text)

  parse multilines text containing simple ‘key=value’ lines and return a dict of {‘key’: ‘value’}. When the same key is encountered multiple time, value is turned into a list containing all values.

      >>> d = text_to_dict('''multiple=1
      ... multiple= 2
      ... single =3
      ... ''')
      >>> d['single']
      '3'
      >>> d['multiple']
      ['1', '2']

logilab.common.textutils.unormalize(ustring: str, ignorenonascii: Optional[Any] = None, substitute: Optional[str] = None) → str

  replace diacritical characters with their corresponding ascii characters

  Convert the unicode string to its long normalized form (unicode character will be transform into several characters) and keep the first one only. The normal form KD (NFKD) will apply the compatibility decomposition, i.e. replace all compatibility characters with their equivalents.

  Parameters

    substitute (str) – replacement character to use if decomposition fails

  See

    Another project about ASCII transliterations of Unicode text http://pypi.python.org/pypi/Unidecode

logilab.common.textutils.unquote(string: str) → str

  remove optional quotes (simple or double) from the string

  Parameters

    string (str or unicode) – an optionally quoted string

  Return type

    str or unicode

  Returns

    the unquoted string (or the input string if it wasn’t quoted)

## logilab.common.tree module

Base class to represent a tree structure.

class logilab.common.tree.BinaryNode(lhs=None, rhs=None)

  Bases: `logilab.common.tree.VNode`

  a binary node (i.e. only two children

  get_parts()

    return the left hand side and the right hand side of this node

  remove(child)

    remove the child and replace this node with the other child

class logilab.common.tree.ListNode

  Bases: `logilab.common.tree.VNode`, `list`

  Used to manipulate Nodes as Lists

  append(child)

    add a node to children

  insert(index, child)

    add a node to children

  pop(index)

    add a node to children

  remove(child)

    add a node to children

class logilab.common.tree.Node(nid: Optional[str] = None)

  Bases: `object`

  a basic tree node, characterized by an id

  append(child: Any) → None

    add a node to children

  depth() → int

    return depth of this node in the tree

  depth_down() → int

    return depth of the tree from this node

  flatten(_list: Optional[List[Any]] = None) → List[Any]

    return a list with all the nodes descendant from this node

  get_child_by_id(nid: str, recurse: Optional[bool] = None) → Any

    return child of given id

  get_child_by_path(path: List[str]) → Any

    return child of given path (path is a list of ids)

  get_node_by_id(nid: str) → Any

    return node in whole hierarchy that has given id

  get_sibling(nid: str) → Any

    return the sibling node that has given id

  insert(index: int, child: Any) → None

    insert a child node

  is_leaf()
  leaves() → List[Any]

    return a list with all the leaves nodes descendant from this node

  lineage() → List[Any]

    return list of parents up to root node

  next_sibling()

    return the next sibling for this node if any

  previous_sibling()

    return the previous sibling for this node if any

  remove(child: Any) → None

    remove a child node

  replace(old_child: Any, new_child: Any) → None

    replace a child node with another

  root() → Any

    return the root node of the tree

  width() → int

    return the width of the tree from this node

exception logilab.common.tree.NodeNotFound

  Bases: `Exception`

  raised when a node has not been found

class logilab.common.tree.PostfixedDepthFirstIterator(node: logilab.common.tree.Node, filter_func: Optional[Any] = None)

  Bases: `logilab.common.visitor.FilteredIterator`

  a postfixed depth first iterator, designed to be used with visitors

class logilab.common.tree.PrefixedDepthFirstIterator(node: logilab.common.tree.Node, filter_func: Optional[Any] = None)

  Bases: `logilab.common.visitor.FilteredIterator`

  a prefixed depth first iterator, designed to be used with visitors

class logilab.common.tree.VNode(nid: Optional[str] = None)

  Bases: `logilab.common.tree.Node`, `logilab.common.visitor.VisitedMixIn`

  a visitable node

logilab.common.tree.post_order_list(node: Optional[logilab.common.tree.Node], filter_func: Callable = <function no_filter>) → List[logilab.common.tree.Node]

  create a list with tree nodes for which the <filter> function returned true in a post order fashion

logilab.common.tree.pre_order_list(node: Optional[logilab.common.tree.Node], filter_func: Callable = <function no_filter>) → List[logilab.common.tree.Node]

  create a list with tree nodes for which the <filter> function returned true in a pre order fashion

## logilab.common.umessage module

Unicode email support (extends email from stdlib)

class logilab.common.umessage.UMessage(message: email.message.Message)

  Bases: `object`

  Encapsulates an email.Message instance and returns only unicode objects.

  date(alternative_source=False, return_str=False)

    return a datetime object for the email’s date or None if no date is set or if it can’t be parsed

  get(header: str, default: Optional[Any] = None) → Optional[str]
  get_all(header: str, default: Tuple[()] = ()) → List[str]
  get_boundary()
  get_content_maintype()
  get_content_type()
  get_filename(failobj=None)
  get_payload(index: Optional[Any] = None, decode: bool = False) → Union[str, logilab.common.umessage.UMessage, List[logilab.common.umessage.UMessage]]
  headers()

    return an unicode string containing all the message’s headers

  is_multipart()
  multi_addrs(header)

    return a list of 2-uple (name, address) for the given address (which is expected to be an header containing address such as from, to, cc…)

  walk()

logilab.common.umessage.decode_QP(string: str) → str

logilab.common.umessage.message_from_file(fd)

logilab.common.umessage.message_from_string(string: str) → Union[logilab.common.umessage.UMessage, str]

## logilab.common.urllib2ext module

## logilab.common.vcgutils module

Functions to generate files readable with Georg Sander’s vcg (Visualization of Compiler Graphs).

You can download vcg at http://rw4.cs.uni-sb.de/~sander/html/gshome.html Note that vcg exists as a debian package.

See vcg’s documentation for explanation about the different values that maybe used for the functions parameters.

class logilab.common.vcgutils.VCGPrinter(output_stream)

  Bases: `object`

  A vcg graph writer.

  close_graph()

    close a vcg graph

  edge(from_node, to_node, edge_type='', **args)

    draw an edge from a node to another.

  node(title, **args)

    draw a node

  open_graph(**args)

    open a vcg graph

logilab.common.vcgutils.latin_to_vcg(st)

  Convert latin characters using vcg escape sequence.

## logilab.common.visitor module

A generic visitor abstract implementation.

class logilab.common.visitor.FilteredIterator(node: logilab.common.types.Node, list_func: Callable, filter_func: Optional[Any] = None)

  Bases: `object`

  next() → Optional[logilab.common.types.Node]

class logilab.common.visitor.VisitedMixIn

  Bases: `object`

  Visited interface allow node visitors to use the node

  accept(visitor: Union[logilab.common.types.HTMLWriter, logilab.common.types.TextWriter], *args: Any, **kwargs: Any) → Optional[Any]
  get_visit_name() → str

    return the visit name for the mixed class. When calling ‘accept’, the method <’visit_’ + name returned by this method> will be called on the visitor

  leave(visitor, *args, **kwargs)

class logilab.common.visitor.Visitor(iterator_class, filter_func=None)

  Bases: `object`

  close_visit(result)

    method called at the end of the visit

  open_visit(*args, **kargs)

    method called at the beginning of the visit

  visit(node, *args, **kargs)

    launch the visit on a given node

    call ‘open_visit’ before the beginning of the visit, with extra args given when all nodes have been visited, call the ‘close_visit’ method

logilab.common.visitor.no_filter(_: logilab.common.types.Node) → int

## logilab.common.xmlutils module

XML utilities.

This module contains useful functions for parsing and using XML data. For the moment, there is only one function that can parse the data inside a processing instruction and return a Python dictionary.

logilab.common.xmlutils.parse_pi_data(pi_data: str) → Dict[str, Optional[str]]

  Utility function that parses the data contained in an XML processing instruction and returns a dictionary of keywords and their associated values (most of the time, the processing instructions contain data like `keyword="value"`, if a keyword is not associated to a value, for example `keyword`, it will be associated to `None`).

  Parameters

    pi_data (unicode) – data contained in an XML processing instruction.

  Returns

    Dictionary of the keywords (Unicode strings) associated to their values (Unicode strings) as they were defined in the data.

  Return type

    dict

## Module contents

Logilab common library (aka Logilab’s extension to the standard library).

type STD_BLACKLIST

  tuple

var STD_BLACKLIST

  directories ignored by default by the functions in this package which have to recurse into directories

type IGNORED_EXTENSIONS

  tuple

var IGNORED_EXTENSIONS

  file extensions that may usually be ignored

class logilab.common.attrdict

  Bases: `dict`

  A dictionary for which keys are also accessible as attributes.

class logilab.common.dictattr(proxy)

  Bases: `dict`

logilab.common.flatten(iterable, tr_func=None, results=None)

  Flatten a list of list with any level.

  If tr_func is not None, it should be a one argument function that’ll be called on each final element.

  Return type

    list

      >>> flatten([1, [2, 3]])
      [1, 2, 3]

logilab.common.make_domains(lists)

  Given a list of lists, return a list of domain for each list to produce all combinations of possibles values.

  Return type

    list

  Example:

      >>> make_domains(['a', 'b'], ['c','d', 'e'])
      [['a', 'b', 'a', 'b', 'a', 'b'], ['c', 'c', 'd', 'd', 'e', 'e']]

class logilab.common.nullobject

  Bases: `object`

class logilab.common.tempattr(obj, attr, value)

  Bases: `object`