logilab.common package

Subpackages

• logilab.common.ureports package
  • Submodules
  • logilab.common.ureports.docbook_writer module
    • DocbookWriter
      • DocbookWriter.begin_format()
      • DocbookWriter.end_format()
      • DocbookWriter.visit_link()
      • DocbookWriter.visit_list()
      • DocbookWriter.visit_paragraph()
      • DocbookWriter.visit_section()
      • DocbookWriter.visit_span()
      • DocbookWriter.visit_table()
      • DocbookWriter.visit_text()
      • DocbookWriter.visit_title()
      • DocbookWriter.visit_verbatimtext()
  • logilab.common.ureports.html_writer module
    • HTMLWriter
      • HTMLWriter.begin_format()
      • HTMLWriter.end_format()
      • HTMLWriter.handle_attrs()
      • HTMLWriter.visit_link()
      • HTMLWriter.visit_list()
      • HTMLWriter.visit_paragraph()
      • HTMLWriter.visit_section()
      • HTMLWriter.visit_span()
      • HTMLWriter.visit_table()
      • HTMLWriter.visit_text()
      • HTMLWriter.visit_title()
      • HTMLWriter.visit_verbatimtext()
  • logilab.common.ureports.nodes module
    • BaseComponent
    • BaseLayout
      • BaseLayout.add_text()
      • BaseLayout.append()
      • BaseLayout.parents()
    • Image
    • Link
    • List
    • Paragraph
    • Section
    • Span
    • Table
    • Text
    • Title
    • VerbatimText
  • logilab.common.ureports.text_writer module
    • TextWriter
      • TextWriter.begin_format()
      • TextWriter.default_table()
      • TextWriter.field_table()
      • TextWriter.visit_link()
      • TextWriter.visit_list()
      • TextWriter.visit_paragraph()
      • TextWriter.visit_section()
      • TextWriter.visit_span()
      • TextWriter.visit_table()
      • TextWriter.visit_text()
      • TextWriter.visit_title()
      • TextWriter.visit_verbatimtext()
  • Module contents
    • BaseWriter
      • BaseWriter.begin_format()
      • BaseWriter.compute_content()
      • BaseWriter.end_format()
      • BaseWriter.format()
      • BaseWriter.format_children()
      • BaseWriter.get_table_content()
      • BaseWriter.write()
      • BaseWriter.writeln()
    • build_summary()
    • get_nodes()
    • layout_title()

Submodules

logilab.common.cache module

Cache module, with a least recently used algorithm for the management of the deletion of entries.

class logilab.common.cache.Cache(size: int = 100)

Bases: dict

A dictionary like cache.

inv:

len(self._usage) <= self.size len(self.data) <= self.size

clear() → None.  Remove all items from D.

pop(k[, d]) → v, remove specified key and return the corresponding value.

If key is not found, d is returned if given, otherwise KeyError is raised

popitem() → (k, v), remove and return some (key, value) pair as a

2-tuple; but raise KeyError if D is empty.

setdefault(key, default=None)

Insert key with a value of default if key is not in the dictionary.

Return the value for key if key is in the dictionary, else default.

update([E, ]**F) → None.  Update D from dict/iterable E and F.

If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]

logilab.common.changelog module

Manipulation of upstream change log files.

The upstream change log files format handled is simpler than the one often used such as those generated by the default Emacs changelog mode.

Sample ChangeLog format:

Change log for project Yoo
==========================

 --
    * add a new functionality

2002-02-01 -- 0.1.1
    * fix bug #435454
    * fix bug #434356

2002-01-01 -- 0.1
    * initial release

There is 3 entries in this change log, one for each released version and one for the next version (i.e. the current entry). Each entry contains a set of messages corresponding to changes done in this release. All the non empty lines before the first entry are considered as the change log title.

class logilab.common.changelog.ChangeLog(changelog_file: str, title: str = '')

Bases: object

object representation of a whole ChangeLog file

add(msg, create=None)

add a new message to the latest opened entry

add_entry(entry: ChangeLogEntry) → None

add a new entry to the change log

entry_class

alias of ChangeLogEntry

format_title() → str

get_entry(version='', create=None)

return a given changelog entry if version is omitted, return the current entry

load() → None

read a logilab’s ChangeLog from file

save()

write back change log

write(stream: ~_io.StringIO = <_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>) → None

write changelog to stream

class logilab.common.changelog.ChangeLogEntry(date: Optional[str] = None, version: Optional[str] = None, **kwargs: Any)

Bases: object

a change log entry, i.e. a set of messages associated to a version and its release date

add_message(msg: str) → None

add a new message

add_sub_message(sub_msg: str, key: Optional[Any] = None) → None

complete_latest_message(msg_suite: str) → None

complete the latest added message

version_class

alias of Version

write(stream: ~_io.StringIO = <_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>) → None

write the entry to file

exception logilab.common.changelog.EntryNotFound

Bases: Exception

raised when we are unable to find a given entry

exception logilab.common.changelog.NoEntry

Bases: Exception

raised when we are unable to find an entry

class logilab.common.changelog.Version(versionstr)

Bases: tuple

simple class to handle soft version number has a tuple while correctly printing it as X.Y.Z

classmethod parse(versionstr: str) → List[int]

logilab.common.clcommands module

Helper functions to support command line tools providing more than one command.

e.g called as “tool command [options] args…” where <options> and <args> are command’specific

exception logilab.common.clcommands.BadCommandUsage

Bases: Exception

Raised when an unknown command is used or when a command is not correctly used (bad options, too much / missing arguments…).

Trigger display of command usage.

class logilab.common.clcommands.Command(logger)

Bases: Configuration

Base class for command line commands.

Class attributes:

• name, the name of the command

• min_args, minimum number of arguments, None if unspecified

• max_args, maximum number of arguments, None if unspecified

• arguments, string describing arguments, used in command usage

• hidden, boolean flag telling if the command should be hidden, e.g. does not appear in help’s commands list

• options, options list, as allowed by :mod:configuration

arguments = ''

check_args(args)

check command’s arguments are provided

classmethod description()

hidden = False

main_run(args, rcfile=None)

Run the command and return status 0 if everything went fine.

If CommandError is raised by the underlying command, simply log the error and return status 2.

Any other exceptions, including BadCommandUsage will be propagated.

max_args = None

min_args = None

name = ''

run(args)

run the command with its specific arguments

classmethod short_description()

exception logilab.common.clcommands.CommandError

Bases: Exception

Raised when a command can’t be processed and we want to display it and exit, without traceback nor usage displayed.

class logilab.common.clcommands.CommandLine(pgm=None, doc=None, copyright=None, version=None, rcfile=None, logthreshold=40, check_duplicated_command=True)

Bases: dict

Usage:

>>> LDI = cli.CommandLine('ldi', doc='Logilab debian installer',
                          version=version, rcfile=RCFILE)
>>> LDI.register(MyCommandClass)
>>> LDI.register(MyOtherCommandClass)
>>> LDI.run(sys.argv[1:])

Arguments:

• pgm, the program name, default to basename(sys.argv[0])

• doc, a short description of the command line tool

• copyright, additional doc string that will be appended to the generated doc

• version, version number of string of the tool. If specified, global –version option will be available.

• rcfile, path to a configuration file. If specified, global –C/–rc-file option will be available? self.rcfile = rcfile

• logger, logger to propagate to commands, default to logging.getLogger(self.pgm))

create_logger(handler, logthreshold=None)

get_command(cmd, logger=None)

register(cls, force=False)

register the given Command subclass

run(args)

main command line access point: * init logging * handle global options (-h/–help, –version, -C/–rc-file) * check command * run command

Terminate by SystemExit

usage()

display usage for the main program (i.e. when no command supplied) and exit

usage_and_exit(status)

class logilab.common.clcommands.ListCommandsCommand(logger)

Bases: Command

list available commands, useful for bash completion.

arguments = '[command]'

hidden = True

name = 'listcommands'

option_groups: List[Tuple[Any, str]]

options_providers: List[ConfigurationMixIn]

run(args)

run the command with its specific arguments

logilab.common.compat module

Wrappers around some builtins introduced in python 2.3, 2.4 and 2.5, making them available in for earlier versions of python.

See another compatibility snippets from other projects:

lib2to3.fixes coverage.backward unittest2.compatibility

logilab.common.compat.method_type(callable, instance, klass)

logilab.common.compat.str_to_bytes(string)

logilab.common.configuration module

Classes to handle advanced configuration in simple to complex applications.

Allows to load the configuration from a file or from command line options, to generate a sample configuration file or to display program’s usage. Fills the gap between optik/optparse and ConfigParser by adding data types (which are also available as a standalone optik extension in the optik_ext module).

Quick start: simplest usage

class logilab.common.configuration.Configuration(config_file=None, options=None, name=None, usage=None, doc=None, version=None)

Bases: ConfigurationMixIn

class for simple configurations which don’t need the manager / providers model and prefer delegation to inheritance

configuration values are accessible through a dict like interface

class logilab.common.configuration.ConfigurationMixIn(*args: Any, **kwargs: Any)

Bases: OptionsManagerMixIn, OptionsProviderMixIn

basic mixin for simple configurations which don’t need the manager / providers model

get(key, default=None)

load_defaults()

initialize the provider using default values

register_options(options)

add some options to the configuration

class logilab.common.configuration.OptionsManager2ConfigurationAdapter(provider)

Bases: object

Adapt an option manager to behave like a logilab.common.configuration.Configuration instance

get(key, default=None)

class logilab.common.configuration.OptionsManagerMixIn(usage: Optional[str], config_file: Optional[Any] = None, version: Optional[Any] = None, quiet: int = 0)

Bases: object

MixIn to handle a configuration from both a configuration file and command line options

add_help_section(title: str, description: str, level: int = 0) → None

add a dummy option section for help purpose

add_optik_option(provider: ConfigurationMixIn, optikcontainer: Union[OptionParser, OptionGroup], opt: str, optdict: Dict[str, Any]) → None

add_option_group(group_name: str, doc: Optional[str], options: Union[List[Tuple[str, Dict[str, Any]]], List[Tuple[str, Dict[str, str]]]], provider: ConfigurationMixIn) → None

add an option group including the listed options

cb_set_provider_option(option: Option, opt: str, value: Union[List[str], int, str], parser: OptionParser) → None

optik callback for option setting

generate_config(stream: Optional[Union[StringIO, TextIOWrapper]] = None, skipsections: Tuple[()] = (), encoding: Optional[Any] = None, header_message: Optional[str] = None) → None

write a configuration file according to the current configuration into the given stream or stdout

generate_manpage(pkginfo: attrdict, section: int = 1, stream: Optional[StringIO] = None) → None

write a man page for the current configuration into the given stream or stdout

global_set_option(opt: str, value: Union[List[str], int, str]) → None

set option on the correct option provider

help(level: int = 0) → str

return the usage string for available options

input_config(onlysection=None, inputlevel=0, stream=None)

interactively get configuration values by asking to the user and generate a configuration file

load_command_line_configuration(args: Optional[List[str]] = None) → List[str]

override configuration according to command line parameters

return additional arguments

load_config_file() → None

dispatch values previously read from a configuration file to each options provider)

load_configuration(**kwargs: Any) → None

override configuration according to given parameters

load_file_configuration(config_file: Optional[str] = None) → None

load the configuration from file

load_provider_defaults() → None

initialize configuration using default values

optik_option(provider: ConfigurationMixIn, opt: str, optdict: Dict[str, Any]) → Tuple[List[str], Dict[str, Any]]

get our personal option definition and return a suitable form for use with optik/optparse

read_config_file(config_file: Optional[str] = None) → None

read the configuration file but do not load it (i.e. dispatching values to each options provider)

register_options_provider(provider: ConfigurationMixIn, own_group: bool = True) → None

register an options provider

reset_parsers(usage: Optional[str] = '', version: Optional[Any] = None) → None

class logilab.common.configuration.OptionsProviderMixIn

Bases: object

Mixin to provide options to an OptionsManager

all_options()

return an iterator on available options for this provider option are actually described by a 3-uple: (section, option name, option dictionary)

get_option_def(opt)

return the dictionary defining an option given it’s name

input_option(option, optdict, inputlevel=99)

level = 0

load_defaults() → None

initialize the provider using default values

name = 'default'

option_attrname(opt, optdict=None)

get the config attribute corresponding to opt

option_default(opt, optdict=None)

return the default value for an option

option_value(opt)

get the current value for the given option

options: Tuple = ()

options_and_values(options=None)

options_by_section() → Iterator[Any]

return an iterator on options grouped by section

(section, [list of (optname, optdict, optvalue)])

priority = -1

set_option(opt, value, action=None, optdict=None)

method called to set an option (registered in the options list)

logilab.common.daemon module

A daemonize function (for Unices)

logilab.common.daemon.daemonize(pidfile=None, uid=None, umask=63)

daemonize a Unix process. Set paranoid umask by default.

Return 1 in the original process, 2 in the first fork, and None for the second fork (eg daemon process).

logilab.common.daemon.setugid(user)

Change process user and group ID

Argument is a numeric user id or a user name

logilab.common.date module

Date manipulation helper functions.

logilab.common.date.add_days_worked(start: date, days: int) → date

adds date but try to only take days worked into account

logilab.common.date.date_range(begin: date, end: date, incday: Optional[Any] = None, incmonth: Optional[bool] = None) → Generator[date, Any, None]

yields each date between begin and end

Parameters:

• begin – the start date

• end – the end date

• incr – the step to use to iterate over dates. Default is one day.

• include – None (means no exclusion) or a function taking a date as parameter, and returning True if the date should be included.

When using mx datetime, you should NOT use incmonth argument, use instead oneDay, oneHour, oneMinute, oneSecond, oneWeek or endOfMonth (to enumerate months) as incday argument

logilab.common.date.datefactory(year: int, month: int, day: int, sampledate: Union[date, datetime]) → Union[date, datetime]

logilab.common.date.datetime2ticks(somedate: Union[date, datetime]) → int

logilab.common.date.datetime_to_seconds(date)

return the number of seconds since the begining of the day for that date

logilab.common.date.days_between(start: Union[date, datetime], end: Union[date, datetime]) → int

logilab.common.date.days_in_month(somedate: date) → int

logilab.common.date.days_in_year(somedate)

logilab.common.date.first_day(somedate)

logilab.common.date.get_national_holidays(begin: Union[date, datetime], end: Union[date, datetime]) → Union[List[date], List[datetime]]

return french national days off between begin and end

logilab.common.date.get_step(dateobj: Union[date, datetime], nbdays: int = 1) → timedelta

logilab.common.date.last_day(somedate: date) → date

logilab.common.date.nb_open_days(start: Union[date, datetime], end: Union[date, datetime]) → int

logilab.common.date.next_month(somedate: date, nbmonth: int = 1) → date

logilab.common.date.previous_month(somedate, nbmonth=1)

logilab.common.date.str2date(datestr: str, sampledate: Union[date, datetime]) → Union[date, datetime]

logilab.common.date.strptime_time(value, format='%H:%M')

logilab.common.date.ticks2datetime(ticks: int) → datetime

logilab.common.date.timedelta_to_days(delta)

return the time delta as a number of seconds

logilab.common.date.timedelta_to_seconds(delta)

return the time delta as a fraction of days

logilab.common.date.todate(somedate: date) → date

return a date from a date (leaving unchanged) or a datetime

logilab.common.date.todatetime(somedate)

return a date from a date (leaving unchanged) or a datetime

logilab.common.date.totime(somedate)

return a time from a time (leaving unchanged), date or datetime

logilab.common.date.ustrftime(somedate: datetime, fmt: str = '%Y-%m-%d') → str

like strftime, but returns a unicode string instead of an encoded string which may be problematic with localized date.

logilab.common.date.utcdatetime(dt: datetime) → datetime

logilab.common.date.utctime(dt)

logilab.common.date.weekday(dateobj: Union[date, datetime]) → int

logilab.common.debugger module

Customized version of pdb’s default debugger.

• sets up a history file

• uses ipython if available to colorize lines of code

• overrides list command to search for current block instead of using 5 lines of context

class logilab.common.debugger.Debugger(tcbk=None)

Bases: Pdb

custom debugger

• sets up a history file

• uses ipython if available to colorize lines of code

• overrides list command to search for current block instead of using 5 lines of context

attr_matches(text, namespace)

implementation coming from rlcompleter.Completer.attr_matches Compute matches when text contains a dot.

Assuming the text is of the form NAME.NAME….[NAME], and is evaluatable in self.namespace, it will be evaluated and its attributes (as revealed by dir()) are used as possible completions. (For class instances, class members are also considered.)

WARNING: this can still invoke arbitrary C code, if an object with a __getattr__ hook is evaluated.

complete_p(text, line, begin_idx, end_idx)

provide variable names completion for the p command

do_l(arg)

overrides default list command to display the surrounding block instead of 5 lines of context

do_list(arg)

overrides default list command to display the surrounding block instead of 5 lines of context

do_o(arg)

opens source file corresponding to the current stack level

do_open(arg)

opens source file corresponding to the current stack level

get_class_members(klass)

implementation coming from rlcompleter.get_class_members

set_quit()

quit hook: save commands in the history file

setup(frame, tcbk)

setup hook: set up history file

setup_history_file()

if readline is available, read pdb history file

start()

starts the interactive mode

logilab.common.debugger.colorize(source, start_lineno, curlineno)

fallback colorize function

logilab.common.debugger.colorize_source(source)

logilab.common.debugger.getsource(obj)

Return the text of the source code for an object.

The argument may be a module, class, method, function, traceback, frame, or code object. The source code is returned as a single string. An IOError is raised if the source code cannot be retrieved.

logilab.common.debugger.pm()

use our custom debugger

logilab.common.debugger.set_trace()

logilab.common.decorators module

A few useful function/method decorators.

logilab.common.decorators.cached(callableobj: None = None, keyarg: Optional[int] = None, **kwargs: Any) → Callable[[_T], _T]

logilab.common.decorators.cached(callableobj: _T = None, keyarg: Optional[int] = None, **kwargs: Any) → _T

Simple decorator to cache result of method call.

class logilab.common.decorators.cached_decorator(cacheattr: Optional[str] = None, keyarg: Optional[int] = None)

Bases: object

class logilab.common.decorators.cachedproperty(wrapped)

Bases: object

wrapped

class logilab.common.decorators.classproperty(get)

Bases: object

this is a simple property-like class but for class attributes.

logilab.common.decorators.clear_cache(obj, funcname)

Clear a cache handled by the cached() decorator. If ‘x’ class has @cached on its method foo, type

>>> clear_cache(x, 'foo')

to purge this method’s cache on the instance.

logilab.common.decorators.copy_cache(obj, funcname, cacheobj)

Copy cache for <funcname> from cacheobj to obj.

logilab.common.decorators.get_cache_impl(obj, funcname)

class logilab.common.decorators.iclassmethod(func)

Bases: object

Descriptor for method which should be available as class method if called on the class or instance method if called on an instance.

logilab.common.decorators.locked(acquire, release)

Decorator taking two methods to acquire/release a lock as argument, returning a decorator function which will call the inner method after having called acquire(self) et will call release(self) afterwards.

logilab.common.decorators.monkeypatch(klass: type, methodname: Optional[str] = None) → Callable

Decorator extending class with the decorated callable. This is basically a syntactic sugar vs class assignment.

>>> class A:
...     pass
>>> @monkeypatch(A)
... def meth(self):
...     return 12
...
>>> a = A()
>>> a.meth()
12
>>> @monkeypatch(A, 'foo')
... def meth(self):
...     return 12
...
>>> a.foo()
12

logilab.common.decorators.timed(f)

class logilab.common.decorators.wproperty(setfunc)

Bases: object

Simple descriptor expecting to take a modifier function as first argument and looking for a _<function name> to retrieve the attribute.

logilab.common.deprecation module

Deprecation utilities.

class logilab.common.deprecation.CallableDeprecatedCallable(*args, **kwds)

Bases: Protocol

class logilab.common.deprecation.DeprecationWarningKind(value)

Bases: Enum

An enumeration.

ARGUMENT = 'argument'

ATTRIBUTE = 'attribute'

CALLABLE = 'callable'

CLASS = 'class'

MODULE = 'module'

class logilab.common.deprecation.DeprecationWarningOperation(value)

Bases: Enum

An enumeration.

DEPRECATED = 'deprecated'

MOVED = 'moved'

REMOVED = 'removed'

RENAMED = 'renamed'

class logilab.common.deprecation.DeprecationWrapper(proxied: Any, msg: Optional[str] = None, version: Optional[str] = None)

Bases: object

proxy to print a warning on access to any attribute of the wrapped object

class logilab.common.deprecation.FakeDistribution(*args, **kwargs)

Bases: Distribution

see https://github.com/python/importlib_metadata/blob/main/CHANGES.rst#v600

locate_file()

Given a path to a file in this distribution, return a path to it.

read_text()

Attempt to load metadata file given by the name.

Parameters:

filename – The name of the file in the distribution info.

Returns:

The text if found, otherwise None.

exception logilab.common.deprecation.StructuredDeprecationWarning(reason: str, package: Optional[str] = None, version: Optional[str] = None)

Bases: DeprecationWarning

Base class for all structured DeprecationWarning Mostly used with isinstance

exception logilab.common.deprecation.TargetDeprecatedDeprecationWarning(reason: str, kind: DeprecationWarningKind, package: Optional[str] = None, version: Optional[str] = None)

Bases: StructuredDeprecationWarning

exception logilab.common.deprecation.TargetMovedDeprecationWarning(reason: str, kind: DeprecationWarningKind, old_name: str, new_name: str, old_module: str, new_module: str, package: Optional[str] = None, version: Optional[str] = None)

Bases: StructuredDeprecationWarning

exception logilab.common.deprecation.TargetRemovedDeprecationWarning(reason: str, kind: DeprecationWarningKind, name: str, package: Optional[str] = None, version: Optional[str] = None)

Bases: StructuredDeprecationWarning

exception logilab.common.deprecation.TargetRenamedDeprecationWarning(reason: str, kind: DeprecationWarningKind, old_name: str, new_name: str, package: Optional[str] = None, version: Optional[str] = None)

Bases: StructuredDeprecationWarning

logilab.common.deprecation.argument_removed(old_argument_name: str, version: Optional[str] = None) → Callable

callable decorator to allow getting backward compatibility for renamed keyword arguments.

>>> @argument_removed("old")
... def some_function(new):
...     return new
>>> some_function(old=42)
sample.py:15: DeprecationWarning: argument old of callable some_function has been renamed and
is deprecated, use keyword argument new instead some_function(old=42)
42

logilab.common.deprecation.argument_renamed(old_name: str, new_name: str, version: Optional[str] = None) → Callable

callable decorator to allow getting backward compatibility for renamed keyword arguments.

>>> @argument_renamed(old_name="old", new_name="new")
... def some_function(new):
...     return new
>>> some_function(old=42)
sample.py:15: DeprecationWarning: argument old of callable some_function has been renamed and
is deprecated, use keyword argument new instead
  some_function(old=42)
42

logilab.common.deprecation.attribute_renamed(old_name: str, new_name: str, version: Optional[str] = None) → Callable

class decorator to allow getting backward compatibility for renamed attributes.

>>> @attribute_renamed(old_name="old", new_name="new")
... class SomeClass:
...     def __init__(self):
...         self.new = 42

>>> some_class = SomeClass()
>>> print(some_class.old)
sample.py:15: DeprecationWarning: SomeClass.old has been renamed and is deprecated, use
SomeClass.new instead
  print(some_class.old)
42
>>> some_class.old = 43
sample.py:16: DeprecationWarning: SomeClass.old has been renamed and is deprecated, use
SomeClass.new instead
  some_class.old = 43
>>> some_class.old == some_class.new
True

logilab.common.deprecation.callable_deprecated(reason: Optional[str] = None, version: Optional[str] = None, stacklevel: int = 2) → Callable

Display a deprecation message only if the version is older than the compatible version.

logilab.common.deprecation.callable_moved(module_name: str, object_name: str, version: Optional[str] = None, stacklevel: int = 2, new_name: Optional[str] = None) → Callable

use to tell that a callable has been moved to a new module.

It returns a callable wrapper, so that when its called a warning is printed telling where the object can be found, import is done (and not before) and the actual object is called.

NOTE: the usage is somewhat limited on classes since it will fail if the wrapper is use in a class ancestors list, use the class_moved function instead (which has no lazy import feature though).

logilab.common.deprecation.callable_renamed(old_name: str, new_function: Callable, version: Optional[str] = None) → Callable

use to tell that a callable has been renamed.

It returns a callable wrapper, so that when its called a warning is printed telling what is the object new name.

>>> old_function = renamed('old_function', new_function)
>>> old_function()
sample.py:57: DeprecationWarning: old_function has been renamed and is deprecated, uses
new_function instead old_function()
>>>

logilab.common.deprecation.class_deprecated

alias of _class_deprecated

logilab.common.deprecation.class_moved(new_class: type, old_name: Optional[str] = None, message: Optional[str] = None, version: Optional[str] = None) → type

nice wrapper around class_renamed when a class has been moved into another module

logilab.common.deprecation.class_renamed(old_name: str, new_class: type, message: ~typing.Optional[str] = None, version: ~typing.Optional[str] = None, module_name: ~typing.Optional[str] = None, deprecated_warning_class=<class 'logilab.common.deprecation.TargetRenamedDeprecationWarning'>, deprecated_warning_kwargs=None) → type

automatically creates a class which fires a DeprecationWarning when instantiated.

>>> Set = class_renamed('Set', set, 'Set is now replaced by set')
>>> s = Set()
sample.py:57: DeprecationWarning: Set is now replaced by set
s = Set()
>>>

logilab.common.deprecation.get_real__module__(some_callable: Callable) → str

logilab.common.deprecation.get_real__name__(some_callable: Callable) → str

logilab.common.deprecation.lazy_wraps(wrapped: Callable) → Callable

This is the equivalent of the @wraps decorator of functools except it won’t try to grabs attributes of the targeted function on decoration but on access.

This is needed because of logilab.common.modutils.LazyObject.

Indeed: if you try to decorate a LazyObject with @wraps, wraps will try to access attributes of LazyObject and this will trigger the attempt to import the module decorated by LazyObject which you don’t want to do when you just want to mark this LazyObject has been a deprecated objet that you only wants to trigger if the user try to use it.

Usage: like @wraps()

>>> @lazy_wraps(function)
>>> def wrapper(*args, **kwargs): ...

logilab.common.deprecation.send_warning(reason: str, deprecation_class: Type[DeprecationWarning], deprecation_class_kwargs: Dict[str, Any], version: Optional[str] = None, stacklevel: int = 2, module_name: Optional[str] = None) → None

Display a deprecation message only if the version is older than the compatible version.

logilab.common.fileutils module

File and file-path manipulation utilities.

group path manipulation:

first_level_directory, relative_path, is_binary,get_by_ext, remove_dead_links

group file manipulation:

norm_read, norm_open, lines, stream_lines, lines,write_open_mode, ensure_fs_mode, export

sort:

path manipulation, file manipulation

class logilab.common.fileutils.ProtectedFile(filepath: str, mode: str)

Bases: FileIO

A special file-object class that automatically does a ‘chmod +w’ when needed.

XXX: for now, the way it is done allows ‘normal file-objects’ to be created during the ProtectedFile object lifetime. One way to circumvent this would be to chmod / unchmod on each write operation.

One other way would be to :

• catch the IOError in the __init__

• if IOError, then create a StringIO object

• each write operation writes in this StringIO object

• on close()/del(), write/append the StringIO content to the file and do the chmod only once

close() → None

restore mode before closing

exception logilab.common.fileutils.UnresolvableError

Bases: Exception

Exception raised by relative path when it’s unable to compute relative path between two paths.

logilab.common.fileutils.abspath_listdir(path)

Lists path’s content using absolute paths.

logilab.common.fileutils.ensure_fs_mode(filepath, desired_mode=128)

Check that the given file has the given mode(s) set, else try to set it.

Parameters:

• filepath (str) – path of the file

• desired_mode (int) – ORed flags describing the desired mode. Use constants from the stat module for file permission’s modes

logilab.common.fileutils.export(from_dir: str, to_dir: str, blacklist: Tuple[str, str, str, str, str, str, str, str] = ('CVS', '.svn', '.hg', '.git', '.tox', 'debian', 'dist', 'build'), ignore_ext: Tuple[str, str, str, str, str, str] = ('.pyc', '.pyo', '.elc', '~', '.swp', '.orig'), verbose: int = 0) → None

Make a mirror of from_dir in to_dir, omitting directories and files listed in the black list or ending with one of the given extensions.

Parameters:

• from_dir (str) – directory to export

• to_dir (str) – destination directory

• blacklist (list or tuple) – list of files or directories to ignore, default to the content of BASE_BLACKLIST

• ignore_ext (list or tuple) – list of extensions to ignore, default to the content of IGNORED_EXTENSIONS

• verbose (bool) – flag indicating whether information about exported files should be printed to stderr, default to False

logilab.common.fileutils.first_level_directory(path: str) → str

Return the first level directory of a path.

>>> first_level_directory('home/syt/work')
'home'
>>> first_level_directory('/home/syt/work')
'/'
>>> first_level_directory('work')
'work'
>>>

Parameters:

path (str) – the path for which we want the first level directory

Return type:

str

Returns:

the first level directory appearing in path

logilab.common.fileutils.is_binary(filename: str) → int

Return true if filename may be a binary file, according to it’s extension.

Parameters:

filename (str) – the name of the file

Return type:

bool

Returns:

true if the file is a binary file (actually if it’s mime type isn’t beginning by text/)

logilab.common.fileutils.lines(path: str, comments: Optional[str] = None) → List[str]

Return a list of non empty lines in the file located at path.

Parameters:

• path (str) – path to the file

• comments (str or None) – optional string which can be used to comment a line in the file (i.e. lines starting with this string won’t be returned)

Return type:

list

Returns:

a list of stripped line in the file, without empty and commented lines

Warning:

at some point this function will probably return an iterator

logilab.common.fileutils.relative_path(from_file, to_file)

Try to get a relative path from from_file to to_file (path will be absolute if to_file is an absolute file). This function is useful to create link in from_file to to_file. This typical use case is used in this function description.

If both files are relative, they’re expected to be relative to the same directory.

>>> relative_path( from_file='toto/index.html', to_file='index.html')
'../index.html'
>>> relative_path( from_file='index.html', to_file='toto/index.html')
'toto/index.html'
>>> relative_path( from_file='tutu/index.html', to_file='toto/index.html')
'../toto/index.html'
>>> relative_path( from_file='toto/index.html', to_file='/index.html')
'/index.html'
>>> relative_path( from_file='/toto/index.html', to_file='/index.html')
'../index.html'
>>> relative_path( from_file='/toto/index.html', to_file='/toto/summary.html')
'summary.html'
>>> relative_path( from_file='index.html', to_file='index.html')
''
>>> relative_path( from_file='/index.html', to_file='toto/index.html')
Traceback (most recent call last):
  File "<string>", line 1, in ?
  File "<stdin>", line 37, in relative_path
UnresolvableError
>>> relative_path( from_file='/index.html', to_file='/index.html')
''
>>>

Parameters:

• from_file (str) – source file (where links will be inserted)

• to_file (str) – target file (on which links point)

Raises:

UnresolvableError – if it has been unable to guess a correct path

Return type:

str

Returns:

the relative path of to_file from from_file

logilab.common.fileutils.remove_dead_links(directory, verbose=0)

Recursively traverse directory and remove all dead links.

Parameters:

• directory (str) – directory to cleanup

• verbose (bool) – flag indicating whether information about deleted links should be printed to stderr, default to False

logilab.common.fileutils.stream_lines(stream: TextIOWrapper, comments: Optional[str] = None) → List[str]

Return a list of non empty lines in the given stream.

Parameters:

• stream (object implementing 'xreadlines' or 'readlines') – file like object

• comments (str or None) – optional string which can be used to comment a line in the file (i.e. lines starting with this string won’t be returned)

Return type:

list

Returns:

a list of stripped line in the file, without empty and commented lines

Warning:

at some point this function will probably return an iterator

logilab.common.fileutils.write_open_mode(filename: str) → str

Return the write mode that should used to open file.

Parameters:

filename (str) – the name of the file

Return type:

str

Returns:

the mode that should be use to open the file (‘w’ or ‘wb’)

logilab.common.graph module

Graph manipulation utilities.

(dot generation adapted from pypy/translator/tool/make_dot.py)

class logilab.common.graph.DotBackend(graphname, rankdir=None, size=None, ratio=None, charset='utf-8', renderer='dot', additionnal_param={})

Bases: object

Dot File backend.

emit(line)

Adds <line> to final output.

emit_edge(name1, name2, **props)

emit an edge from <name1> to <name2>. edge properties: see http://www.graphviz.org/doc/info/attrs.html

emit_node(name, **props)

emit a node with given properties. node properties: see http://www.graphviz.org/doc/info/attrs.html

generate(outputfile=None, dotfile=None, mapfile=None)

Generates a graph file.

Parameters:

• outputfile – filename and path [defaults to graphname.png]

• dotfile – filename and path [defaults to graphname.dot]

Return type:

str

Returns:

a path to the generated file

get_source()

returns self._source

property source

returns self._source

class logilab.common.graph.GraphGenerator(backend)

Bases: object

generate(visitor, propshdlr, outputfile=None, mapfile=None)

exception logilab.common.graph.UnorderableGraph

Bases: Exception

logilab.common.graph.escape(value)

Make <value> usable in a dot file.

logilab.common.graph.get_cycles(graph_dict: Dict[V, List[V]], vertices: Optional[Iterable] = None) → List[List]

given a dictionary representing an ordered graph (i.e. key are vertices and values is a list of destination vertices representing edges), return a list of detected cycles

logilab.common.graph.has_path(graph_dict: Dict[str, List[str]], fromnode: str, tonode: str, path: Optional[List[str]] = None) → Optional[List[str]]

generic function taking a simple graph definition as a dictionary, with node has key associated to a list of nodes directly reachable from it.

Return None if no path exists to go from fromnode to tonode, else the first path found (as a list including the destination node at last)

logilab.common.graph.normalize_node_id(nid)

Returns a suitable DOT node id for nid.

logilab.common.graph.ordered_nodes(graph: Dict[V, List[V]]) → Tuple[V, ...]

takes a dependency graph dict as arguments and return an ordered tuple of nodes starting with nodes without dependencies and up to the outermost node.

If there is some cycle in the graph, UnorderableGraph will be raised.

Also the given graph dict will be emptied.

logilab.common.graph.target_info_from_filename(filename)

Transforms /some/path/foo.png into (‘/some/path’, ‘foo.png’, ‘png’).

logilab.common.interface module

Bases class for interfaces to provide ‘light’ interface handling.

TODO:

_ implements a check method which check that an object implements the interface _ Attribute objects

This module requires at least python 2.2

class logilab.common.interface.Interface

Bases: object

Base class for interfaces.

classmethod is_implemented_by(instance: type) → bool

logilab.common.interface.extend(klass: type, interface: type, _recurs: bool = False) → None

Add interface to klass’__implements__ if not already implemented in.

If klass is subclassed, ensure subclasses __implements__ it as well.

NOTE: klass should be e new class.

logilab.common.interface.implements(obj: type, interface: type) → bool

Return true if the give object (maybe an instance or class) implements the interface.

logilab.common.logging_ext module

Extends the logging module from the standard library.

class logilab.common.logging_ext.ColorFormatter(fmt=None, datefmt=None, colors=None)

Bases: Formatter

A color Formatter for the logging standard module.

By default, colorize CRITICAL and ERROR in red, WARNING in orange, INFO in green and DEBUG in yellow.

self.colors is customizable via the ‘color’ constructor argument (dictionary).

self.colorfilters is a list of functions that get the LogRecord and return a color name or None.

format(record)

Format the specified record as text.

The record’s attribute dictionary is used as the operand to a string formatting operation which yields the returned string. Before formatting the dictionary, a couple of preparatory steps are carried out. The message attribute of the record is computed using LogRecord.getMessage(). If the formatting string uses the time (as determined by a call to usesTime(), formatTime() is called to format the event time. If there is exception information, it is formatted using formatException() and appended to the message.

logilab.common.logging_ext.get_formatter(logformat='%(asctime)s - (%(name)s) %(levelname)s: %(message)s', logdateformat='%Y-%m-%d %H:%M:%S')

logilab.common.logging_ext.get_handler(debug=False, syslog=False, logfile=None, rotation_parameters=None)

get an apropriate handler according to given parameters

logilab.common.logging_ext.get_threshold(debug=False, logthreshold=None)

logilab.common.logging_ext.init_log(debug=False, syslog=False, logthreshold=None, logfile=None, logformat='%(asctime)s - (%(name)s) %(levelname)s: %(message)s', logdateformat='%Y-%m-%d %H:%M:%S', fmt=None, rotation_parameters=None, handler=None)

init the log service

logilab.common.logging_ext.set_color_formatter(logger=None, **kw)

Install a color formatter on the ‘logger’. If not given, it will defaults to the default logger.

Any additional keyword will be passed as-is to the ColorFormatter constructor.

logilab.common.logging_ext.set_log_methods(cls, logger)

bind standard logger’s methods as methods on the class

logilab.common.logging_ext.xxx_cyan(record)

logilab.common.modutils module

Python modules manipulation utility functions.

class logilab.common.modutils.LazyObject(module, obj)

Bases: object

This class allows to lazyly declare a object (most likely only a callable according to the code) from a module without importing it.

The import will be triggered when the user tries to access attributes of the object/callable or call it.

Trying to set or delete attributes of the wrapped object/callable will not works as expected.

logilab.common.modutils.clean_sys_modules(names)

remove submodules starting with name from names from sys.modules

logilab.common.modutils.cleanup_sys_modules(directories)

remove submodules of directories from sys.modules

logilab.common.modutils.get_module_files(src_directory: str, blacklist: Sequence[str] = ('CVS', '.svn', '.hg', '.git', '.tox', 'debian', 'dist', 'build')) → List[str]

given a package directory return a list of all available python module’s files in the package and its subpackages

Parameters:

• src_directory (str) – path of the directory corresponding to the package

• blacklist (list or tuple) – optional list of files or directory to ignore, default to the value of logilab.common.STD_BLACKLIST

Return type:

list

Returns:

the list of all available python module’s files in the package and its subpackages

logilab.common.optik_ext module

Add an abstraction level to transparently import optik classes from optparse (python >= 2.3) or the optik package.

It also defines three new types for optik/optparse command line parser :

• regexp argument of this type will be converted using re.compile

• csv argument of this type will be converted using split(‘,’)

• yn argument of this type will be true if ‘y’ or ‘yes’, false if ‘n’ or ‘no’

• named argument of this type are in the form <NAME>=<VALUE> or <NAME>:<VALUE>

• password argument of this type wont be converted but this is used by other tools such as interactive prompt for configuration to double check value and use an invisible field

• multiple_choice same as default “choice” type but multiple choices allowed

• file argument of this type wont be converted but checked that the given file exists

• color argument of this type wont be converted but checked its either a named color or a color specified using hexadecimal notation (preceded by a #)

• time argument of this type will be converted to a float value in seconds according to time units (ms, s, min, h, d)

• bytes argument of this type will be converted to a float value in bytes according to byte units (b, kb, mb, gb, tb)

class logilab.common.optik_ext.Option(*opts: str, **attrs: Any)

Bases: Option

override optik.Option to add some new option types

ATTRS = ['action', 'type', 'dest', 'default', 'nargs', 'const', 'choices', 'callback', 'callback_args', 'callback_kwargs', 'help', 'metavar', 'hide', 'level']

TYPES = ('string', 'int', 'long', 'float', 'complex', 'choice', 'regexp', 'csv', 'yn', 'named', 'password', 'multiple_choice', 'file', 'color', 'time', 'bytes')

TYPE_CHECKER = {'bytes': <function check_bytes>, 'choice': <function check_choice>, 'color': <function check_color>, 'complex': <function check_builtin>, 'csv': <function check_csv>, 'file': <function check_file>, 'float': <function check_builtin>, 'int': <function check_builtin>, 'long': <function check_builtin>, 'multiple_choice': <function check_csv>, 'named': <function check_named>, 'password': <function check_password>, 'regexp': <function check_regexp>, 'time': <function check_time>, 'yn': <function check_yn>}

process(opt: str, value: str, values: Values, parser: OptionParser) → int

class logilab.common.optik_ext.OptionGroup(parser, title, description=None)

Bases: OptionContainer

destroy()

see OptionParser.destroy().

format_help(formatter)

level = 0

set_title(title)

class logilab.common.optik_ext.OptionParser(option_class: type = <class 'logilab.common.optik_ext.Option'>, *args: ~typing.Any, **kwargs: ~typing.Any)

Bases: OptionParser

override optik.OptionParser to use our Option class

format_option_help(formatter: Optional[HelpFormatter] = None) → str

exception logilab.common.optik_ext.OptionValueError(msg)

Bases: OptParseError

Raised if an invalid option value is encountered on the command line.

class logilab.common.optik_ext.Values(defaults=None)

Bases: object

ensure_value(attr, value)

read_file(filename, mode='careful')

read_module(modname, mode='careful')

logilab.common.optparser module

logilab.common.proc module

module providing: * process information (linux specific: rely on /proc) * a class for resource control (memory / time / cpu time)

This module doesn’t work on windows platforms (only tested on linux)

organization:

Logilab

exception logilab.common.proc.LineageMemoryError

Bases: ResourceError

Error raised when the total amount of memory used by a process and it’s child is reached

limit = 'Lineage total Memory'

class logilab.common.proc.MemorySentinel(interval, memory_limit, gpid=None)

Bases: Thread

A class checking a process don’t use too much memory in a separated daemonic thread

stop()

stop ap

exception logilab.common.proc.NoSuchProcess

Bases: Exception

class logilab.common.proc.ProcInfo(pid)

Bases: Node

provide access to process information found in /proc

age()

return the age of the process

lineage_memory_usage()

memory_usage()

return the memory usage of the process in Ko

name()

return the process name found in /proc/<pid>/stat

status()

return the list of fields found in /proc/<pid>/stat

time(children=0)

return the number of jiffies that this process has been scheduled in user and kernel mode

class logilab.common.proc.ProcInfoLoader

Bases: object

manage process information

list_pids()

return a list of existent process ids

load(pid)

get a ProcInfo object for a given pid

load_all()

load all processes information

class logilab.common.proc.ResourceController(max_cpu_time=None, max_time=None, max_memory=None, max_reprieve=60)

Bases: object

clean_limit()

reinstall the old process limit

setup_limit()

set up the process limit

exception logilab.common.proc.ResourceError

Bases: Exception

Error raise when resource limit is reached

limit = 'Unknown Resource Limit'

exception logilab.common.proc.TimeoutError

Bases: ResourceError

Error raised when the process is running for to much time

limit = 'Real Time'

exception logilab.common.proc.XCPUError

Bases: ResourceError

Error raised when CPU Time limit is reached

limit = 'CPU Time'

logilab.common.proc.proc_exists(pid)

check the a pid is registered in /proc raise NoSuchProcess exception if not

logilab.common.pytest module

logilab.common.registry module

This module provides bases for predicates dispatching (the pattern in use here is similar to what’s refered as multi-dispatch or predicate-dispatch in the literature, though a bit different since the idea is to select across different implementation ‘e.g. classes), not to dispatch a message to a function or method. It contains the following classes:

• RegistryStore, the top level object which loads implementation objects and stores them into registries. You’ll usually use it to access registries and their contained objects;

• Registry, the base class which contains objects semantically grouped (for instance, sharing a same API, hence the ‘implementation’ name). You’ll use it to select the proper implementation according to a context. Notice you may use registries on their own without using the store.

Note

implementation objects are usually designed to be accessed through the registry and not by direct instantiation, besides to use it as base classe.

The selection procedure is delegated to a selector, which is responsible for scoring the object according to some context. At the end of the selection, if an implementation has been found, an instance of this class is returned. A selector is built from one or more predicates combined together using AND, OR, NOT operators (actually &, | and ~). You’ll thus find some base classes to build predicates:

• Predicate, the abstract base predicate class

• AndPredicate, OrPredicate, NotPredicate, which you shouldn’t have to use directly. You’ll use &, | and ‘~’ operators between predicates directly

• objectify_predicate()

You’ll eventually find one concrete predicate: yes

class logilab.common.registry.RegistryStore(debugmode: bool = False)

This class is responsible for loading objects and storing them in their registry which is created on the fly as needed.

It handles dynamic registration of objects and provides a convenient api to access them. To be recognized as an object that should be stored into one of the store’s registry (Registry), an object must provide the following attributes, used control how they interact with the registry:

__registries__

list of registry names (string like ‘views’, ‘templates’…) into which the object should be registered

__regid__

object identifier in the registry (string like ‘main’, ‘primary’, ‘folder_box’)

__select__

the object predicate selectors

Moreover, the __abstract__ attribute may be set to True to indicate that an object is abstract and should not be registered (such inherited attributes not considered).

Note

When using the store to load objects dynamically, you always have to use super() to get the methods and attributes of the superclasses, and not use the class identifier. If not, you’ll get into trouble at reload time.

For example, instead of writing:

class Thing(Parent):
    __regid__ = 'athing'
    __select__ = yes()

    def f(self, arg1):
        Parent.f(self, arg1)

You must write:

class Thing(Parent):
    __regid__ = 'athing'
    __select__ = yes()

    def f(self, arg1):
        super(Thing, self).f(arg1)

Controlling object registration

Dynamic loading is triggered by calling the register_modnames() method, given a list of modules names to inspect.

register_modnames(modnames: List[str]) → None

register all objects found in <modnames>

For each module, by default, all compatible objects are registered automatically. However if some objects come as replacement of other objects, or have to be included only if some condition is met, you’ll have to define a registration_callback(vreg) function in the module and explicitly register all objects in this module, using the api defined below.

register_all(objects: Iterable, modname: str, butclasses: Sequence = ()) → None

register registrable objects into objects.

Registrable objects are properly configured subclasses of RegistrableObject. Objects which are not defined in the module modname or which are in butclasses won’t be registered.

Typical usage is:

store.register_all(globals().values(), __name__, (ClassIWantToRegisterExplicitly,))

So you get partially automatic registration, keeping manual registration for some object (to use register_and_replace() for instance).

register_and_replace(obj, replaced, registryname=None)

register obj object into registryname or obj.__registries__ if not specified. If found, the replaced object will be unregistered first (else a warning will be issued as it is generally unexpected).

register(obj: Any, registryname: Optional[Any] = None, oid: Optional[Any] = None, clear: bool = False) → None

register obj implementation into registryname or obj.__registries__ if not specified, with identifier oid or obj.__regid__ if not specified.

If clear is true, all objects with the same identifier will be previously unregistered.

unregister(obj, registryname=None)

unregister obj object from the registry registryname or obj.__registries__ if not specified.

Note

Once the function registration_callback(vreg) is implemented in a module, all the objects from this module have to be explicitly registered as it disables the automatic object registration.

Examples:

def registration_callback(store):
   # register everything in the module except BabarClass
   store.register_all(globals().values(), __name__, (BabarClass,))

   # conditionally register BabarClass
   if 'babar_relation' in store.schema:
       store.register(BabarClass)

In this example, we register all application object classes defined in the module except BabarClass. This class is then registered only if the ‘babar_relation’ relation type is defined in the instance schema.

def registration_callback(store):
   store.register(Elephant)
   # replace Babar by Celeste
   store.register_and_replace(Celeste, Babar)

In this example, we explicitly register classes one by one:

• the Elephant class

• the Celeste to replace Babar

If at some point we register a new appobject class in this module, it won’t be registered at all without modification to the registration_callback implementation. The first example will register it though, thanks to the call to the register_all method.

Controlling registry instantiation

The REGISTRY_FACTORY class dictionary allows to specify which class should be instantiated for a given registry name. The class associated to None key will be the class used when there is no specific class for a name.

class logilab.common.registry.Registry(debugmode: bool)

The registry store a set of implementations associated to identifier:

• to each identifier are associated a list of implementations

• to select an implementation of a given identifier, you should use one of the select() or select_or_none() method

• to select a list of implementations for a context, you should use the possible_objects() method

• dictionary like access to an identifier will return the bare list of implementations for this identifier.

To be usable in a registry, the only requirement is to have a __select__ attribute.

At the end of the registration process, the __registered__() method is called on each registered object which have them, given the registry in which it’s registered as argument.

Registration methods:

register(obj: Any, oid: Optional[Any] = None, clear: bool = False) → None

base method to add an object in the registry

unregister(obj)

remove object <obj> from this registry

Selection methods:

select(_Registry__oid, *args, **kwargs)

return the most specific object among those with the given oid according to the given context.

raise ObjectNotFound if there are no object with id oid in this registry

raise NoSelectableObject if no object can be selected

select_or_none(_Registry__oid, *args, **kwargs)

return the most specific object among those with the given oid according to the given context, or None if no object applies.

possible_objects(*args, **kwargs)

return an iterator on possible objects in this registry for the given context

object_by_id(oid, *args, **kwargs)

return object with the oid identifier. Only one object is expected to be found.

raise ObjectNotFound if there are no object with id oid in this registry

raise AssertionError if there is more than one object there

Predicates

class logilab.common.registry.Predicate

base class for selector classes providing implementation for operators &, | and ~

This class is only here to give access to binary operators, the selector logic itself should be implemented in the __call__() method. Notice it should usually accept any arbitrary arguments (the context), though that may vary depending on your usage of the registry.

a selector is called to help choosing the correct object for a particular context by returning a score (int) telling how well the implementation given as first argument fit to the given context.

0 score means that the class doesn’t apply.

logilab.common.registry.objectify_predicate(selector_func: Callable) → Any

Most of the time, a simple score function is enough to build a selector. The objectify_predicate() decorator turn it into a proper selector class:

@objectify_predicate
def one(cls, req, rset=None, **kwargs):
    return 1

class MyView(View):
    __select__ = View.__select__ & one()

class logilab.common.registry.yes(score: float = 0.5)

Return the score given as parameter, with a default score of 0.5 so any other selector take precedence.

Usually used for objects which can be selected whatever the context, or also sometimes to add arbitrary points to a score.

Take care, yes(0) could be named ‘no’…

class logilab.common.registry.AndPredicate(*selectors: Any)

and-chained selectors

class logilab.common.registry.OrPredicate(*selectors: Any)

or-chained selectors

class logilab.common.registry.NotPredicate(selector)

negation selector

Debugging

class logilab.common.registry.traced_selection(traced='all')

Typical usage is :

>>> from logilab.common.registry import traced_selection
>>> with traced_selection():
...     # some code in which you want to debug selectors
...     # for all objects

This will yield lines like this in the logs:

selector one_line_rset returned 0 for <class 'elephant.Babar'>

You can also give to traced_selection the identifiers of objects on which you want to debug selection (‘oid1’ and ‘oid2’ in the example above).

>>> with traced_selection( ('regid1', 'regid2') ):
...     # some code in which you want to debug selectors
...     # for objects with __regid__ 'regid1' and 'regid2'

A potentially useful point to set up such a tracing function is the logilab.common.registry.Registry.select method body.

Exceptions

class logilab.common.registry.RegistryException

Base class for registry exception.

class logilab.common.registry.RegistryNotFound

Raised when an unknown registry is requested.

This is usually a programming/typo error.

class logilab.common.registry.ObjectNotFound

Raised when an unregistered object is requested.

This may be a programming/typo or a misconfiguration error.

class logilab.common.registry.NoSelectableObject(args, kwargs, objects)

Raised when no object is selectable for a given context.

class logilab.common.registry.AndPredicate(*selectors: Any)

Bases: MultiPredicate

and-chained selectors

class logilab.common.registry.MultiPredicate(*selectors: Any)

Bases: Predicate

base class for compound selector classes

classmethod merge_selectors(selectors: Sequence[Predicate]) → List[Predicate]

deal with selector instanciation when necessary and merge multi-selectors if possible:

AndPredicate(AndPredicate(sel1, sel2), AndPredicate(sel3, sel4)) ==> AndPredicate(sel1, sel2, sel3, sel4)

search_selector(selector: Predicate) → Optional[Predicate]

search for the given selector or selector instance (or tuple of selectors) in the selectors tree. Return None if not found

exception logilab.common.registry.NoSelectableObject(args, kwargs, objects)

Bases: RegistryException

Raised when no object is selectable for a given context.

class logilab.common.registry.NotPredicate(selector)

Bases: Predicate

negation selector

exception logilab.common.registry.ObjectNotFound

Bases: RegistryException

Raised when an unregistered object is requested.

This may be a programming/typo or a misconfiguration error.

class logilab.common.registry.OrPredicate(*selectors: Any)

Bases: MultiPredicate

or-chained selectors

class logilab.common.registry.Predicate

Bases: object

base class for selector classes providing implementation for operators &, | and ~

This class is only here to give access to binary operators, the selector logic itself should be implemented in the __call__() method. Notice it should usually accept any arbitrary arguments (the context), though that may vary depending on your usage of the registry.

a selector is called to help choosing the correct object for a particular context by returning a score (int) telling how well the implementation given as first argument fit to the given context.

0 score means that the class doesn’t apply.

property func_name

search_selector(selector: Predicate) → Optional[Predicate]

search for the given selector, selector instance or tuple of selectors in the selectors tree. Return None if not found.

class logilab.common.registry.PredicateMetaClass(*args, **kwargs)

Bases: type

class logilab.common.registry.RegistrableInstance(*args, **kwargs)

Bases: RegistrableObject

Inherit this class if you want instances of the classes to be automatically registered.

class logilab.common.registry.RegistrableObject

Bases: object

This is the base class for registrable objects which are selected according to a context.

__registry__

name of the registry for this object (string like ‘views’, ‘templates’…). You may want to define __registries__ directly if your object should be registered in several registries.

__regid__

object’s identifier in the registry (string like ‘main’, ‘primary’, ‘folder_box’)

__select__

class’selector

Moreover, the __abstract__ attribute may be set to True to indicate that a class is abstract and should not be registered.

You don’t have to inherit from this class to put it in a registry (having __regid__ and __select__ is enough), though this is needed for classes that should be automatically registered.

class logilab.common.registry.Registry(debugmode: bool)

Bases: dict

The registry store a set of implementations associated to identifier:

• to each identifier are associated a list of implementations

• to select an implementation of a given identifier, you should use one of the select() or select_or_none() method

• to select a list of implementations for a context, you should use the possible_objects() method

• dictionary like access to an identifier will return the bare list of implementations for this identifier.

To be usable in a registry, the only requirement is to have a __select__ attribute.

At the end of the registration process, the __registered__() method is called on each registered object which have them, given the registry in which it’s registered as argument.

Registration methods:

register(obj: Any, oid: Optional[Any] = None, clear: bool = False) → None

base method to add an object in the registry

unregister(obj)

remove object <obj> from this registry

Selection methods:

select(_Registry__oid, *args, **kwargs)

return the most specific object among those with the given oid according to the given context.

raise ObjectNotFound if there are no object with id oid in this registry

raise NoSelectableObject if no object can be selected

select_or_none(_Registry__oid, *args, **kwargs)

return the most specific object among those with the given oid according to the given context, or None if no object applies.

possible_objects(*args, **kwargs)

return an iterator on possible objects in this registry for the given context

object_by_id(oid, *args, **kwargs)

return object with the oid identifier. Only one object is expected to be found.

raise ObjectNotFound if there are no object with id oid in this registry

raise AssertionError if there is more than one object there

add_select_best_listener(listener)

Add a listener to the list of one parameters callables (function/method) that will be called everytime the selection Registry._select_best is done and they will recieve a dict of the following form:

{"all_objects": [], "end_score": 0, "winners": [], "winner": None or winner,
"self": self, "args": args, "kwargs": kwargs, }

all_objects()

return a list containing all objects in this registry.

critical(msg, *args, **kwargs)

Log ‘msg % args’ with severity ‘CRITICAL’.

To pass exception information, use the keyword argument exc_info with a true value, e.g.

logger.critical(“Houston, we have a %s”, “major disaster”, exc_info=1)

debug(msg, *args, **kwargs)

Log ‘msg % args’ with severity ‘DEBUG’.

To pass exception information, use the keyword argument exc_info with a true value, e.g.

logger.debug(“Houston, we have a %s”, “thorny problem”, exc_info=1)

error(msg, *args, **kwargs)

Log ‘msg % args’ with severity ‘ERROR’.

To pass exception information, use the keyword argument exc_info with a true value, e.g.

logger.error(“Houston, we have a %s”, “major problem”, exc_info=1)

exception(msg, *args, exc_info=True, **kwargs)

Convenience method for logging an ERROR with exception information.

info(msg, *args, **kwargs)

Log ‘msg % args’ with severity ‘INFO’.

To pass exception information, use the keyword argument exc_info with a true value, e.g.

logger.info(“Houston, we have a %s”, “interesting problem”, exc_info=1)

initialization_completed() → None

call method __registered__() on registered objects when the callback is defined

object_by_id(oid, *args, **kwargs)

return object with the oid identifier. Only one object is expected to be found.

raise ObjectNotFound if there are no object with id oid in this registry

raise AssertionError if there is more than one object there

classmethod objid(obj: Any) → str

returns a unique identifier for an object stored in the registry

classmethod objname(obj: Any) → str

returns a readable name for an object stored in the registry

possible_objects(*args, **kwargs)

return an iterator on possible objects in this registry for the given context

register(obj: Any, oid: Optional[Any] = None, clear: bool = False) → None

base method to add an object in the registry

register_and_replace(obj, replaced)

remove <replaced> and register <obj>

select(_Registry__oid, *args, **kwargs)

return the most specific object among those with the given oid according to the given context.

raise ObjectNotFound if there are no object with id oid in this registry

raise NoSelectableObject if no object can be selected

select_or_none(_Registry__oid, *args, **kwargs)

return the most specific object among those with the given oid according to the given context, or None if no object applies.

selected(winner, args, kwargs)

override here if for instance you don’t want “instanciation”

unregister(obj)

remove object <obj> from this registry

warning(msg, *args, **kwargs)

Log ‘msg % args’ with severity ‘WARNING’.

To pass exception information, use the keyword argument exc_info with a true value, e.g.

logger.warning(“Houston, we have a %s”, “bit of a problem”, exc_info=1)

exception logilab.common.registry.RegistryException

Bases: Exception

Base class for registry exception.

exception logilab.common.registry.RegistryNotFound

Bases: RegistryException

Raised when an unknown registry is requested.

This is usually a programming/typo error.

class logilab.common.registry.RegistryStore(debugmode: bool = False)

Bases: dict

This class is responsible for loading objects and storing them in their registry which is created on the fly as needed.

It handles dynamic registration of objects and provides a convenient api to access them. To be recognized as an object that should be stored into one of the store’s registry (Registry), an object must provide the following attributes, used control how they interact with the registry:

__registries__

list of registry names (string like ‘views’, ‘templates’…) into which the object should be registered

__regid__

object identifier in the registry (string like ‘main’, ‘primary’, ‘folder_box’)

__select__

the object predicate selectors

Moreover, the __abstract__ attribute may be set to True to indicate that an object is abstract and should not be registered (such inherited attributes not considered).

Note

When using the store to load objects dynamically, you always have to use super() to get the methods and attributes of the superclasses, and not use the class identifier. If not, you’ll get into trouble at reload time.

For example, instead of writing:

class Thing(Parent):
    __regid__ = 'athing'
    __select__ = yes()

    def f(self, arg1):
        Parent.f(self, arg1)

You must write:

class Thing(Parent):
    __regid__ = 'athing'
    __select__ = yes()

    def f(self, arg1):
        super(Thing, self).f(arg1)

Controlling object registration

Dynamic loading is triggered by calling the register_modnames() method, given a list of modules names to inspect.

register_modnames(modnames: List[str]) → None

register all objects found in <modnames>

For each module, by default, all compatible objects are registered automatically. However if some objects come as replacement of other objects, or have to be included only if some condition is met, you’ll have to define a registration_callback(vreg) function in the module and explicitly register all objects in this module, using the api defined below.

register_all(objects: Iterable, modname: str, butclasses: Sequence = ()) → None

register registrable objects into objects.

Registrable objects are properly configured subclasses of RegistrableObject. Objects which are not defined in the module modname or which are in butclasses won’t be registered.

Typical usage is:

store.register_all(globals().values(), __name__, (ClassIWantToRegisterExplicitly,))

So you get partially automatic registration, keeping manual registration for some object (to use register_and_replace() for instance).

register_and_replace(obj, replaced, registryname=None)

register obj object into registryname or obj.__registries__ if not specified. If found, the replaced object will be unregistered first (else a warning will be issued as it is generally unexpected).

register(obj: Any, registryname: Optional[Any] = None, oid: Optional[Any] = None, clear: bool = False) → None

register obj implementation into registryname or obj.__registries__ if not specified, with identifier oid or obj.__regid__ if not specified.

If clear is true, all objects with the same identifier will be previously unregistered.

unregister(obj, registryname=None)

unregister obj object from the registry registryname or obj.__registries__ if not specified.

Note

Once the function registration_callback(vreg) is implemented in a module, all the objects from this module have to be explicitly registered as it disables the automatic object registration.

Examples:

def registration_callback(store):
   # register everything in the module except BabarClass
   store.register_all(globals().values(), __name__, (BabarClass,))

   # conditionally register BabarClass
   if 'babar_relation' in store.schema:
       store.register(BabarClass)

In this example, we register all application object classes defined in the module except BabarClass. This class is then registered only if the ‘babar_relation’ relation type is defined in the instance schema.

def registration_callback(store):
   store.register(Elephant)
   # replace Babar by Celeste
   store.register_and_replace(Celeste, Babar)

In this example, we explicitly register classes one by one:

• the Elephant class

• the Celeste to replace Babar

If at some point we register a new appobject class in this module, it won’t be registered at all without modification to the registration_callback implementation. The first example will register it though, thanks to the call to the register_all method.

Controlling registry instantiation

The REGISTRY_FACTORY class dictionary allows to specify which class should be instantiated for a given registry name. The class associated to None key will be the class used when there is no specific class for a name.

REGISTRY_FACTORY: Dict[Union[None, str], type] = {None: <class 'logilab.common.registry.Registry'>}

critical(msg, *args, **kwargs)

Log ‘msg % args’ with severity ‘CRITICAL’.

To pass exception information, use the keyword argument exc_info with a true value, e.g.

logger.critical(“Houston, we have a %s”, “major disaster”, exc_info=1)

debug(msg, *args, **kwargs)

Log ‘msg % args’ with severity ‘DEBUG’.

To pass exception information, use the keyword argument exc_info with a true value, e.g.

logger.debug(“Houston, we have a %s”, “thorny problem”, exc_info=1)

error(msg, *args, **kwargs)

Log ‘msg % args’ with severity ‘ERROR’.

To pass exception information, use the keyword argument exc_info with a true value, e.g.

logger.error(“Houston, we have a %s”, “major problem”, exc_info=1)

exception(msg, *args, exc_info=True, **kwargs)

Convenience method for logging an ERROR with exception information.

info(msg, *args, **kwargs)

Log ‘msg % args’ with severity ‘INFO’.

To pass exception information, use the keyword argument exc_info with a true value, e.g.

logger.info(“Houston, we have a %s”, “interesting problem”, exc_info=1)

init_registration(path: List[str], extrapath: Optional[Any] = None) → List[Tuple[str, str]]

reset registry and walk down path to return list of (path, name) file modules to be loaded

initialization_completed() → None

call initialization_completed() on all known registries

classmethod is_registrable(obj: Any) → bool

ensure obj should be registered

as arbitrary stuff may be registered, do a lot of check and warn about weird cases (think to dumb proxy objects)

is_reload_needed(path)

return True if something module changed and the registry should be reloaded

load_file(filepath: str, modname: str) → None

load registrable objects (if any) from a python file

load_module(module: module) → None

Automatically handle module objects registration.

Instances are registered as soon as they are hashable and have the following attributes:

• __regid__ (a string)

• __select__ (a callable)

• __registries__ (a tuple/list of string)

For classes this is a bit more complicated :

• first ensure parent classes are already registered

• class with __abstract__ == True in their local dictionary are skipped

• object class needs to have registries and identifier properly set to a non empty string to be registered.

register(obj: Any, registryname: Optional[Any] = None, oid: Optional[Any] = None, clear: bool = False) → None

register obj implementation into registryname or obj.__registries__ if not specified, with identifier oid or obj.__regid__ if not specified.

If clear is true, all objects with the same identifier will be previously unregistered.

register_all(objects: Iterable, modname: str, butclasses: Sequence = ()) → None

register registrable objects into objects.

Registrable objects are properly configured subclasses of RegistrableObject. Objects which are not defined in the module modname or which are in butclasses won’t be registered.

Typical usage is:

store.register_all(globals().values(), __name__, (ClassIWantToRegisterExplicitly,))

So you get partially automatic registration, keeping manual registration for some object (to use register_and_replace() for instance).

register_and_replace(obj, replaced, registryname=None)

register obj object into registryname or obj.__registries__ if not specified. If found, the replaced object will be unregistered first (else a warning will be issued as it is generally unexpected).

register_modnames(modnames: List[str]) → None

register all objects found in <modnames>

registry_class(regid: str) → type

return existing registry named regid or use factory to create one and return it

reset() → None

clear all registries managed by this store

setdefault(regid: str) → Registry

Insert key with a value of default if key is not in the dictionary.

Return the value for key if key is in the dictionary, else default.

unregister(obj, registryname=None)

unregister obj object from the registry registryname or obj.__registries__ if not specified.

warning(msg, *args, **kwargs)

Log ‘msg % args’ with severity ‘WARNING’.

To pass exception information, use the keyword argument exc_info with a true value, e.g.

logger.warning(“Houston, we have a %s”, “bit of a problem”, exc_info=1)

exception logilab.common.registry.SelectAmbiguity

Bases: RegistryException

Raised when several objects compete at selection time with an equal score.

logilab.common.registry.obj_registries(cls: Any, registryname: Optional[Any] = None) → Tuple[str]

return a tuple of registry names (see __registries__)

logilab.common.registry.objectify_predicate(selector_func: Callable) → Any

Most of the time, a simple score function is enough to build a selector. The objectify_predicate() decorator turn it into a proper selector class:

@objectify_predicate
def one(cls, req, rset=None, **kwargs):
    return 1

class MyView(View):
    __select__ = View.__select__ & one()

class logilab.common.registry.traced_selection(traced='all')

Bases: object

Typical usage is :

>>> from logilab.common.registry import traced_selection
>>> with traced_selection():
...     # some code in which you want to debug selectors
...     # for all objects

This will yield lines like this in the logs:

selector one_line_rset returned 0 for <class 'elephant.Babar'>

You can also give to traced_selection the identifiers of objects on which you want to debug selection (‘oid1’ and ‘oid2’ in the example above).

>>> with traced_selection( ('regid1', 'regid2') ):
...     # some code in which you want to debug selectors
...     # for objects with __regid__ 'regid1' and 'regid2'

A potentially useful point to set up such a tracing function is the logilab.common.registry.Registry.select method body.

logilab.common.registry.wrap_predicates(decorator: Callable) → None

class logilab.common.registry.yes(score: float = 0.5)

Bases: Predicate

Return the score given as parameter, with a default score of 0.5 so any other selector take precedence.

Usually used for objects which can be selected whatever the context, or also sometimes to add arbitrary points to a score.

Take care, yes(0) could be named ‘no’…

logilab.common.shellutils module

shell/term utilities, useful to write some python scripts instead of shell scripts.

class logilab.common.shellutils.DummyProgressBar

Bases: object

finish()

refresh()

text

update()

class logilab.common.shellutils.ProgressBar(nbops: int, size: int = 20, stream: ~_io.StringIO = <_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>, title: str = '')

Bases: object

A simple text progression bar.

finish()

refresh() → None

Refresh the progression bar display.

property text

update(offset: int = 1, exact: bool = False) → None

Move FORWARD to new cursor position (cursor will never go backward).

Offset:

fraction of size

Exact:

• False: offset relative to current cursor position if True

• True: offset as an asbsolute position

class logilab.common.shellutils.RawInput(input_function: Optional[Callable] = None, printer: Optional[Callable] = None, **kwargs: Any)

Bases: object

ask(question: str, options: Tuple[str, ...], default: str) → str

confirm(question: str, default_is_yes: bool = True) → bool

logilab.common.shellutils.chown(path, login=None, group=None)

Same as os.chown function but accepting user login or group name as argument. If login or group is omitted, it’s left unchanged.

Note: you must own the file to chown it (or be root). Otherwise OSError is raised.

logilab.common.shellutils.cp(source, destination)

A shell-like cp, supporting wildcards.

logilab.common.shellutils.find(directory: str, exts: Union[Tuple[str, ...], str], exclude: bool = False, blacklist: Tuple[str, ...] = ('CVS', '.svn', '.hg', '.git', '.tox', 'debian', 'dist', 'build')) → List[str]

Recursively find files ending with the given extensions from the directory.

Parameters:

• directory (str) – directory where the search should start

• exts (basestring or list or tuple) – extensions or lists or extensions to search

• exts – if this argument is True, returning files NOT ending with the given extensions

• blacklist (list or tuple) – optional list of files or directory to ignore, default to the value of logilab.common.STD_BLACKLIST

Return type:

list

Returns:

the list of all matching files

logilab.common.shellutils.generate_password(length=8, vocab='abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789')

dumb password generation function

logilab.common.shellutils.getlogin()

avoid using os.getlogin() because of strange tty / stdin problems (man 3 getlogin) Another solution would be to use $LOGNAME, $USER or $USERNAME

logilab.common.shellutils.globfind(directory: str, pattern: str, blacklist: Tuple[str, str, str, str, str, str, str, str] = ('CVS', '.svn', '.hg', '.git', '.tox', 'debian', 'dist', 'build')) → Iterator[str]

Recursively finds files matching glob pattern under directory.

This is an alternative to logilab.common.shellutils.find.

Parameters:

• directory (str) – directory where the search should start

• pattern (basestring) – the glob pattern (e.g .py, foo.py, etc.)

• blacklist (list or tuple) – optional list of files or directory to ignore, default to the value of logilab.common.STD_BLACKLIST

Return type:

iterator

Returns:

iterator over the list of all matching files

logilab.common.shellutils.mv(source, destination, _action=<function move>)

A shell-like mv, supporting wildcards.

class logilab.common.shellutils.progress(nbops=<object object>, size=<object object>, stream=<object object>, title=<object object>, enabled=True)

Bases: object

class logilab.common.shellutils.pushd(directory)

Bases: object

logilab.common.shellutils.rm(*files)

A shell-like rm, supporting wildcards.

class logilab.common.shellutils.tempdir

Bases: object

logilab.common.shellutils.unzip(archive, destdir)

logilab.common.sphinx_ext module

logilab.common.sphinxutils module

Sphinx utils

ModuleGenerator: Generate a file that lists all the modules of a list of packages in order to pull all the docstring. This should not be used in a makefile to systematically generate sphinx documentation!

Typical usage:

>>> from logilab.common.sphinxutils import ModuleGenerator
>>> mgen = ModuleGenerator('logilab common', '/home/adim/src/logilab/common')
>>> mgen.generate('api_logilab_common.rst', exclude_dirs=('test',))

class logilab.common.sphinxutils.ModuleGenerator(project_title, code_dir)

Bases: object

class_def = '\n\n.. autoclass:: %s\n   :members: %s\n\n'

file_header = '.. -*- coding: utf-8 -*-\n\n%s\n'

find_modules(exclude_dirs)

gen_modules(exclude_dirs)

generate all modules

generate(dest_file, exclude_dirs=('CVS', '.svn', '.hg', '.git', '.tox', 'debian', 'dist', 'build'))

make the module file

module_def = '\n:mod:`%s`\n=======%s\n\n.. automodule:: %s\n   :members: %s\n'

logilab.common.sphinxutils.class_members(klass)

logilab.common.sphinxutils.module_members(module)

logilab.common.table module

Table management module.

class logilab.common.table.DocbookRenderer(**properties: Any)

Bases: TableCellRenderer

Defines how to render a cell for a docboook table

define_col_header(col_index: int, table_style: TableStyle) → str

Computes the colspec element according to the style

class logilab.common.table.DocbookTableWriter(stream: StringIO, table: Table, style: Optional[Any], **properties: Any)

Bases: TableWriter

Defines an implementation of TableWriter to write a table in Docbook

write_table(title: str = '') → None

Writes the table

class logilab.common.table.Table(default_value: int = 0, col_names: Optional[List[str]] = None, row_names: Optional[Any] = None)

Bases: object

Table defines a data table with column and row names. inv:

len(self.data) <= len(self.row_names)
forall(self.data, lambda x: len(x) <= len(self.col_names))

append_column(col_data: range, col_name: str) → None

Appends the ‘col_index’ column pre:

type(col_data) == types.ListType
len(col_data) == len(self.row_names)

append_row(row_data: Union[List[Union[float, str]], List[int]], row_name: Optional[str] = None) → int

Appends a row to the table pre:

type(row_data) == types.ListType
len(row_data) == len(self.col_names)

apply_stylesheet(stylesheet: TableStyleSheet) → None

Applies the stylesheet to this table

as_text()

create_column(col_name: str) → None

Creates a colname to the col_names list

create_columns(col_names: List[str]) → None

Appends col_names to the list of existing columns

create_row(row_name: Optional[str] = None) → None

Creates a rowname to the row_names list

create_rows(row_names: List[str]) → None

Appends row_names to the list of existing rows

delete_column(index: int) → List[int]

Deletes the ‘index’ column in the table, and returns it. Raises an IndexError if index is out of range

delete_column_by_id(col_id: str) → None

Deletes the ‘col_id’ col in the table. Raises a KeyError if col_id was not found.

delete_row(index: int) → List[str]

Deletes the ‘index’ row in the table, and returns it. Raises an IndexError if index is out of range

delete_row_by_id(row_id: str) → None

Deletes the ‘row_id’ row in the table. Raises a KeyError if row_id was not found.

get_cell_by_ids(row_id, col_id)

Returns the element at [row_id][col_id]

get_column(col_index, distinct=False)

get a column by index

get_column_by_id(col_id, distinct=False)

Returns the ‘col_id’ col

get_columns() → List[List[int]]

Returns all the columns in the table

get_row_by_id(row_id)

Returns the ‘row_id’ row

get_shape() → Tuple[int, int]

Returns a tuple which represents the table’s shape

groupby(colname: str, *others: str) → Union[Dict[str, Dict[str, Table]], Dict[str, Table]]

builds indexes of data :returns: nested dictionaries pointing to actual rows

insert_column(index: int, col_data: range, col_name: str) → None

Appends col_data before ‘index’ in the table. To make ‘insert’ behave like ‘list.insert’, inserting in an out of range index will insert col_data to the end of the list pre:

type(col_data) == types.ListType
len(col_data) == len(self.row_names)

insert_row(index: int, row_data: List[str], row_name: Optional[str] = None) → None

Appends row_data before ‘index’ in the table. To make ‘insert’ behave like ‘list.insert’, inserting in an out of range index will insert row_data to the end of the list pre:

type(row_data) == types.ListType
len(row_data) == len(self.col_names)

pprint() → str

returns a string representing the table in a pretty printed ‘text’ format.

remove(colname, value)

select(colname: str, value: str) → Table

set_cell(row_index: int, col_index: int, data: int) → None

sets value of cell ‘row_indew’, ‘col_index’ to data

set_cell_by_ids(row_id: str, col_id: str, data: Union[int, str]) → None

sets value of cell mapped by row_id and col_id to data Raises a KeyError if row_id or col_id are not found in the table

set_column(col_index: int, col_data: Union[List[int], range]) → None

sets the ‘col_index’ column pre:

type(col_data) == types.ListType
len(col_data) == len(self.row_names)

set_column_by_id(col_id: str, col_data: Union[List[int], range]) → None

sets the ‘col_id’ column pre:

type(col_data) == types.ListType
len(col_data) == len(self.col_names)

Raises a KeyError if col_id is not found

set_row(row_index: int, row_data: Union[List[float], List[int], List[str]]) → None

sets the ‘row_index’ row pre:

type(row_data) == types.ListType
len(row_data) == len(self.col_names)

set_row_by_id(row_id: str, row_data: List[str]) → None

sets the ‘row_id’ column pre:

type(row_data) == types.ListType
len(row_data) == len(self.row_names)

Raises a KeyError if row_id is not found

property shape: Tuple[int, int]

Returns a tuple which represents the table’s shape

sort_by_column_id(col_id: str, method: str = 'asc') → None

Sorts the table (in-place) according to data stored in col_id

sort_by_column_index(col_index: int, method: str = 'asc') → None

Sorts the table ‘in-place’ according to data stored in col_index

method should be in (‘asc’, ‘desc’)

transpose() → Table

Keeps the self object intact, and returns the transposed (rotated) table.

class logilab.common.table.TableCellRenderer(**properties: Any)

Bases: object

Defines a simple text renderer

render_cell(cell_coord: Tuple[int, int], table: Table, table_style: TableStyle) → Union[str, int]

Renders the cell at ‘cell_coord’ in the table, using table_style

render_col_cell(col_name: str, table: Table, table_style: TableStyle) → Union[str, int]

Renders the cell for ‘col_id’ row

render_row_cell(row_name: str, table: Table, table_style: TableStyle) → Union[str, int]

Renders the cell for ‘row_id’ row

class logilab.common.table.TableStyle(table: Table)

Bases: object

Defines a table’s style

get_alignment(col_id: str) → str

Returns the alignment of the specified col_id

get_alignment_by_index(col_index: int) → str

Allors to get the alignment according to the column index rather than using the column’s id. BE CAREFUL : the ‘0’ column is the ‘__row_column__’ one !

get_size(col_id: str) → str

Returns the size of the specified col_id

get_size_by_index(col_index: int) → str

Allows to get the size according to the column index rather than using the column’s id. BE CAREFUL : the ‘0’ column is the ‘__row_column__’ one !

get_unit(col_id: str) → str

Returns the unit of the specified col_id

get_unit_by_index(col_index: int) → str

Allors to get the unit according to the column index rather than using the column’s id. BE CAREFUL : the ‘0’ column is the ‘__row_column__’ one !

set_alignment(value: str, col_id: str) → None

sets the alignment of the specified col_id to value

set_alignment_by_index(value: str, col_index: int) → None

Allows to set the alignment according to the column index rather than using the column’s id. BE CAREFUL : the ‘0’ column is the ‘__row_column__’ one !

set_size(value: str, col_id: str) → None

sets the size of the specified col_id to value

set_size_by_index(value: str, col_index: int) → None

Allows to set the size according to the column index rather than using the column’s id. BE CAREFUL : the ‘0’ column is the ‘__row_column__’ one !

set_unit(value: str, col_id: str) → None

sets the unit of the specified col_id to value

set_unit_by_index(value: str, col_index: int) → None

Allows to set the unit according to the column index rather than using the column’s id. BE CAREFUL : the ‘0’ column is the ‘__row_column__’ one ! (Note that in the ‘unit’ case, you shouldn’t have to set a unit for the 1st column (the __row__column__ one))

class logilab.common.table.TableStyleSheet(rules: Optional[List[str]] = None)

Bases: object

A simple Table stylesheet Rules are expressions where cells are defined by the row_index and col_index separated by an underscore (‘_’). For example, suppose you want to say that the (2,5) cell must be the sum of its two preceding cells in the row, you would create the following rule

2_5 = 2_3 + 2_4

You can also use all the math.* operations you want. For example:

2_5 = sqrt(2_3**2 + 2_4**2)

add_colavg_rule(dest_cell: Tuple[int, int], col_index: int, start_row: int, end_row: int) → None

Creates and adds a rule to make the col average (from start_row to end_row) dest_cell is a tuple of two elements (x,y) of the destination cell No check is done for indexes ranges. pre:

start_row >= 0
end_row > start_row

add_colsum_rule(dest_cell: Tuple[int, int], col_index: int, start_row: int, end_row: int) → None

Creates and adds a rule to sum over the col at col_index from start_row to end_row. dest_cell is a tuple of two elements (x,y) of the destination cell No check is done for indexes ranges. pre:

start_row >= 0
end_row > start_row

add_rowavg_rule(dest_cell: Tuple[int, int], row_index: int, start_col: int, end_col: int) → None

Creates and adds a rule to make the row average (from start_col to end_col) dest_cell is a tuple of two elements (x,y) of the destination cell No check is done for indexes ranges. pre:

start_col >= 0
end_col > start_col

add_rowsum_rule(dest_cell: Tuple[int, int], row_index: int, start_col: int, end_col: int) → None

Creates and adds a rule to sum over the row at row_index from start_col to end_col. dest_cell is a tuple of two elements (x,y) of the destination cell No check is done for indexes ranges. pre:

start_col >= 0
end_col > start_col

add_rule(rule: str) → None

Adds a rule to the stylesheet rules

class logilab.common.table.TableWriter(stream: StringIO, table: Table, style: Optional[Any], **properties: Any)

Bases: object

A class to write tables

set_renderer(renderer: DocbookRenderer) → None

sets the way to render cell

set_style(style)

sets the table’s associated style

update_properties(**properties)

Updates writer’s properties (for cell rendering)

write_table(title: str = '') → None

Writes the table

logilab.common.tasksqueue module

Prioritized tasks queue

class logilab.common.tasksqueue.PrioritizedTasksQueue(maxsize=0)

Bases: Queue

remove(tid: str) → None

remove a specific task from the queue

class logilab.common.tasksqueue.Task(tid: str, priority: int = 0)

Bases: object

merge(other: Task) → None

logilab.common.testlib module

Run tests.

This will find all modules whose name match a given prefix in the test directory, and run them. Various command line options provide additional facilities.

Command line options:

-v

verbose – run tests in verbose mode with output to stdout

-q

quiet – don’t print anything except if a test fails

-t

testdir – directory where the tests will be found

-x

exclude – add a test to exclude

-p

profile – profiled execution

-d

dbc – enable design-by-contract

-m

match – only run test matching the tag pattern which follow

If no non-option arguments are present, prefixes used are ‘test’, ‘regrtest’, ‘smoketest’ and ‘unittest’.

logilab.common.testlib.find_tests(testdir, prefixes=('test', 'regrtest', 'smoketest', 'unittest', 'func', 'validation'), suffix='.py', excludes=(), remove_suffix=True)

Return a list of all applicable test modules.

logilab.common.testlib.nocoverage(func: Callable) → Callable

Function decorator that pauses tracing functions

logilab.common.testlib.pause_trace(trace: Optional[Callable] = None) → Iterator

A context manager that temporary replaces the trace function

logilab.common.testlib.unittest_main

alias of TestProgram

logilab.common.textutils module

Some text manipulation utility functions.

group text formatting:

normalize_text, normalize_paragraph, pretty_match,unquote, colorize_ansi

group text manipulation:

searchall, splitstrip

sort:

text formatting, text manipulation

type ANSI_STYLES:

dict(str)

var ANSI_STYLES:

dictionary mapping style identifier to ANSI terminal code

type ANSI_COLORS:

dict(str)

var ANSI_COLORS:

dictionary mapping color identifier to ANSI terminal code

type ANSI_PREFIX:

str

var ANSI_PREFIX:

ANSI terminal code notifying the start of an ANSI escape sequence

type ANSI_END:

str

var ANSI_END:

ANSI terminal code notifying the end of an ANSI escape sequence

type ANSI_RESET:

str

var ANSI_RESET:

ANSI terminal code resetting format defined by a previous ANSI escape sequence

logilab.common.textutils.apply_units(string: str, units: ~typing.Dict[str, int], inter: ~typing.Union[~typing.Callable, None, type] = None, final: type = <class 'float'>, blank_reg: ~re.Pattern = re.compile('(\\s|,)+'), value_reg: ~re.Pattern = re.compile('(?P<value>-?(([0-9]+\\.[0-9]*)|((0x?)?[0-9]+)))(?P<unit>[a-zA-Z]+)?')) → Union[float, int]

Parse the string applying the units defined in units (e.g.: “1.5m”,{‘m’,60} -> 80).

Parameters:

• string (str or unicode) – the string to parse

• units (dict (or any object with __getitem__ using basestring key)) – a dict mapping a unit string repr to its value

• inter (type) – used to parse every intermediate value (need __sum__)

• blank_reg (regexp) – should match every blank char to ignore.

• value_reg (regexp with "value" and optional "unit" group) – match a value and it’s unit into the

logilab.common.textutils.colorize_ansi(msg: str, color: Optional[str] = None, style: Optional[str] = None) → str

colorize message by wrapping it with ansi escape codes

Parameters:

• msg (str or unicode) – the message string to colorize

• color (str or None) – the color identifier (see ANSI_COLORS for available values)

• style (str or None) – style string (see ANSI_COLORS for available values). To get several style effects at the same time, use a coma as separator.

Raises:

KeyError – if an unexistent color or style identifier is given

Return type:

str or unicode

Returns:

the ansi escaped string

logilab.common.textutils.diff_colorize_ansi(lines, out=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>, style={'add': 'green', 'remove': 'red', 'separator': 'cyan'})

logilab.common.textutils.normalize_paragraph(text: str, line_len: int = 80, indent: str = '') → str

normalize a text to display it with a maximum line size and optionally arbitrary indentation. Line jumps are normalized. The indentation string may be used top insert a comment mark for instance.

Parameters:

• text (str or unicode) – the input text to normalize

• line_len (int) – expected maximum line’s length, default to 80

• indent (str or unicode) – optional string to use as indentation

Return type:

str or unicode

Returns:

the input text normalized to fit on lines with a maximized size inferior to line_len, and optionally prefixed by an indentation string

logilab.common.textutils.normalize_rest_paragraph(text: str, line_len: int = 80, indent: str = '') → str

normalize a ReST text to display it with a maximum line size and optionally arbitrary indentation. Line jumps are normalized. The indentation string may be used top insert a comment mark for instance.

Parameters:

• text (str or unicode) – the input text to normalize

• line_len (int) – expected maximum line’s length, default to 80

• indent (str or unicode) – optional string to use as indentation

Return type:

str or unicode

Returns:

the input text normalized to fit on lines with a maximized size inferior to line_len, and optionally prefixed by an indentation string

logilab.common.textutils.normalize_text(text: str, line_len: int = 80, indent: str = '', rest: bool = False) → str

normalize a text to display it with a maximum line size and optionally arbitrary indentation. Line jumps are normalized but blank lines are kept. The indentation string may be used to insert a comment (#) or a quoting (>) mark for instance.

Parameters:

• text (str or unicode) – the input text to normalize

• line_len (int) – expected maximum line’s length, default to 80

• indent (str or unicode) – optional string to use as indentation

Return type:

str or unicode

Returns:

the input text normalized to fit on lines with a maximized size inferior to line_len, and optionally prefixed by an indentation string

logilab.common.textutils.pretty_match(match: Match, string: str, underline_char: str = '^') → str

return a string with the match location underlined:

>>> import re
>>> print(pretty_match(re.search('mange', 'il mange du bacon'), 'il mange du bacon'))
il mange du bacon
   ^^^^^
>>>

Parameters:

• match (_sre.SRE_match) – object returned by re.match, re.search or re.finditer

• string (str or unicode) – the string on which the regular expression has been applied to obtain the match object

• underline_char (str or unicode) – character to use to underline the matched section, default to the carret ‘^’

Return type:

str or unicode

Returns:

the original string with an inserted line to underline the match location

logilab.common.textutils.split_url_or_path(url_or_path)

return the latest component of a string containing either an url of the form <scheme>://<path> or a local file system path

logilab.common.textutils.splitstrip(string: str, sep: str = ',') → List[str]

return a list of stripped string by splitting the string given as argument on sep (‘,’ by default). Empty string are discarded.

>>> splitstrip('a, b, c   ,  4,,')
['a', 'b', 'c', '4']
>>> splitstrip('a')
['a']
>>>

Parameters:

• string (str or unicode) – a csv line

• sep (str or unicode) – field separator, default to the comma (‘,’)

Return type:

str or unicode

Returns:

the unquoted string (or the input string if it wasn’t quoted)

logilab.common.textutils.splittext(text: str, line_len: int) → Tuple[str, str]

split the given text on space according to the given max line size

return a 2-uple: * a line <= line_len if possible * the rest of the text which has to be reported on another line

logilab.common.textutils.text_to_dict(text)

parse multilines text containing simple ‘key=value’ lines and return a dict of {‘key’: ‘value’}. When the same key is encountered multiple time, value is turned into a list containing all values.

>>> d = text_to_dict('''multiple=1
... multiple= 2
... single =3
... ''')
>>> d['single']
'3'
>>> d['multiple']
['1', '2']

logilab.common.textutils.unormalize(ustring: str, substitute: Optional[str] = None) → str

replace diacritical characters with their corresponding ascii characters

Convert the unicode string to its long normalized form (unicode character will be transform into several characters) and keep the first one only. The normal form KD (NFKD) will apply the compatibility decomposition, i.e. replace all compatibility characters with their equivalents.

Parameters:

substitute (str) – replacement character to use if decomposition fails

See:

Another project about ASCII transliterations of Unicode text http://pypi.python.org/pypi/Unidecode

logilab.common.textutils.unquote(string: str) → str

remove optional quotes (simple or double) from the string

Parameters:

string (str or unicode) – an optionally quoted string

Return type:

str or unicode

Returns:

the unquoted string (or the input string if it wasn’t quoted)

logilab.common.tree module

Base class to represent a tree structure.

class logilab.common.tree.BinaryNode(lhs=None, rhs=None)

Bases: VNode

a binary node (i.e. only two children

get_parts()

return the left hand side and the right hand side of this node

remove(child)

remove the child and replace this node with the other child

class logilab.common.tree.ListNode

Bases: VNode, list

Used to manipulate Nodes as Lists

append(child)

add a node to children

insert(index, child)

add a node to children

pop(index)

add a node to children

remove(child)

add a node to children

class logilab.common.tree.Node(nid: Optional[str] = None)

Bases: object

a basic tree node, characterized by an id

append(child: Any) → None

add a node to children

depth() → int

return depth of this node in the tree

depth_down() → int

return depth of the tree from this node

flatten(_list: Optional[List[Any]] = None) → List[Any]

return a list with all the nodes descendant from this node

get_child_by_id(nid: str, recurse: Optional[bool] = None) → Any

return child of given id

get_child_by_path(path: List[str]) → Any

return child of given path (path is a list of ids)

get_node_by_id(nid: str) → Any

return node in whole hierarchy that has given id

get_sibling(nid: str) → Any

return the sibling node that has given id

insert(index: int, child: Any) → None

insert a child node

is_leaf()

leaves() → List[Any]

return a list with all the leaves nodes descendant from this node

lineage() → List[Any]

return list of parents up to root node

next_sibling()

return the next sibling for this node if any

previous_sibling()

return the previous sibling for this node if any

remove(child: Any) → None

remove a child node

replace(old_child: Any, new_child: Any) → None

replace a child node with another

root() → Any

return the root node of the tree

width() → int

return the width of the tree from this node

exception logilab.common.tree.NodeNotFound

Bases: Exception

raised when a node has not been found

class logilab.common.tree.PostfixedDepthFirstIterator(node: Node, filter_func: Optional[Any] = None)

Bases: FilteredIterator

a postfixed depth first iterator, designed to be used with visitors

class logilab.common.tree.PrefixedDepthFirstIterator(node: Node, filter_func: Optional[Any] = None)

Bases: FilteredIterator

a prefixed depth first iterator, designed to be used with visitors

class logilab.common.tree.VNode(nid: Optional[str] = None)

Bases: Node, VisitedMixIn

a visitable node

logilab.common.tree.post_order_list(node: ~typing.Optional[~logilab.common.tree.Node], filter_func: ~typing.Callable = <function no_filter>) → List[Node]

create a list with tree nodes for which the <filter> function returned true in a post order fashion

logilab.common.tree.pre_order_list(node: ~typing.Optional[~logilab.common.tree.Node], filter_func: ~typing.Callable = <function no_filter>) → List[Node]

create a list with tree nodes for which the <filter> function returned true in a pre order fashion

logilab.common.umessage module

Unicode email support (extends email from stdlib)

class logilab.common.umessage.UMessage(message: Message)

Bases: object

Encapsulates an email.Message instance and returns only unicode objects.

date(alternative_source=False, return_str=False)

return a datetime object for the email’s date or None if no date is set or if it can’t be parsed

get(header: str, default: Optional[Any] = None) → Optional[str]

get_all(header: str, default: Tuple[()] = ()) → List[str]

get_boundary()

get_content_maintype()

get_content_type()

get_filename(failobj=None)

get_payload(index: Optional[Any] = None, decode: bool = False) → Union[str, UMessage, List[UMessage]]

headers()

return an unicode string containing all the message’s headers

is_multipart()

multi_addrs(header)

return a list of 2-uple (name, address) for the given address (which is expected to be an header containing address such as from, to, cc…)

walk()

logilab.common.umessage.decode_QP(string: str) → str

logilab.common.umessage.message_from_file(fd)

logilab.common.umessage.message_from_string(string: str) → Union[UMessage, str]

logilab.common.urllib2ext module

logilab.common.vcgutils module

Functions to generate files readable with Georg Sander’s vcg (Visualization of Compiler Graphs).

You can download vcg at http://rw4.cs.uni-sb.de/~sander/html/gshome.html Note that vcg exists as a debian package.

See vcg’s documentation for explanation about the different values that maybe used for the functions parameters.

class logilab.common.vcgutils.VCGPrinter(output_stream)

Bases: object

A vcg graph writer.

close_graph()

close a vcg graph

edge(from_node, to_node, edge_type='', **args)

draw an edge from a node to another.

node(title, **args)

draw a node

open_graph(**args)

open a vcg graph

logilab.common.vcgutils.latin_to_vcg(st)

Convert latin characters using vcg escape sequence.

logilab.common.visitor module

A generic visitor abstract implementation.

class logilab.common.visitor.FilteredIterator(node: Node, list_func: Callable, filter_func: Optional[Any] = None)

Bases: object

next() → Optional[Node]

class logilab.common.visitor.VisitedMixIn

Bases: object

Visited interface allow node visitors to use the node

accept(visitor: Union[HTMLWriter, TextWriter], *args: Any, **kwargs: Any) → Optional[Any]

get_visit_name() → str

return the visit name for the mixed class. When calling ‘accept’, the method <’visit_’ + name returned by this method> will be called on the visitor

leave(visitor, *args, **kwargs)

class logilab.common.visitor.Visitor(iterator_class, filter_func=None)

Bases: object

close_visit(result)

method called at the end of the visit

open_visit(*args, **kargs)

method called at the beginning of the visit

visit(node, *args, **kargs)

launch the visit on a given node

call ‘open_visit’ before the beginning of the visit, with extra args given when all nodes have been visited, call the ‘close_visit’ method

logilab.common.visitor.no_filter(_: Node) → int

logilab.common.xmlutils module

XML utilities.

This module contains useful functions for parsing and using XML data. For the moment, there is only one function that can parse the data inside a processing instruction and return a Python dictionary.

logilab.common.xmlutils.parse_pi_data(pi_data: str) → Dict[str, Optional[str]]

Utility function that parses the data contained in an XML processing instruction and returns a dictionary of keywords and their associated values (most of the time, the processing instructions contain data like keyword="value", if a keyword is not associated to a value, for example keyword, it will be associated to None).

Parameters:

pi_data (unicode) – data contained in an XML processing instruction.

Returns:

Dictionary of the keywords (Unicode strings) associated to their values (Unicode strings) as they were defined in the data.

Return type:

dict

Module contents

Logilab common library (aka Logilab’s extension to the standard library).

type STD_BLACKLIST:

tuple

var STD_BLACKLIST:

directories ignored by default by the functions in this package which have to recurse into directories

type IGNORED_EXTENSIONS:

tuple

var IGNORED_EXTENSIONS:

file extensions that may usually be ignored

class logilab.common.attrdict

Bases: dict

A dictionary for which keys are also accessible as attributes.

class logilab.common.dictattr(proxy)

Bases: dict

logilab.common.flatten(iterable, tr_func=None, results=None)

Flatten a list of list with any level.

If tr_func is not None, it should be a one argument function that’ll be called on each final element.

Return type:

list

>>> flatten([1, [2, 3]])
[1, 2, 3]

logilab.common.make_domains(lists)

Given a list of lists, return a list of domain for each list to produce all combinations of possibles values.

Return type:

list

Example:

>>> make_domains(['a', 'b'], ['c','d', 'e'])
[['a', 'b', 'a', 'b', 'a', 'b'], ['c', 'c', 'd', 'd', 'e', 'e']]

class logilab.common.nullobject

Bases: object

class logilab.common.tempattr(obj, attr, value)

Bases: object