Python decorator mini-study (part 3 of 3)


In part one of this article series I introduced the Params2attribs decorator that copies object initialiser parameters to object attributes.
In part two we took a closer look at the Python decorator mechanics.

This is the final part in which we are going to dissect the Params2attribs decorator.


From the code below you can glean that the decorator is implemented as a class with three methods:

  • initialiser (lines 20 – 22)
  • the special __call__ method that gets invoked when you treat an object like a function (lines 23 – 42)
  • an auxilliary method (copy2attribs) that’s in charge of the parameter copying business

Before diving into the code, please bear in mind that the Params2attribs::__call__() method gets executed whenever Python loads any code decorated with Params2attribs (e.g. lines 19 – 20 in part one). At that point the wrapper function is only defined and returned to Python as a replacement for the decorated function.

In contrast, the wrapper function is executed whenever the decorated function is called (e.g. line 27 in part one).

If any of this is unclear please refer back to part two where the workings of Python decorators were explained in greater detail.

 1  #!/usr/bin/env python
 2  """
 3  Module with a decorator class for initialiser (__init__()) methods. The
 4  decorator wraps initialiser methods in functions that copy the initialiser
 5  parameter values to attributes of the respective object.
 6  """
 8  # Copyright: (c) 2006 Muharem Hrnjadovic
 9  # created: 16/10/2006 06:49:38
11  __version__ = "$Id$"
12  # $HeadURL $
14  import inspect
16  class Params2attribs(object):
17      """Decorator class for initialiser (__init__()) methods, wraps them
18      in functions that copy the initialiser parameter values to attributes
19      of the respective object."""
20      def __init__(self, exclude_params=None):
21          """what parameters should we exclude from copying (if any)?"""
22          self.exclude_params = exclude_params

In order to see what is going on here I will use the debugger to display the values of various variables of interest.

For example, when the Params2attribs::__call__() method runs, the value of argspec (as returned by the inspect.getargspec() utility (line 27)) is as follows:

argspec = ( ['self', 'arg_1', ['arg_2', 'arg_3'], 'arg_4', 'arg_5'],

What we are after here are the names of the parameters (passed to the decorated function) and they are all contained in the first element of the argspec tuple. Please note how the names of embedded parameters are returned in an embedded list which is quite handy as we will see later.

23      def __call__(self, f):
24          """define and return the wrapper for the decorated __init__() method"""
25          # introspect f() for parameter data, 'argspec' becomes part of
26          # the wrapper() closure
27          argspec = inspect.getargspec(f)

The wrapper

At the point of its definition the wrapper function has access to the argspec tuple due to lexical scoping but argspec will be available to it at execution time as well because it is part of its closure.

When a CoolApproach object is instantiated as shown on line 27 in part one, the wrapper function is called and the value of its args parameter is as follows:

args = ( <__main__.CoolApproach object at 0xb7b47f6c>,
         ('E1', 'E2'),

The first argument is a CoolApproach reference (due to invocation on a CoolApproach object). The others were passed on line 27 in part one.

28          def wrapper(*args, **kwargs):
29              """wrapper function for a decorated __init__ method"""
30              # the first parameter is the initialiser's object reference
31              obj = args[0]
32              # combine the initialiser's parameter names with their
33              # respective values (but ommit the initialiser's object reference)
34              argvl = zip(argspec[0][1:], args[1:])
35              # copy the initialiser parameters to attributes of its object
36              self.copy2attribs(obj, argvl)
37              # finally, invoke the wrapped initialiser method
38              f(*args, **kwargs)
39          # assign the wrapped method's doc string to the wrapper
40          wrapper.func_doc = f.func_doc
41          # .. and eventually return the wrapper function
42          return wrapper

The first step is to combine the parameter names and values in one list, namely argvl (see line 34 above) which then has the following value:

argvl = [ ('arg_1', 'aa'),
          (['arg_2', 'arg_3'], ('E1', 'E2')),
          ('arg_4', 22),
          ('arg_5', 3.3300000000000001)]

Subsequently, the argvl list is passed (along with the object reference) to the copy2attribs helper function which will do the copying for us (line 36 above).
Last but not least, the wrapped function itself is invoked on line 38 (above).

The helper function

Eventually, the code below copies the parameter name/value pairs to object attributes. While doing so, it observes the list of parameters that are not to be copied (see line 52).

43      def copy2attribs(self, obj, argvl):
44          """copy the attribute/value pairs in 'argvl' to attributes of object
45             'obj'. Resolve any embedded attribute/value pair lists through
46             recursive calls"""
47          for argn, argv in argvl:
48              # resolve embedded parameters via a recursive call
49              if isinstance(argn, list):
50                  self.copy2attribs(obj, zip(argn, argv))
51              # skip unwanted initialiser parameters
52              elif self.exclude_params and argn in self.exclude_params:
53                  continue
54              else: setattr(obj, argn, argv)

If you were to run the code in the debugger (highly recommended :-)) and placed a breakpoint inside the copy2attribs function you would see that it is invoked twice.

The argvl value seen for the first call is:

argvl = [ ('arg_1', 'aa'),
          (['arg_2', 'arg_3'], ('E1', 'E2')),
          ('arg_4', 22),
          ('arg_5', 3.3300000000000001)]

The argvl value for the second call is:

argvl = [('arg_2', 'E1'), ('arg_3', 'E2')]

How is this to be understood? The first invocation is from the wrapper function (line 36). The second invocation occurs through a recursive call (line 50) when the top-level copy2attribs invocation processes the embedded list (second element of argvl: (['arg_2', 'arg_3'], ('E1', 'E2'))).


This concludes the Python decorator mini-study. I hope you liked it and would appreciate your comments. Also, I will be making the sources available as soon as I have figured out how I can upload something that is not a picture to WordPress.


Please note: the fix for the issues pointed out in the comments as well as the links to the source code can be found here.

About these ads

3 thoughts on “Python decorator mini-study (part 3 of 3)

  1. I think this won’t work when using default arguments, as they’ll get ignored when not provided. You should be able to handle them just by appending the default list to the end of argvl (Taking advantage of the fact that zip will truncate to the size of the shorter sequence). Also, passing arguments by keyword has the same problem. I think you can deal with both of these if you change the “argvl=…” line to:

    argvl = [(name, kwargs.get(name, val)) for (name, val) in
    zip(argspec[0][1:], args[1:] + argspec[3])]

    (Always fetching from kwargs if present should be safe, as python doesn’t allow you to pass the same argument both positionally and by keyword)

  2. Oops – the above won’t work for the nested argument example – the values are given as lists (Which is odd – I’d have expected tuples), so an exception gets thrown (list objects are unhashable) when it trys to look them up in the kwargs dict.
    The easiest fix would probably be to use:

    argvl = [(name, kwargs.get(str(name), val)) for (name, val) in zip(argspec[0][1:], args[1:] + argspec[3])]

    Calling str() will have no effect on the actual names, and will turn nested args into something like ‘[1, 2]‘ which can never be found in the kwargs dict, so these will always use the positional version. It looks like you can only pass nested args by position, so this doesn’t add any restriction.

  3. Hello Brian,

    thank you very much for your comments and enhancement suggestions above. I am in the process of moving house and hance a little bit strapped for time. However, I’ll have a look at them as soon as I get to it.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s