Code refactoring with python’s functools.partial

In order to get a better feeling for what can be done with functools.partial() I am taking some “real world” python code of mine and refactoring it to use curried functions.

In the example below the construct_getopt_data() function (lines 7-17) takes the data structure shown on lines 44-54 (below) and returns a 2-tuple where

  • the first element is a string of option letters (second argument to getopt() (see variable shortflags in the output below)) and
  • the second element is a list of strings with the names of long options (third argument to getopt() (see variable longflags below))

The resulting data is thus as follows:

mhr@playground2:~/src/published$ python2.5
shortflags = 'l:ep:a:x:r:o:i:dc'
longflags =
(   'lines=',

>> test ok

And the code that’s producing the output above looks as follows:

 1  #!/usr/bin/env python
 2  from pprint import PrettyPrinter as PP
 3  from itertools import (imap, repeat)
 4  from functools import partial
 5  from operator import (ge, lt)
 7  def construct_getopt_data(args):
 8      """uses lambdas"""
 9      # single and multi-character flag iterators
10      shortiter = lambda args: argiter(args, lambda s: s <= 2)
11      longiter = lambda args: argiter(args, lambda s: s > 2)
13      # single character flags
14      shortfs = imap(formatf, shortiter(args), repeat(':'))
15      # multi-character flags
16      longfs = imap(formatf, longiter(args), repeat('='))
17      return(''.join(shortfs), tuple(longfs))

Please note:

The analogous function construct_getopt_data2() (lines 19-29) below performs the same taks but uses curried functions as opposed to lambdas.

19  def construct_getopt_data2(args):
20      """uses functools.partial"""
21      # single and multi-character flag iterators
22      shortiter = partial(argiter, op=partial(ge, 2))
23      longiter = partial(argiter, op=partial(lt, 2))
25      # single character flags
26      shortfs = map(partial(formatf, fchar=':'), shortiter(args))
27      # multi-character flags
28      longfs = map(partial(formatf, fchar='='), longiter(args))
29      return(''.join(shortfs), tuple(longfs))

It utilises functools.partial()

  1. on lines 22-23: to customise the argument iterator by
    1. currying the built-in operator functions (greater or equal) and (less than)
    2. using these curried operator functions to preset the op parameter of the argiter() generator
  2. on lines 26 and 28 to preset the fchar parameter of the formatf() function (the ruse with itertools.repeat() is hence not needed any more)
31  def argiter(args, op):
32      """pair short/long flags will their respective types"""
33      for flags, argdata in args.iteritems():
34          for flag in flags:
35              if op(len(flag)): yield (flag, argdata[1])

The argiter() generator above facilitates the iteration over the input data structure (lines 44-54) in (single character flag, type) and (multi-character parameter, type) pairs respectively.

37  def formatf((argn, argt), fchar):
38      """format for getopt(),
39      argn is the flag, argt is its type, fchar is one of ':' or '='"""
40      return argt == bool and argn.lstrip('-') or "%s%s" % (argn.lstrip('-'), fchar)

The formatf() function above returns the single and multi-character command line flags in the format required by getopt().

42  if __name__ == '__main__':
43      # dictionary with command line args along with their types and defaults
44      args = {
45          ('-a', '-x', '--algo')      :   ('algo', str, None),
46          ('-c', '--crypt')           :   ('crypt', bool, None),
47          ('-d', '--decrypt')         :   ('decrypt', bool, None),
48          ('-e', '--echo', '--fyo')   :   ('echo', bool, None),
49          ('-l', '--lines')           :   ('lines', int, '25'),
50          ('-i', )                    :   ('input', str, None),
51          ('-o', )                    :   ('output', str, None),
52          ('-p', '--pager', '--pgr')  :   ('pager', str, '/usr/bin/less'),
53          ('-r', '--recipient')       :   ('recipient', str, None)
54      }
55      pp = PP(indent=4)
56      sfs1, lfs1 = construct_getopt_data(args)
57      sfs2, lfs2 = construct_getopt_data2(args)
58      print "shortflags =", pp.pformat(sfs1)
59      print "longflags =\\n", pp.pformat(lfs1)
61      if (sfs1 == sfs2) and (lfs1 == lfs2): print "\\n>> test ok"
62      else: print "\\n>> test failed"

In conclusion

While functools.partial() is certainly an interesting and cool addition to the python toolchest it would appear that it’s not indispensible for functional style programming.

I would love to see examples or code snippets that are made possible and/or improved greatly by leveraging functools.partial().