First experiments with python’s functools.partial

I started playing with one of the python2.5 features, namely functools.partial() that allows one to effectively “construct variants of existing functions that have some of the parameters filled in” (quote from Functional Programming HOWTO).

In the code below I am trying to preset the second paramater of the operator.lt() built-in function.

 1  #!/usr/bin/env python2.5
 2  """
 3  python 2.5 partial functions experiments (part 1)
 4  """
 5  import operator
 6  import functools
 7
 8  if __name__ == '__main__':
 9      #>>> help(operator.lt)
10      #Help on built-in function lt in module operator:
11      #
12      #lt(...)
13      #    lt(a, b) -- Same as a<b.
14
15      #>>> operator.lt
16      #<built-in function lt>
17
18      less_than_two = functools.partial(operator.lt, b=2)
19      print "1 < 2: ", less_than_two(1)
20      print "2 < 2: ", less_than_two(2)

However, when I run the code I get the following error:

mhr@playground2:~/src/published$ python2.5 partialf3.py
1 < 2:
Traceback (most recent call last):
  File "partialf3.py", line 19, in <module>
    print "1 < 2: ", less_than_two(1)
TypeError: lt() takes no keyword arguments

For some reason python’s built-in functions do not accept keyword arguments. Hence, my attempt to preset the second parameter (but leave the first alone (line 18 above)) does not work.

Now, I understand I can achieve the same effect by using the opposite (greater than) operator (operator.gt) in conjunction with preset positional parameters (see line 18 below).

 1  #!/usr/bin/env python2.5
 2  """
 3  python 2.5 partial functions experiments (part 2)
 4  """
 5  import operator
 6  import functools
 7
 8  if __name__ == '__main__':
 9      #>>> help(operator.gt)
10      #Help on built-in function gt in module operator:
11      #
12      #gt(...)
13      #    gt(a, b) -- Same as a>b.
14
15      #>>> operator.gt
16      #<built-in function gt>
17
18      less_than_two = functools.partial(operator.gt, 2)
19      print "1 < 2: ", less_than_two(1)
20      print "2 < 2: ", less_than_two(2)

This yields the expected behaviour:

mhr@playground2:~/src/published$ python2.5 partialf_2_5.py
1 < 2:  True
2 < 2:  False

The refusal on the part of python’s built-in functions to accept keyword arguments seems odd since it introduces somewhat of an inconsistency (built-in functions differing from “normal” functions in that regard).

I am wondering why this seeming inconsistency was introduced to python, a language that prides itself on a clean design.

About these ads

12 thoughts on “First experiments with python’s functools.partial

  1. Bugger – your comments system doesn’t escape <.

    But achieving this yourself should be trivial right ?

    def lessthansomething(something):
    … return lambda x : x < something

    Followed by :

    In fact you can do it with one liners using nested lambdas, which is quite fun :

    >>> x = lambda a : lambda x : x < a
    >>> x
    <function <lambda> at 0x00B3DF30>
    >>> x(3)
    <function <lambda> at 0x00B3DFB0>
    >>> x(3)(3)
    False
    >>>

  2. Makes sense that it doesn’t take keyword args there; it’s not an optional arg after all (thus passing it as a keyword would be whacky).

    Think what would be a bit more useful is if partial were split into pre/post for arg stacking for the target offhand; use post_curry(func, 2) to get operator.lt(a, 2)

  3. ferringb, not quite sure about the optional nature of keyword arguments. I always thought that keyword arguments are an alternative to positional parameters but a “first order” parameter passing mechanism in its own right.
    I like your idea about currying from the right hand side very much! Maybe it could be suggested for the next release of the language (version 2.6 if there is one)

  4. @muharem:

    Regarding keyword passing… nope. If you do
    def f(foo, bar): pass

    foo and bar *must* be positionally passed;

    def f2(foo, bar=None):pass
    foo must be positional; bar can be positional, or passed keyword arg style.

    I’d suggest taking a look at PyArg_ParseTupleAndKeywords in http://docs.python.org/api/arg-parsing.html when you get a chance; description is dry, so would be worthwhile tracking down other usages of it.

    Rambling lecture mode… ;)

    For (most) func invocation, python builds a tuple and dict (if any keywords), and passes that into the actual function, and the target uses one of the unpack functions from above to break it back down into seperate variables; optional keywords technically are accessible via their position (it’s a non-changed position unless folks change the prototype).

    So… which func is used, it’ll basically consume from the passed in tuple until it runs off the end; at that point, it *has* to have filled all required positional args; from there, starts doing gets into the passed in dict (if any; NULL can be passed in to indicate no keywords) to try and fill in any missing args.

    Now… it probably *could* do what you’re requesting; just mangle the unpack handlers, but that would require all functors to take kwds; not all actually do.

    Note the tuple/dict comment above; if you could pass all args as keywords, that would mean that the dict would have to be created for a heck of a lot more func invocations, and that code would *always* have to look into the dict.

    Thing is, building/peeking into a tuple is a helluva lot faster then doing the same for a dict; Keywords are nice, not denying it, but forcing dict as an always supported arg passing mechanism means func invocation/setup gets quite a bit slower (pythons func overhead is already a bit much).

    Note that I’m not saying keywords should die; just that they serve a singular purpose, optional args; so… internally, supporting passing for keywords always gets a bit ugly, but (probably more important to others) allowing such passing that way means that keywords no longer are just optional args, they can handle required args.

    Which is a bit confusing also.

  5. ferringb,
    I just tried the following:
    mhr@playground2:~$ python2.5
    Python 2.5 (r25:51908, Dec 1 2006, 08:05:06)
    [GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2
    Type “help”, “copyright”, “credits” or “license” for more information.
    >>> def f(a,b):
    … print a
    … print b

    >>> f(a=1,b=2)
    1
    2
    >>> f(b=2,a=’s’)
    s
    2
    i.e. I am passing non-optional parameters through the keyword parameter passing mechanism.

    I do appreciate your comments re. function invocation efficiency. Since I am not using python for anything performance critical, however, a certain degree of consistency when it comes to dealing with functions would weigh heavier on my personal scale.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s