Python for “quick jobs”

Introduction

Every so often I need to hack together a quick “shell script”-like tool. My first impulse is to do it in Perl since this is what Perl is good for.

Since my Perl skills are quite rusty I have to look up a lot of stuff and what was supposed to be a “quick job” ends up taking a lot of time. Usually much more time than is needed to write the same thing in Python, my favourite programming language.

The issue at hand

In order to lower the threshold for programming “shell script”-like stuff in Python I hence started putting together a module with functions I need for these “quick jobs”.

Something I end up needing more often than not is e.g. a command like this:

find doc -type f -name os_\\* -exec grep settings {} \\; -print

It will

  • search the directory tree starting with the ‘doc’ directory and find all files that contain the string ‘settings’
  • print the found lines along with the file names

Here’s an example (I was experimenting with the help files of the vim (vi improved) editor here):

bbox33:vim62 $ pwd
/usr/share/vim/vim62
bbox33:vim62 $ ls doc/os_*
doc/os_390.txt    doc/os_mac.txt    doc/os_qnx.txt    doc/os_win32.txt
doc/os_amiga.txt  doc/os_mint.txt   doc/os_risc.txt
doc/os_beos.txt   doc/os_msdos.txt  doc/os_unix.txt
doc/os_dos.txt    doc/os_os2.txt    doc/os_vms.txt

And now the output of the command above:

bbox33:vim62 $ find ./doc -type f -name os_\\* -exec grep settings {} \\; -print
simply reverse the notion of foreground and background color settings. To do
settings do not seem to work properly. This has been the case since DR7 at
you use the default Mouse preference settings these names indeed correspond to
./doc/os_beos.txt
without any settings.
./doc/os_os2.txt
Following the name, you can include optional settings to control the size and
./doc/os_qnx.txt
(SYS$LOGIN) to overwrite default settings.
number, try these settings. >
settings from |diff-diffexpr| and change the call to the external diff
./doc/os_vms.txt

The output above shows that four files were found to contain the string of interest, you get the idea..

The solution

scriptutil.py is a module that provides similar functionality for Python programmers and more (syntax highlighted source code here, plain code here).

The following code uses the scriptutil.py module to perform the same example search as above.

 1 import scriptutil, re
 2 
 3 flist = scriptutil.findFiles(
 4     './doc',
 5     namefs=(lambda s:s.find('os_') != -1,),
 6     contentfs=(re.compile('^.*settings.*$', re.M).findall,))
 7 
 8 scriptutil.printResults(flist)

The code calls the function scriptutil.findFiles and passes the following parameters to it:

  • './doc': the path to the top level directory of the tree to be searched
  • namefs: a sequence of functions to be used for file name filtering
  • contentfs: a sequence of functions to be applied to the content of the files that passed the file name tests

Subsequently the scriptutil.printResults helper function is invoked to pretty-print the results which are as follows:

bbox33:vim62 $ python /tmp/su.py
simply reverse the notion of foreground and background color settings. To do
settings do not seem to work properly. This has been the case since DR7 at
you use the default Mouse preference settings these names indeed correspond to
./doc/os_beos.txt
without any settings.
./doc/os_os2.txt
Following the name, you can include optional settings to control the size and
./doc/os_qnx.txt
(SYS$LOGIN) to overwrite default settings.
number, try these settings. >
settings from |diff-diffexpr| and change the call to the external diff
./doc/os_vms.txt

In conclusion

There is no reason why Python should not be used to program “shell script”-like tools. In forthcoming weblog entries I will be presenting some more sophisticated examples of what the scriptutil.findFiles function is capable of. Stay tuned!

About these ads

3 thoughts on “Python for “quick jobs”

  1. “There is no reason why Python should not be used to program “shell script”-like tools.”

    This is exactly what I began using Python for. As a Windows admin at the time, Perl was available, but Python seemed “friendlier”.

  2. The shell command ‘find’ is a perfect example of something that can (and probably should) be abstracted into a python procedure. It is so complicated that whenever I have to do some thing that is nontrivial with it I have to look at the man page. It is (IMHO) the Swiss Army Chainsaw of shell commands. Good work.

    Since learning Perl and then abandoning it for Python I almost never use a shell script if I can’t do it as a one liner.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s