Just a quick update: Andy Lester read the original Combining ack-grep and xargs post and was so kind to point out a much better way to ignore unwanted files while searching i.e.
$ ack-grep --python --ignore-dir=tests/ -C 3 -w 'Message\('
as opposed to
$ find . -name \*.py | grep -v tests/ | xargs ack-grep -C 3 -w 'Message\('
I use ack-grep a lot and really, really like it. Kudos to the author and to the maintainer who takes care of the ubuntu package 
Sometimes though I was missing grep‘s --exclude feature that allows me to ignore certain paths while searching.
There are occasions where I e.g. want to see calls to a certain function in the code base but I am not interested in tests. Today I found an (embarrassingly) easy way to get that behaviour using xargs:
$ find . -name \*.py | grep -v tests/ | xargs ack-grep -C 3 -w 'Message\('
The snippet above first accumulates the paths of interests, then filters them and finally lets ack-grep loose on them.
Ta-da! There you go
I received my Motorola Milestone android smartphone last Friday and have now first hand experience of the utility of such a device. It is a
all neatly packaged in a gadget that fits in your pocket. Oh, and yes, you can also use it place or receive phone calls
So, just in case you don’t have a smartphone yet: get one! It’s a game changer.
Recent example: I took my mum to airport this morning (30 minute drive). After seeing her off I checked for new audio-casts (using Google listen) and saw one that interested me (Hadoop with Philip Zeyliger).
It was downloaded in no time and I could listen to it while driving back home which was a great use of my time.
Here’s some additional software I installed over the last couple of days:
Any other cool stuff out there that I missed?
I have been merging packages from debian unstable to ubuntu karmic lately and every so often I need to look at a merge conflict.
Sometimes it’s not so obvious why a conflict occurred at all. Here’s an example conflict from the libsepol package:
1 <<<<<<< libsepol-2.0.32-1ubuntu1 (ubuntu) 3 /* Get the scope info for this boolean to see if this is the declaration, 4 * if so set the state */ 5 scope = hashtab_search(state->cur->policy->p_bools_scope.table, id); 6 if (!scope) 7 return SEPOL_ERR; 8 if (scope->scope == SCOPE_DECL) 9 base_bool->state = booldatum->state; 10 11 ======= 13 /* Get the scope info for this boolean to see if this is the declaration, 14 * if so set the state */ 15 scope = hashtab_search(state->cur->policy->p_bools_scope.table, id); 16 if (!scope) 17 return SEPOL_ERR; 18 if (scope->scope == SCOPE_DECL) 19 base_bool->state = booldatum->state; 20 21 >>>>>>> libsepol-2.0.36-1 (debian)
In order to avoid staring at the screen until my eyes start to bleed I came up with the following bash function:
1 function dsec() { 2 xclip -selection clipboard -o > /tmp/patch-seg-to-diff.txt 3 rm -f /tmp/ubuntu-snippet.txt /tmp/debian-snippet.txt 4 ed /tmp/patch-seg-to-diff.txt > /dev/null 2>&1 <<-EOUBUNTU 5 1 d 6 /^===/,$ d 7 wq /tmp/ubuntu-snippet.txt 8 EOUBUNTU 9 ed /tmp/patch-seg-to-diff.txt > /dev/null 2>&1 <<-EODEBIAN 10 1,/^===/d 11 $ d 12 wq /tmp/debian-snippet.txt 13 EODEBIAN 14 echo 'diff -Nru /tmp/ubuntu-snippet.txt /tmp/debian-snippet.txt | colordiff | less' 15 diff -Nru /tmp/ubuntu-snippet.txt /tmp/debian-snippet.txt | colordiff | less 16 }
I can now use my favourite editor to copy the conflicting code to the clipboard. When invoked on the command line the dsec function reads the clipboard content and chops it apart (using the ed utility) into a debian and an ubuntu snippet respectively. Subsequently it invokes the diff utility on the two snippets showing me how they differ.
1 --- /tmp/ubuntu-snippet.txt 2009-06-26 12:42:04.000000000 +0200 2 +++ /tmp/debian-snippet.txt 2009-06-26 12:42:04.000000000 +0200 3 @@ -1,5 +1,5 @@ 4 /* Get the scope info for this boolean to see if this is the declaration, 5 - * if so set the state */ 6 + * if so set the state */ 7 scope = hashtab_search(state->cur->policy->p_bools_scope.table, id); 8 if (!scope) 9 return SEPOL_ERR;
Hmm .. this looks like a white space issue. Since the dsec function printed the actual diff command to the terminal, I can copy and paste that to the command line and add the -b parameter so that the diff utility ignores changes in the amount of white space and ..
1 $ diff -Nrub /tmp/ubuntu-snippet.txt /tmp/debian-snippet.txt 2 $ echo $? 3 0
.. voila! the diff output this time around is empty.
In order for this to work the utilities used must be installed, so e.g. on a ubuntu system run:
sudo apt-get install xclip colordiff
And .. don't forget to have fun
I was attending the UDS in Mountain View last week and it proved to be one of those fascinating albeit somewhat exhausting events (please see either of these resources for UDS reports and commentary).
Anyway, while being there I managed to get my hands on a paper copy of Real World Haskell. Being busy with the UDS I only started reading it on the plane back to Frankfurt. Despite being very, very tired I enjoyed it thoroughly and have to say it’s one of the best technical books I have ever perused.
All successful (technical) projects have a vibrant community and great documentation. This book makes Haskell so accessible, it may well be the last bit that’s needed for a great break-through for Haskell and functional programming in general!
Whether you are interested in functional programming or merely seeking to broaden your horizon, don’t delay, go out and grab a copy of this book. You won’t be disappointed.
My moaning about Python(2) was apparently “snow of yesterday”. Python3 just came out featuring extended iterable unpacking.
1 Python 3.0rc1+ (py3k, Oct 28 2008, 09:22:29) 2 [GCC 4.3.2] on linux2 3 Type "help", "copyright", "credits" or "license" for more information. 4 >>> def f(a, b, c): 5 ... print('%s, %s, %s' % (a, b, c)) 6 ... 7 >>> f(1,2,3) 8 1, 2, 3 9 >>> f(1,*(2,3)) 10 1, 2, 3 11 >>> f(*(1,2),3) 12 File "<stdin>", line 1 13 SyntaxError: only named arguments may follow *expression 14 >>> f(*(1,2),c=3) 15 1, 2, 3
Hmm, not ideal but it does the job
1 Python 2.5.2 (r252:60911, Oct 5 2008, 19:29:17) 2 [GCC 4.3.2] on linux2 3 Type "help", "copyright", "credits" or "license" for more information. 4 5 >>> def f(*args): 6 ... print ' '.join([str(a) for a in args]) 7 ... 8 >>> f(1,2,3) 9 1 2 3 10 >>> f(1,*(2,3)) 11 1 2 3 12 >>> f(*(1,2),3) 13 File "<stdin>", line 1 14 f(*(1,2),3) 15 ^ 16 SyntaxError: invalid syntax
Sigh.
I have cleaned up the documentation for the scriptutil module which is available on the web now. If you happen to run ubuntu you can also install it as a package straight from my PPA.
Please have a look at this tutorial in case you’re interested in scriptutil usage examples.
Enjoy!
I have to admit that I didn’t fully understand the apfelmus example code. Nevertheless, I made an effort to address both of his criticisms:
mergesort_ function now does not use the haskell list length operator any more. Instead, the length of the list to be sorted is passed down the recursive chain.Please see the optimised mergesort implementation for details.
These improvements reduced the RAM utilisation and improved the run-time performance by another 20% respectively.
Nothing to scoff at, eh?
After my (naive) mergesort implementation from yesterday used around 730 MB of RAM to sort a (26 MB) file containing approx. 400,000 strings I consulted the good folks on the #haskell IRC channel.
Their advice was to use byte strings as opposed to normal strings since the former perform much better.
I tried that and observed that the RAM utilisation and run-time went down by approximately 85% !
The reduced RAM utilisation was attributed to the more efficient byte strings and the improved run-time performance to the reduced garbage collection overhead respectively.
The difference between the source files (mergesort.hs, Scaffolding.hs) is minimal and switching over to byte strings was facilitated by the fact that they expose the same interface as normal strings.
Nice!
