pbzip2: The luxury of multiprocessing

This is one of those times when a screenshot will tell you a lot more than I can, with words:

pbzip2, the parallificated bzip2, chopping a good 20 seconds of the compression time on a 256Mb clump of random text.

In that situation, nothing else is running and this laptop has an SSD in it, so it’s fairly quick to start with. But pbzip2 still manages to slash the time it takes to smush it down a bit.

The fun part of pbzip2 is watching htop while it’s running. In the case of vanilla bzip2, the system load meter on one processor spikes to 100 percent, while the other sits near idle.

But pbzip2 kicks both of them up to max on this Core2 Duo, and the fan suddenly starts to whine a little louder. 😉

That does, of course, suggest that on a single core machine, you might not see any improvement at all. Logic says without an advanced CPU, there’s little space to share.

Give it a try and see what happens; you never know, there might be a tiny bump.

In closing, I’m a little surprised pbzip2 isn’t more famous. Perhaps there’s something sketchy in its history that I don’t know about.

For now, I’m going to tempt fate and try

ls random-{1,2}.txt | parallel pbzip2 -f -k -9

and see what happens. Yes, combining parallel and pbzip2 might just trigger a black hole in the center of my computer. But just let me press Enter now and see wha