Tag Archives: line

the_silver_searcher: Intergalactic searcher smackdown

Uh-oh. I can already sense some friction between the_silver_searcher and ack, which you might remember as a sort of replacement for grep.

2014-10-30-6m47421-ag-ack 2014-10-30-6m47421-ag-ag 2014-10-30-6m47421-ag-grep

Left-to-right, ack, the_silver_searcher and grep. Please pay attention to the results of time.

Because it seems to me that both the_silver_searcher (which I will henceforth abbreviate by its executable ag, because that name is a little cumbersome to type) and ack pound their chests on their home pages, and suggest they are faster than grep. And ag definitely holds itself out as faster than ack. And I could swear ack claimed a speed difference over grep.

And yet in subsequent tests, I get results a lot like this …

grep ack ag
time … “Time” list.txt real 0m0.025s
user 0m0.010s
sys 0m0.003s
real 0m0.387s
user 0m0.280s
sys 0m0.010s
real 0m0.051s
user 0m0.017s
sys 0m0.007s

And grep is regularly faster than ack or ag. Q.E.D. :\

For the record, list.txt is a 75K list of television and movie titles that I scraped off the Internet just for this searching smackdown. No funky characters or encoding issues that I could see, and nothing but simple ASCII characters.

Maybe it’s my imagination and none of these programs attempted to be faster than another. And it’s important to remember that my tests are particularly gummy. I have an unnatural knack for screwing up some of the simplest setups, and an unscientific, unprofessional, unstatistical freestyle search-to-the-death would be no exception.

All that aside, ag strikes me as every bit as useful or friendly as ack, and seems to follow a lot of the same flag and output conventions. In fact, some of them appear identical. ๐Ÿ˜•

I suppose that means your choice of search tools will depend mostly on … your choice. In the mean time I’m going to turn up the heat on all three of these and check their results when I turn them loose on time {grep,ack,ag} main /lib/ … ๐Ÿ˜ฏ That will separate the men from the boys. ๐Ÿ‘ฟ

P.S.: Thanks to riptor for the tip. ๐Ÿ˜‰

nl: Line numbering, in a variety of ways

nl, out of coreutils again, looks like a fairly simple program.

2014-09-26-6m47421-nl-01

Given the proper flags though, nl suddenly becomes a wild and crazy guy.

2014-09-26-6m47421-nl-02

Like a lot of things hiding in coreutils, what looks simple has a lot of very cool options. And it’s clear that nl is capable of handling much more complex arrangements. Watch for options for headers and footers, logical pages and so forth. Most of those things … I don’t even know what they are. ๐Ÿ˜

I can attest to nl‘s usefulness; I’ve used it in the past to add sequence numbers to lists, and to number lines from (of all places) within vim.

And considering the acrobatics it might take to do the same thing without nl, it is definitely a keeper.

grep: The time has come

I saved grep for last because grep deserves the attention. grep is one of those tools that probably deserves its own spot in history, like sed or rsync. It’s just that powerful.

You probably know what grep does — screens through streams of text and spits out matching results. So at its most basic,

grep Text file.txt

should highlight lines in file.txt that match “Text” … simple, isn’t it?

grep makes it simple, that’s for sure. Have three files and you’re not sure where something is?

grep Text file-01.txt file-02.txt file-03.txt

grep finds it, adding the name of the file on the front, whenever “Text” appeared.

So what else can grep handle? How about numbering results, so I don’t have to pick through lines by hand?

grep -n Text file.txt

Yes, numbers! But wait, the numbers are the lines “Text” appears in, not the number in the list. Keep that in mind.

There’s another option that might not seem valuable. Here’s the -o flag, which plays havoc on my test file.

grep -o ie test.txt

Output looks like this:

ie
ie
ie
ie

Well that’s dumb. Or not. Here’s the same thing with the line numbering option:

grep -on ie test.txt

And you get:

36:ie
54:ie
97:ie
100:ie

So sometimes, for example on long lines of text, maybe I don’t care about the actual output, only the line it appears on. Easier to find typos, I should think.

What’s next … how about -A, -B and -C. A will show lines after the matching string, B shows before the string, and C is sort of a combination of A and B. Ergo,

grep -A2 ie test.txt

yields:

coefficients
guildhall
customs
--
energies
slurring
benight
--
ladies
infringements
meets
metier

Note that the second and third lines in each set there don’t necessarily contain the “ie” pair, but I asked that they be included. grep adds the double dashes.

And that last line, “metier”? Where’s its following couple? Nowhere. That’s the end of the file, and it’s only three lines after the word “ladies,” so it appears to be a string of four.

Probably the most important flag is -v for the simple reason that it reverses the results — instead of finding matches, grep reports nonmatching lines. To wit:

grep -vn e test.txt

And the results include …

78:bursty
82:lobby
83:harrows
85:skulk
91:propyl
93:cowling
94:tracy
95:unflagging

And so forth, with no letter “e” in a line.

grep comes into play a lot more than I would think. The Index page for this site is built off a massive XML export, and I need grep to skim through it all and find the links.

Or, here’s a list of movies, skimmed to find anything from the 1940s.

grep 194 list-01.txt

Yielding …

Fantasia (1940)
It's a Wonderful Life (1946)
Notorious (1946)
The First of the Few (1942)
The Great Dictator (1940)

Try piping directory lists through grep; it might be easier to handle than ls at times. I’ve seen other sites that use grep in conjunction with find as a kind of screen that displays directory paths. Clever.

I should mention that grep comes with egrep and fgrep, the former handling regular expressions more deftly than the -E flag for regular grep. The latter is a “faster” grep, and might be more useful for those really, really big searches.

One more flag, and I’m done. You guessed it: --color !

2013-11-17-lv-r1fz6-grep

Nothing is complete without color. ๐Ÿ˜‰

efax: Could possibly be functional

Ordinarily I would have probably left efax to the end of the E section, and listed as something I couldn’t try because I lacked the requisite hardware.

But I was feeling ambitious, so I gave it a chance. There’s not much to report though.

2013-10-19-lv-r1fz6-efax-01 2013-10-19-lv-r1fz6-efax-02

I don’t have a fax line to test — heck, I don’t even have a POTS cord in the house any more. I haven’t used a land line phone in a decade, and I don’t think I ever had a home fax line.

On the other hand, I know fax service is still around. Twelve years ago my office used a fax line, but it was antiquated then.

On the other hand, a few years ago in Japan our office used it on a daily basis (for reasons that I believe are mostly cultural). So it’s not like the demand is completely gone.

Oddly though, as you can see in the screenshot for the virtual console, efax seems to be spitting out an error both times.

On the other hand, the GTK interface — which is probably preferable anyway — seems comfortable with hovering in the system tray.

I can’t say for sure that this will work (this is the Arch version of efax-gtk, by the way) but there’s always the possibility.

P.S.: Check the “enhanced” version of efax too.