Tag Archives: find

cfind: A search tool built on a search tool

I had to read the home page for cfind more than once to figure out what it does, but I don’t think reading it any more will help me understand why it would be useful.

cfind “provides functionality similar to that of Google Desktop from the command line” — and that much I understand. But the little intricacies are escaping me.

cfind needs locate, which is part of mlocate in Arch and not installed by default. Before you can use locate, you have to run updatedb. After that, locate can scramble through the database of your files and pull out just about anything you want, in fractions of a second. That we saw long ago.

Before you can use cfind, you have to run cindex, it’s partner application. Once all those steps are complete, you can query cfind for search strings. So once more: updatedb, then locate if you want, then cindex, and finally cfind.

And here’s the kicker: cfind and locate will generate almost identical output, with two exceptions that I can see. cfind apparently only searches through text and TeX files, and cfind orders its results by “relevance.”

Here’s what it looks like when it’s finished, sort of.

2014-11-03-2sjx281-cfind

There I sent the results of locate, on the left, and cfind, on the right, through paste with column -t | most for readability’s sake. ๐Ÿ˜‰

I guess my confusion lies in the need for a tool to skim a database made from another database, when there’s already a tool that skims the original database. O_o How is cfind better than using straight locate, and filtering the results for TeX or text files?

These are the questions that keep me awake at night. ๐Ÿ˜

I appreciate that cfind might fill a niche, but I only rarely work with TeX files and don’t have enough text files to justify a search tool built on a search tool. And without any guidance on its algorithm for “relevance,” I don’t think I’ll be keeping this around.

For the record, this is the second tool I’ve come across in two years that lays claim to the “Google Desktop for the CLI” title. The other, you might remember, was doodle. Neither one seems to have usurped the standby, Unix-esque tools — find or locate — that are already available. ๐Ÿ˜ฆ

ngp: Find it and edit it, in one deft motion

These days I seem to be finding less and less tools that really grab me, and more and more that seem to fall short of the mark somehow. ngp is one of the former, and this one I think I shall keep around.

ngp is like a recursive, colorized, interactive grep. If you take the output of ack, make it navigable with a selection bar, then add the ability to jump straight into a selected file with your $EDITOR, you’ll have an idea of what ngp does.

So for example, this

 ngp -r ion

Takes me straight to this:

2014-10-03-6m47421-ngp

And from there, if I highlight a line and press enter, I jump into $EDITOR and can adjust as necessary. Leave the editor and ngp reclaims the terminal, showing the same results but with red text, so I know I already traveled there. Very smooth. ๐Ÿ˜Ž

ngp strikes me as a tool someone would invent where they see themselves repeating the same task over and over, and relying on two or three terminal windows to track their progress. And if you work in text files a lot, or through trees of code, I imagine ngp would save you some time finding certain problems, and fixing them.

ngp can be a little finicky, so watch your flags carefully. And I might have compared it to grep but it’s not feature-compatible, so don’t start throwing wild strings of flags at it and expecting it to munch them down the same way grep does.

But for a find-and-replace or find-and-edit or search-and-destroy at the console tool, ngp has everything tied up in one, nice, neat package. A well-earned, albeit completely valueless, K.Mandla gold star: โญ Enjoy! ๐Ÿ˜‰

tmenu: Just one step away

The next time you hear someone whine about having to use the command line, you can provide them with a long list of utilities that deftly convert lists and applications into menu format. Putting aside pdmenu — the dedicated application menu tool for the console — you still have things like slmenu, fzf, sentaku, percol and now Dario Hamidi’s tmenu

2014-09-07-6m47421-tmenu-01 2014-09-07-6m47421-tmenu-02 2014-09-07-6m47421-tmenu-03

tmenu works a lot like sentaku or percol, accepting piped-in lists and returning to the screen. Unlike fzf, tmenu does not assume you want the current list of files, so you have to provide something; just entering tmenu gives you an empty list and a rather pointless tmenu experience.

With a proper list, you have the option to filter by character string, or navigate with CTRL+N, CTRL+P and so forth. Press return, and your selection is returned to STDOUT.

So this too can function in the same way as the slmenu gimmick, without the color that percol offers, without the vertical arrangment that fzf has, and perhaps a little more space-conscious than sentaku.

tmenu takes three flags — one for the number of items in the displayed list, one to change the prompt and one to take out the status line. That’s it. Very simple.

About the only thing I don’t like about tmenu is the lack of arrow key controls for list navigation. I know that’s fairly minor, but my instinct is to jump for the arrow keys when I’m presented with a list. CTRL+N and CTRL+P make sense when I think about them, but there’s always that split second when I don’t think, and just start tapping fruitlessly at the arrow keys. Grr.

All that being said, I’m wondering if it’s not time to just hotwire tmenu — or one of its brethren — into the /usr/bin folder and run it perpetually, like a Grand Unified Menu. And so this

$(ls /usr/bin | tmenu -l $(tput lines) )

Would give you this:

2014-09-07-6m47421-tmenu-04

Not as scary as I thought. I guess we’re all just one step away from a completely menu-driven terminal experience. ๐Ÿ˜‰

fzf: More menu-like options, with fuzzy finding

I said a long time ago that find is one of the greatest creations for computers, probably since the invention of electricity.

Junegunn Choi’s fzf might make you think hard about it though.

2014-08-05-6m47421-fzf

That’s a slooow animated gif, isn’t it? I’m experimenting with stop-motion animation. ๐Ÿ˜†

In much the same vein as sentaku and percol, fzf accepts a list — by default, the list of files in the current directory tree — and allows selection by way of fuzzy finding. As you can see there, not only does fzf pluck out entries that contain “an” exactly, but also retains options for longer, less-strict matches.

Making a selection returns the value to STDOUT, making fzf another candidate for the oft-mentioned slmenu gimmick. Again, though, like percol, you might have to arrange that list of programs as a text file.

Then again, maybe not.

ls /usr/bin | fzf

Oooh, that’s tempting fate right there. ๐Ÿ˜ฎ

fzf is a lot less visual than percol or sentaku, but approaches the concept — picking out a selection from a large list — from a pattern matching perspective. It would take a little more command-line prowess to make percol act in the same way.

fzf is fast, clean, colorful, useful, intuitive, easy to use and doesn’t require a complex set of commands to employ. It can return multiple selections, and it plays well with other Unixy tools. So as Junegunn suggests, something like

vim $(fzf)

is viable. (I should probably mention that I tried to shunt the results of find into fzf, but there were some snags.) It might also play nicely with something that cues music files. Think about that one for a little bit.

I’ll leave you alone to meddle with fzf for a bit. Perhaps even more than slmenu or sentaku or percol, fzf has the potential to change the way you work at the command line. And for that … I think fzf is eligible for a rare K.Mandla gold star: โญ Enjoy! ๐Ÿ˜‰

xargs: Obviously first

I feel strangely underqualified to mention xargs, even though it’s on my list as the first tool in the X section … once X itself is thrown out, of course. ๐Ÿ™„

I’ve known about and used xargs for the better part of a decade now, and it continues to crop up from time to time.

Strangely enough, I couldn’t spit out my own example of how to use it, even if I tried. It’s one of those things that’s around just enough to be useful, but not so often that I take the time to learn it right.

There are a few fun things that you do with xargs, that are worth mentioning.

2014-07-04-6m47421-xargs-01

The problem with that particular example is that you get the same results from

kmandla@6m47421: ~$  find music/ -maxdepth 1 -type d -exec echo "Directory: "{} \;

and so I can’t help but wonder what use it would be.

More helpful is the ability to pass specific information to other programs with xargs. For example. …

2014-07-04-6m47421-xargs-02

There, the results of ls are fed into wc via xargs. The problem again though, is that you get identical results from

kmandla@6m47421: ~/temp$ wc -l output-*

(Some restrictions may apply.) Which, at this point, probably makes you wonder what use xargs is, if so much of it is more easily done with the options available in other software.

Well here’s something that would be a little more tricky without xargs.

2014-07-04-6m47421-xargs-03

There, the -n 2 flag told xargs to lop off the results two at a time, and continue through the list to the end. So each pair of files gets run through wc, a total is shown, and then the next two are sent. An esoteric example, but nifty.

Here’s one more.

2014-07-04-6m47421-xargs-04

This time, xargs encountered the -t flag, which tells it to display the command it has received before showing the output. Useful for troubleshooting your command, or as a visual border for the results you get.

I’m sure you can come up with some more interesting and useful examples than these. Like I said, I encounter xargs just often enough to know what it does and why it’s necessary, but not enough to get into the nitty-gritty.

xargs is in findutils in both Arch and Debian. ๐Ÿ˜‰

slocate: All the makings of an intercontinental grudge match

I could be left alone in a windowless room with only a bare light bulb and find to search out the lost files on my computer, and I’d be fine with that.

Then again slocate, and its evil stepsisters in the mlocate coven, might give find a run for its money.

kmandla@6m47421: ~$ time find /usr/lib -iname *pcmcia*
...
/usr/lib/modules/3.14.1-1-ARCH/build/drivers/char/pcmcia
/usr/lib/modules/3.14.1-1-ARCH/build/sound/pcmcia
/usr/lib/initcpio/install/pcmcia

real	0m0.400s
user	0m0.120s
sys	0m0.090s

And the upstart?

kmandla@6m47421: ~$ time slocate -qi pcmcia | grep /usr/lib
...
/usr/lib/udev/pcmcia-check-broken-cis
/usr/lib/udev/pcmcia-socket-startup
/usr/lib/udev/rules.d/60-pcmcia.rules

real	0m0.303s
user	0m0.240s
sys	0m0.000s

Uh-oh. This could shape up to be the match of the century. ๐Ÿ˜ฏ

Loyalists will insist that slocate is cheating, because it doesn’t really search so much as skim through its database and pick out the stuff you want.

Bolsheviks will counter with reduced time searching, less disk thrashing and adherence to the Second Law of Robotics.

For my own part, I’m not a big fan of databases and meta-information on my system. I appreciate s/m/locate‘s speed and conservation of energy, but my allegiance lies with find.

And really, I know where everything is on my computer. I don’t go searching for stuff that often. ๐Ÿ™„

So I say … down with the revolution! ๐Ÿ˜ˆ

rdfind: Echolocation

I once destroyed — utterly destroyed — a Windows XP installation by playing fast and loose with a utility that sought out duplicate files and arbitrarily removed them.

You can imagine the havoc that caused. I couldn’t tell you the name of the utility now, and it doesn’t really matter except that tinkering with rdfind brought that memory back.

2014-03-31-lv-r1fz6-rdfind

No, I didn’t destroy any Linux installations today, and I daresay that neither Linux nor rdfind would allow me to utterly decimate the system without at least showing some credentials. I get by with a little help from my friends.

It does sound like the author of rdfind may have had similar experiences though, given the explanation on the home page and rdfind’s options for linking files, as well as removing them outright.

I also like that rdfind creates an output file, showing the fruits of its labor and giving you a report on its opinions. Every program should be so polite.

Seeing as rdfind was Ian Munsie’s suggestion, I suppose I should offer one small note of thanks for pushing me in this direction. I do think it’s a step above fdupes, in technical terms.

Now all I need are some duplicate files to thrash. … ๐Ÿ˜ˆ