x_x: The Dead Guy CLI

With barely a week left for this site, I’m beginning to trim away programs that I just probably won’t get to, by virtue of time or technical dilemmas. I’m also making a conscious effort to pick out titles that amuse me in one form or another, so I finish with happy memories. 😛

x_x, which I mentally refer to as “the Dead Guy CLI,” because the home page uses that as a subtitle, is a rather nifty tool that I’m surprised I haven’t seen covered elsewhere. Using a bland, dull, boring Excel spreadsheet borrowed from a corner of the Interweb, Dead Guy CLI transmogrifies it into this:

Well isn’t that clever.

Dead Guy CLI gives you a small measure of control over your output, by allowing you to specify a header row or allow for special encoding. It also works with CSV files, so you’re not strapped trying to convert back and forth to Excel, just to fiddle with x_x.

Aside from that though, Dead Guy CLI seems very simple. Of course, your spreadsheet may need some management if you expect it to fit into a certain dimension, but I am confident that as a skilled and capable member of the information age, you won’t throw a wobbly over a pear-shaped spreadsheet.

Keep x_x in mind when you’re thinking about things like csv2xls or xlhtml, since it may save you a step or prevent you from relying on graphical tools just to extract data from a spreadsheet. And of course, if you’re working with csv files, x_x could supplement what tabview or other tools can do.

For my own recordkeeping, Dead Guy CLI gets points for doing something obvious that I don’t recall seeing elsewhere. And also for the snarky name. I’m a fan of snarky names. 😈

ncdt: An interesting evolution

Quick on the heels of tree and ddir, a loyal reader pointed out ncdt, which takes the tree model and adds a small feature or two.

As you can see, ncdt adds the size of files and directories as part of its default display. So in situations where it’s useful to see directory structure and size — such as labeling removable media, like CDs — it is probably more useful.

Unfortunately, I see no options to adjust what ncdt shows, so there are no “human-readable” (which I prefer) output flags or the like. What you see is what you get.

ncdt also promises to show “special” data for mp3 files, but the Debian version as well as the version I built on my Arch system from the Debian source package showed nothing. Even the sample screenshot in Debian doesn’t show anything “special” for mp3 files. Hmmm. 😦

It’s possible that there is an added dependency that I don’t have, or perhaps the mp3 files I tried post-date what ncdt is capable of analyzing. I checked the ReadMe and source files, but I got no hints. And the only home page I have for ncdt is the Debian package page above.

No matter. ncdt adds a little to the tree model and could probably, at one time in the past, show a little information about mp3 files. It’s an interesting evolution, even if it still needs some attention to reach fruition.

textprint: Visually impressive, in only 18K

You can get graphing and plotting functions in the console from a variety of sources. textprint is easily my favorite for simple data arrays, mostly because it can do this, with only 18K:

  

textprint takes a flat data file as input, and arranges it graphically to fit the terminal without distorting the image. From there, textprint goes from zero-to-60, in about two seconds.

Because on top of a rather bland plotting display, you have the option to pick between about four or five different graphs, including the bar and column charts you see above, and a couple others.

And then, you can shift those displays along the x or y axes, zoom the displays and even “print” the display to a text file that matches what you see on the screen. (I did see some corruption when trying to zoom in too “close,” though.)

So you have essentially a tool to convert data arrays into visual representations, adjust them to your liking, then “print” them in their new format. And all that in only 18K.

textprint is not without shortcomings, but truth be told, I could sift through anything and find small nits to pick at. Color options would be nice, and while it does have onboard help, there aren’t any flag options that I can see. If I could send commands to textprint and have it spit out a “printed” file without the interface, textprint would be doubly useful.

And to be honest, the title “textprint” is a slight misnomer. That’s going to get lost in the endless array of pdf converters, ASCII readers and document translators that already pepper the ‘web. It needs a more accurate, and more descriptive title.

It’s still exceptionally impressive though. And the fact that it has so many options in such a small space. And seeing that it’s a dozen years old and still working is noteworthy too.

Not in Arch/AUR or Debian that I could find. textprint is bundled with a precompiled binary, but it’s going to look for an ncurses library that didn’t exist on my Arch system. The command to compile it is in the readme.txt file.

ascii: Quick and dirty info

This next title is a quick and obvious tool, and I had been holding it back because it seemed a little too simple. This is ascii.

Which, as you can see, either displays a chart of ASCII conversion data, or takes a string of digits and returns their conversion, as would be useful in a script. Escaped characters and control characters are possible too, as well as octal or hex values.

ascii does a few other things that serve in the same kind of situation; check the help flag or the man page for ascii’s second-string tricks.

But … that’s all I can think to do with it. The value in ascii is its quick-fire accessibility to character codes. And beyond that … ?

nwipe: Great trepidation and fierce consternation

Both AUR and Debian agree that nwipe is the core tool once found in the dban disk annihilator tool. I say “once found” because the home page seems to have changed from what I remember, and it appears to be pitching another utility called “Blancco.”

But to add to the confusion, AUR and Debian have different home pages for the nwipe project, one pointing to Sourceforge and another to a Github page, respectively. It’s possible those are just mirrors for the project, but I do wonder if there are differences.

No matter; what you see here is the version available to Arch users.

And it doesn’t look much different from the core dban program. Select a drive, step through your options, and start with F10. If you ever used the dban live system, this single program works much the same way.

You have the ability to set your erasing options as command-line flags, which is something that was technically available to the original project, if you were willing to remaster the ISO.

And the infamous “autonuke” option is still around, which starts up and immediately begins exterminating every bit of data on any drive it can find. The implications are frightening.

I see a few improvements here and there, possibly most unusual being the ability to blank the screen — not erase the screen, just empty it 😳 — while nwipe runs, ostensibly saving you the screen burn-in while nwipe eats up your hours … and your hard drive.

It might also be important since the version I used had a once-per-second flicker effect as it updated the screen. Blanking the screen was the only solution for stopping the flashing blue box in the corner of my eye.

nwipe is comfortable running outside of 80×24, but oddly, all the text will remain in those dimensions even if the frames are drawn larger. Odder still, the on-screen help along the bottom gets arbitrarily cut off at 24 characters, which seems like an oversight.

dban is a time-honored tool that I’ve been using off and on for the better part of a decade, and so I hold it in high esteem. I’ve run it for weeks on end, 24 hours a day for various reasons, and gotten great results with it.

As an offshoot, nwipe is equally regarded, even if I bundle my meager endorsement with a stern warning: Be very very careful when you use it. nwipe does not make apologies. Use with care. 😡

dares: Qui audet adipiscitur

I don’t have a proper home page for dares, so if you can track it down, let us know where the original is found. Until then, here’s the Linux Mint version and the Debian package page.

dares is a CD recovery tool which apparently can retrieve data even from unmountable discs. As luck would have it, I don’t have any unmountable discs, so my leftover PLOP boot disc will have to suffice.

dares needs little more than a device address (or file name, in the case of a corrupted image, which is also a potential target) and a directory to place its files before it will swing into action.

After reading the disc content, you get a straightforward menu for recovering and saving accessible data. Fairly foolproof, and fairly reminiscent of photorec or a few other data recovery tools.

Beyond that point though, I can’t really vouch for dares any more than I could for foremost or safecopy, just because I don’t have a damaged medium to test. If you do, and you’ve found your way to this page, I can only hope it does the trick for you.

Keeping this — as well as the other three I mentioned — in mind when I do stumble across a damaged drive … well, that’s the other trick. 😉

burn-cd: In spite of its age, quite useful

Looking over the Sourceforge page, burn-cd seems to have seen its best days almost six or seven years ago, and if I’m reading that right had its last update in 2009. I know that means it won’t appeal to some people, but bear with me:

Because burn-cd works fine once it’s pointed at your optical drive, and has your files in tow.

It requires no more prodding than the folder or files or ISO you want to write. And it keeps you well informed of its progress, in color and updated constantly. No guessing about what’s happening out there in CD land.

Technically it’s old enough to default to /dev/hdc for your CD burner, so you’ll want to set it to /dev/sr0 or what have you in .burn-cd.conf. While you’re at it, I recommend the verbose = yes setting, which does send a lot more information to the console.

But after that, it’s just burn-cd /home/kmandla/files, and the deed is done. I like that.

Ordinarily I would push for some kind of interface and maybe some push-buttons or a couple of spinny thingies while it’s writing, but the word of the day for burn-cd is “clean and simple.” Nicely done. 😉

burn-cd is in AUR but doesn’t show up on the Debian search pages. Perhaps it was once part of the Debian arsenal but has been sloughed off; if that’s the case, it’s a pity. 😕

paste: What I thought join would be

I just showed paste in the last post but I haven’t mentioned it on this site. I probably should have reversed the order here, since paste is one of the last coreutils toys I was holding back from the leftover slurry.

paste does what I thought join should do — concatenate two separate data files, line by line. Again, this is something easier done than said:

That looks almost identical to what join does; here’s where they differ:

paste at least hints that there were omissions in one column or the other. join, on the other hand, skipped over those items, and demanded they be sorted.

Of course, seeing paste and join side-by-side makes a lot more sense in why they’re named as they are. join links together corresponding entries according to a sorted order. paste just forces them together, even when something is missing.

I’d still like to see paste insert a tab where the first list is missing a line, but at least now I get the picture.

I handed datamash a small gold star for transposing its output, and paste has a similar function in its -s flag.

So you can run out vertical lists horizontally, if you are so inclined.

I’m quickly running out of coreutils titles, and I do so enjoy learning about them. Perhaps one day I shall start a blog that only steps through that and the util-linux package, and looks at each tool one at a time. … Nah, who’s got time for that? 😕

datamash: Statistical tools for raw numerical data

I like tools that do simple things in obvious ways. I like tools that have color too, but sometimes I’m willing to forgive that, and award points on cleverness.

Here’s datamash, a GNU tool, doing something fairly straightforward.

Forgive my rotten formatting. I was trying to line up the sums under their appropriate columns, but I ran out of patience with it. What you should see there are two arbitrary columns of numbers, and datamash summing both on the fly.

It’s not a terribly earth-shattering function, but it does make a lot of otherwise tricky number functions accessible to flat numeric data files. So you don’t have to import into a spreadsheet to get a sum, a mean, a max, a min or whatever.

And you don’t have to rely on statistical packages like r or octave to do some simple budget analysis. 😉

datamash also does some rather clever text formatting tricks, which might be reason enough to keep it installed. Observe:

So you can feed datamash a series of columns vertically, and it will run them out horizontally. Omigoshthatissocool.

datamash is in AUR and testing/unstable. I don’t know why it’s not in the standard repos for either distro, except that it may be too new. The development pages for datamash suggest it started about a year and half ago, but saw most of its activity within the past six months. Give it time.

A lot of what datamash can do — particularly the higher statistical functions — are much more than I would need on a day-to-day basis. But if you keep something like r or octave on board for regular data analysis, you might consider datamash as a lighter alternative.

mkfifo: Pipe panjandrum

I’m going to guess that you probably know what a pipe symbol does on most Linux systems — passes the output of one program to a second. It’s what allows you to do things like this

dmesg | grep ATA

and find local hard drives. Or gives me my topics for the day, with

ls vimwiki/ | shuf -n1

mkfifo is part of coreutils, and allows you to name a pipe, and reuse it over and over. It might sound odd, but it works in much the same way as the symbol.

mkfifo pipe

Now I have a file entry called “pipe” in my directory, marked with a “p” in the leftmost column.

kmandla@6m47421: ~$ ls -lha
prw-r--r--  1 kmandla users    0 Sep 21 07:42 pipe

Now I can jam something in that pipe, if I like.

kmandla@6m47421: ~$ ls vimwiki/ > pipe

And … apparently, nothing happens. My terminal is paused, somehow waiting for an action. The suspense is killing me. 🙄

Actually, until something collects the material in that pipe, it will pause there more or less indefinitely. So if I …

kmandla@6m47421: ~$ grep ch < pipe

The whole flood comes out, and we can all relax again. Crisis averted. 😐

It might seem rather pointless or irrelevant to name a pipe or even to make note of mkfifo for it. But you’ve already seen the subtlety in action, and maybe just didn’t think about it.

That first terminal emulator was paused, waiting for someone to unblock the pipe. A second terminal received the data and did something with it.

So what? So … not only can you jam a program’s output into a holding area, waiting for a recipient, but that also means you can pass data between terminals with a named pipe. So if you’re waiting for one program to end, you can send the output of another into stasis, until it’s ready.

And you can pass information between different programs running in different terminals, at virtual consoles, in and out of a multiplexer … possibly even between users or across distant machines if you’re clever. It’s not the best way to do those things, but it might work in a pinch. Experiment and see.

I know named pipes are not anything new, but little things like this are the reason I wanted to sift through coreutils again. If you need a better explanation of named pipes, and you don’t mind reaching back almost 20 years, Andy Vaught’s explanation from issue #44 of Linux Journal is a great starting point. Nice to see that things haven’t changed that much since ’97. … 😉