mkfifo: Pipe panjandrum

I’m going to guess that you probably know what a pipe symbol does on most Linux systems — passes the output of one program to a second. It’s what allows you to do things like this

dmesg | grep ATA

and find local hard drives. Or gives me my topics for the day, with

ls vimwiki/ | shuf -n1

mkfifo is part of coreutils, and allows you to name a pipe, and reuse it over and over. It might sound odd, but it works in much the same way as the symbol.

mkfifo pipe

Now I have a file entry called “pipe” in my directory, marked with a “p” in the leftmost column.

kmandla@6m47421: ~$ ls -lha
prw-r--r--  1 kmandla users    0 Sep 21 07:42 pipe

Now I can jam something in that pipe, if I like.

kmandla@6m47421: ~$ ls vimwiki/ > pipe

And … apparently, nothing happens. My terminal is paused, somehow waiting for an action. The suspense is killing me. 🙄

Actually, until something collects the material in that pipe, it will pause there more or less indefinitely. So if I …

kmandla@6m47421: ~$ grep ch < pipe

The whole flood comes out, and we can all relax again. Crisis averted. 😐

It might seem rather pointless or irrelevant to name a pipe or even to make note of mkfifo for it. But you’ve already seen the subtlety in action, and maybe just didn’t think about it.

That first terminal emulator was paused, waiting for someone to unblock the pipe. A second terminal received the data and did something with it.

So what? So … not only can you jam a program’s output into a holding area, waiting for a recipient, but that also means you can pass data between terminals with a named pipe. So if you’re waiting for one program to end, you can send the output of another into stasis, until it’s ready.

And you can pass information between different programs running in different terminals, at virtual consoles, in and out of a multiplexer … possibly even between users or across distant machines if you’re clever. It’s not the best way to do those things, but it might work in a pinch. Experiment and see.

I know named pipes are not anything new, but little things like this are the reason I wanted to sift through coreutils again. If you need a better explanation of named pipes, and you don’t mind reaching back almost 20 years, Andy Vaught’s explanation from issue #44 of Linux Journal is a great starting point. Nice to see that things haven’t changed that much since ’97. … 😉

tcat: Just quickly, because it’s fun

I wanted to squeeze tcat in today, not just because I have a string of task managers that will otherwise monopolize the week, but also because it’s fun.

tcat saves you the stress of splicing date and cat, and working out the funky sequence to divide them with a tab character.

The help pages promise that tcat does absolutely nothing other than stream data through, and prepend it with a time stamp. So far, it has kept that promise for me.

tcat will let you wrangle the stamp format, in the same way you would might use the date command alone. Together that allows you to pipe any kind of output you like, through tcat and get the same information, just stamped.

Nifty, huh? 😀

pv: Why won’t you talk to me?

One of the things that I remember being excited about when I started tinkering with Ubuntu all those years ago, was the sheer volume of new, free toys to play with.

There are, after all, some amazing, amazing tools available in Linux, and not just at the command line. Some of them just make your heart leap with joy as soon as you try them out.

Others … well, they’re useful, but they’re far from electrifying. For example:

kmandla@lv-r1fz6: ~/temp$ ls -lha
total 4.1G
drwxr-xr-x  2 kmandla users 4.0K Mar 19 14:25 .
drwxr-xr-x 27 kmandla users 4.0K Mar 19 15:07 ..
-rw-r--r--  1 kmandla users 4.0G Mar 19 13:15 sample-01.txt

kmandla@lv-r1fz6: ~/temp$ cp sample-01.txt /dev/null
_

And it sits there. And sits there. And sits there. With no clue as to what’s going on or what is happening, as if suddenly stricken dumb.

Now this on the other hand, is much better:

See that? Why is it so hard for cp or one of the other core Unix-ish tools to just tell me what the heck is going on? I’m looking at you, dd. And mv.

This is not a new complaint. I have mentioned this before, with things like AdvancedCopy. Just a little clue as to what is going on in your head, computer. That’s all I want.

The beauty of pv, which is what’s working in the screenshot, is that it is very flexible. You can use it for progress bars on compression or decompression, network transfers, erasing floppy disks, spilling files into /dev/null … you name it.

You can even dump it into dialog, and give yourself a proper screenful of blue-and-red progress indicators … a la MS-DOS6.22 or something. 😉

You’ll have to take a look at the help pages to see how the finer points are worked, but I’m sure you can handle it in its basic form. My best advice is to think of pv like cat, but smarter.

Now if only I could get a petition signed to have pv incorporated into the standard tools. … Ha! 😆

pip: A tool to save for later

I forget where I learned about pip. Here’s what it looks like, in action, literally:

(Aren’t animated gifs fun? I should have puzzled that out a long time ago.)

pip is cool because it allows you to use just about any program like a filter, sending data through it and back out again.

It took me a little time to grapple with what was going on there, mostly because I rarely think of things in terms of filters or piping.

But what it means is you can dump text straight into vim (like you see in the gif above), adjust it, then send it on to something else. Clever.

Of course, I can’t remember ever needing something like pip, so it may be a while before I run into an actual case study for pip.

Until then I shall keep it in mind, as a tool to try later. 🙂