Tag Archives: management

tcpflow: That tool you’ve been looking for

If you’ve ever casted about the Internet, looking for a tool that will watch line traffic, keep copies of transferred images, break down network requests into human-readable chunks, then compile everything into a PDF report … you’ve been looking for tcpflow.

2014-05-19-jk7h5f1-tcpflow-all

I realize a large chunk of what tcpflow does isn’t really intended for text-only environments. And while it’s running, tcpflow doesn’t give you a whole lot of information about what’s happening. So tcpflow is hardly worth mentioning as a console application.

But the final results, and its meticulousness in the mean time, are quite impressive. I tried to lump everything into that screenshot above. Something tells me tcpflow could do more, but I’d need a bigger screen to show it. 😯

So far, I see that it can shunt images into a destination folder, tag everything with filenames made out of source-and-destination IPs, break down individual transfers and arrange them in a way that is infinitely easier to read than tcpdump’s Wall of Data, and perhaps best of all, generate a PDF report at the end of the run, showing still more statistics, in living color.

It’s great stuff, and I should think particularly useful to network administrator-types, who might need to see what the office staff is really doing during their work hours. 😈

sqlite3: I won’t begin to try and explain

I have sqlite3 on my list of applications, and I know you can access sqlite database files with it, in console mode.

2014-05-07-6m47421-sqlite3

But I’ll be dead honest: I haven’t a clue what to do next. The .help command ticks off a huge list of commands, and I am hopelessly lost as to what any one of them — except perhaps for .exit — does. 😯

So I mention this only out of a sense of obligation, but I’m just crossing it off my list with this post.

And because I know if I don’t mention it, somewhere down the line, I’ll get that e-mail that starts off with, “Hey, what about sqlite3? Why didn’t you didn’t mention that one?! I use that every day in my job as. …” πŸ™„

So there it is, for that unnamed person in the future. Enjoy. πŸ˜€

socat: You want it? socat’s got it

Back when I mentioned netcat, I said that no matter what I wrote, I’d be underemphasizing how useful and powerful it was.

The same holds true for socat, which is probably more flexible and more detailed than netcat, if such a thing is believable. Like netcat it’s primarily a network piping tool, but just a skim through the man page tells you it’s taken the idea to a different level. 😯

I could claim some experience with netcat, but socat was new to me until today. But here’s what I collected from around the Web, in terms of fun things to do with it:

Create a virtual network interface:

socat -d -d tun:10.0.0.1/8 tun:192.168.0.1/24

Not that I’m short on network interfaces, but that’s a pretty cool stunt. Thanks go to JustChecking for mentioning that one.

socat can also pull in or redirect data delivered from other programs, which makes it useful in other ways.

date | socat - GOPEN:/tmp/capture,append

And of course, if you take a peek in /tmp/capture now, you’ll see the date listed. Not a huge leap forward for mankind, but has potential if you think about it. Thanks to linux.com for that one, and for this one — creating a virtual private network over ssh:

socat -d -d  \
    TUN:192.168.32.2/24,up \
    SYSTEM:"ssh root@server socat -d -d  - 'TUN:192.168.32.1/24,up'"

I have to admit, that’s clever. One more here, this time from My Stuff:

socat UNIX-LISTEN:/tmp/.X11-unix/X1,fork \
SOCKS4:host.victim.org:127.0.0.1:6000,socksuser=nobody,sourceport=20

Supposedly this attaches itself to the Unix socket created in an X window session, and redirects to an external site. I admit this one I didn’t try, and since the My Stuff page dates back to 2008, it’s possible this doesn’t work like described. An interesting idea though, and lots more on that page.

It’s possible to create software that is so flexible and precise that it is both amazing and bewildering at the same time. socat is proof of that.

sendip: Packets and their contents

You’d think for all the esoteric network tools I have blundered through over the past year or so, I would have a better grasp of some fundamentals.

I don’t. πŸ™„

2014-04-19-6m47421-sendip

That’s sendip, which I like for being easy to figure out, giving plenty of information, and looking good in a screenshot. 😳

It does look good though. It spills the contents of the packet to the screen, including whatever piggybacked data you specified, and gives you a report of the transaction. Clean, neat and talkative.

What you do with that … I really don’t know for sure. It must be a science thing. 😯

Of course, without the flag for verbosity, sendip is meek as a kitten. So you have the option, if you prefer the strong, silent type.

sendip is in AUR and Debian, and strikes me as a useful tool to keep around. Even if I don’t know what it does. But when has that ever stopped me?! πŸ˜€

rdiff-backup: Mirrored, with increments

I charged into rdiff-backup thinking it would be only a little more complex than rdiffdir was yesterday. Luckily I wasn’t too far off the mark.

rdiff-backup can make backups while conserving bandwidth, which is probably a great idea on the whole. It also makes incremental backups, and the home page promises file recovery over previous backups too.

I didn’t delve that far into it, but I do have a little to show for my effort:

2014-04-01-lv-r1fz6-rdiff-backup

My hope there was to show that rdiff-backup’s product is not only a mirror image of the source, but also includes data on what changed between runs. It might be a little difficult to follow; trust me if it’s not obvious.

Compared to a straight rsync, I can see where this would be preferable, if it conserves bandwidth and can offer access to past backups as well. I usually just refresh my archives with a simple rsync -ah --progress --delete, and there have been times I wished I could step backward once or twice in history.

On the other hand, this is very clean and straightforward, without a lot of the wrangling that I’ve seen in some other console-based backup tools. Given the need — such as a large-scale networked system — I’d definitely think this over as an option. πŸ˜‰

qpdf: Still more PDF wizardry

Just when you thought all your PDF options were expended, along comes qpdf to rattle your cage once more.

The home page uses “pdf-to-pdf” as a subtext for what qpdf does. And most of what qpdf does is just that: manhandling PDF files and getting the results you want, without distorting or smushing the content. (“Smush” is the scientific term.)

For example, reversing the order of pages in a pdf file.

kmandla@lv-r1fz6: ~/downloads$ qpdf --empty out.pdf --pages intro-linux.pdf z-1 --

--empty signifies a new file named “out.pdf,” --pages tells qpdf to wrangle pages, “intro-linux.pdf” is the source file and z-1 is pages 1 to the end, in reverse order.

qpdf is quite forgiving about page orders, sequences and even multiple sources. I haven’t tried every permutation, but I suspect you could do some real PDF wizardry with the freedom qpdf allows.

There are a lot more ideas in the qpdf documentation pages, which is so up to date that it’s timestamped in the future! πŸ˜‰ On a more serious note, the ins and outs of qdf’s power are all listed there, in precise detail.

It might take a little while to learn all the tricks qpdf makes available to you, but I have the feeling that if every other PDF gizmo fails you, qpdf will satisfy. πŸ˜‰

proxychains: Sorry to be of so little help

I’m into the P section to where proxy tools take over, and unfortunately into another fuzzy area.

I’ve never knowingly used a proxy utility, unless it was part of an office network and I just didn’t know about it.

So again, I’m trying to speak in vague terms because I have so little experience with them.

From what I gather about proxychains though, it looks fairly simple to set up. The howto page doesn’t say much, but the configuration file is well documented and I have a feeling even I could tackle it.

kmandla@lv-r1fz6: ~$ proxychains --help
ProxyChains-3.1 (http://proxychains.sf.net)
/usr/bin/proxychains: line 9: exec: --: invalid option
exec: usage: exec [-cl] [-a name] [command [arguments ...]] [redirection ...]

kmandla@lv-r1fz6: ~$

But beyond that I don’t have any advice. I can’t be sure this is any better than other proxy utilities either, and I don’t dare offer an uneducated opinion on something so important.

Sorry to be of so little help. 😦

paste: I would vote for a different name

I know I’m almost 50 years behind the curve on naming ancient Unix commands, but paste just seems … wrong to me.

paste, if you believe in man pages, should “write lines consisting of the sequentially corresponding lines from each FILE, separated by TABs, to standard output.”

That’s fine. And I even get paste to do some fun things for me. Like …

2014-02-23-lv-r1fz6-paste

Granted, those columns are a little slipshod, but I blame myself for that. If I trim each word to six characters, paste lines things up quite nicely.

(Which incidentally raises another question: How to change tab widths at the console, outside of any application … ? Hmm. … πŸ˜• )

No, I have no real complaints about what paste should or shouldn’t be doing. It’s working as I supposed it would, and without an egregious amount of effort.

But still … “paste”? “splice” might have worked for me. Or “tabulate.” Or “columnify.” paste just seems … oblique. Tangential. A misnomer.

Too late now though. I have about as much chance of ever getting the name changed as I do having this scurrilous little blog declared required reading for all primary school students in the U.K. No way, no how.

No matter. paste it is. Let’s move forward.

P.S.: I’ll give you a hint which package owns paste: It starts with core and ends with utils. πŸ˜‰

num-utils: This and that, for numbers

Most of the tools I run across in this silly little adventure are intended for text manipulation. Something that manhandles numbers is … well, unusual.

Enter num-utils.

2014-02-11-lv-r1fz6-num-utils

num-utils comes with an array of tools, mostly involving basic math functions that might otherwise be a bit cumbersome.

For example, there’s random, which — as you might imagine — generates random numbers, given a range.

random can also generate numbers on a particular interval, through decimal ranges or within certain bounds. Not terrifically flashy, but without it, the task could be a little more arduous.

average — again as you might imagine — calculates the average out of a string of numbers. It can also show a median or mode, if you ask nicely.

There are some more esoteric tools. interval — as you might not imagine — calculates the difference between numbers in sequence. So between the numbers 100 and 50 in a sequence, interval would return -50. Unusual.

The home page for num-utils is exceedingly sparse, and the README file suggests everything in the package is in a beta stage.

I hold no grudges against beta software, but that file is stamped 2003. 😯

So if you’re waiting for a stable version of num-utils so you can arithmetize your checkbook, I advise being proactive. πŸ˜•

I should also mention that “beta” is probably accurate, since I did get one or two errors when I used some of the tools, most notably range.

I should also mention that the names given to the tools might run you into conflicts with software in other packages. The title “random” for example is something that appears in bsd-games, unless I’m mistaken. And “normalize” too — there’s a normalize tool of its own right, as I’ve mentioned here.

I see that Debian sidesteps that issue by prefixing the tools with “num”, which is probably a good idea. Archers might have to whack things into place, to keep everyone happy.

Other than that, it’s good to see numbers get the same love as their text counterparts. πŸ˜‰

nice: It’s nice to be nice to the nice again

You know nice. We all know nice. nice lets you dedicate more processor muscle to an application, and hopefully get things done faster.

2014-02-08-lv-r1fz6-nice

And that’s about it. A couple of things are worth mentioning, I suppose.

  1. nice takes a range of 19 to -20. Minus 20 is the highest priority, and 19 is the lowest.
  2. renice changes the value for a running process.
  3. You can adjust the niceness of an application with the -n flag.
  4. You can set lower and raise your own process between 19 and 0, but to push it higher (meaning, give it values less than 0), you have to have administrator privileges.

nice and renice are useful to a point, and that’s the last thing I wanted to note. It’s only going to do you good if the application is spending a lot of time on your processor.

If it’s writing to your hard drive or somehow bottlenecked by another component in your system, renice-ing it probably won’t make a difference.

In those cases, check ionice and other … nice … tools. πŸ˜‰