Tag Archives: storage

bitpocket: Your in-house drop box

I have three i686 machines at the moment, all three of which are running Arch. One acts as a wireless relay to the other two (thanks, create_ap), and shares its wired connection with the two others.

It also is set up to sync itself against the Arch repositories once a day, and then I manually rsync between the other two and share the downloaded updates. This saves me the drag of triple-updating machines, since my in-house wireless connection is considerably faster than the wired line.

Anything leftover or specific to one piece of hardware — like video drivers — gets downloaded normally, when a specific machine does its update.

I do end up doing some double-back synchronization, from satellite computers to the central hub. That’s more of an insurance measure or to make backups of individual packages that were downloaded to specific machines.

All that rsyncing back and forth relies on sshd of course, and it gets a little tedious having to re-enter passwords on every rsync, but I don’t know if there’s anything to be done about that. It’s the nature of the beast.

I thought perhaps working a little bit with bitpocket might give me some ideas, since bitpocket is intended as a do-it-yourself Dropbox-style network storage tool, built upon the mighty rsync.

2014-11-07-2sjx281-bitpocket

It did and it didn’t. bitpocket runs into much the same issue as I had, where I needed to supply a password four times — actually five — to get the proper access across the network and update the master folder on the central machine.

I don’t hold that out as a fault of bitpocket though, since whatever setting or configuration I’m after to solve my own problem isn’t bitpocket’s responsibility. I shall pursue that independently.

bitpocket itself is rather useful, and very easy to set up. Supply a proper folder and identity for your server machine, create a folder for a corresponding copy on the client, and just call bitpocket from that folder. bitpocket can check for updates, synchronize new files, delete old ones and generally handle everything as it should be done.

bitpocket has some other features that are nifty, such as the ability to tell you what will be updated (in other words, what changes exist since the last sync) and delete protection, where it moves “deleted” files into a hidden folder, in case you make a mistake.

If you already do fairly regular rsyncs against a master machine, or if you just want to streamline the process of keeping work folders up to date, I can see where bitpocket might improve upon a long chain of rsync commands.

In my case, I really need to find out how to authenticate rsync without being prompted each time, and follow through with that. (Edit: I figured it out, thanks. I just needed to generate an id_rsa.pub key, and move it over to the server. 😉 )

rdiffdir: A succinct sync

Forgive me if I jump slightly out of order. I wanted to work with rdiffdir today, and I promise to touch on rdiff-backup tomorrow.

Also please forgive me if I don’t have screenshots this time. I think I can adequately explain what’s happening, and rdiffdir isn’t particularly wordy.

I have practical experience with it, albeit a few years out of date. At a time when I quit lugging an ancient laptop back and forth to work to listen to music, rdiffdir made it easy to synchronize my main music archive at home with the remote one at work … without a network connection.

“What witchcraft is this?!” you might howl. I’ll give you the command sequence, and you work out what’s happening. Office machine first:

rdiffdir signature music/ music.signature

Then at home:

rdiffdir delta music.signature music/ music.delta

Carry that back to the office, and …

rdiffdir patch music/ music.delta

And that’s it (or at least what I remember of it). The signature command creates a distinct impression of what’s available on the office machine. The delta creates a file packed with changed material from the home machine, and the patch command merges it with the destination at the office again.

It’s very clever, really. What you avoid is rsyncing entire folders to USB drives, then USB drives to destination folders — hopefully saving time, and space on your intermediary drive.

I could see where this would also be useful for completely offline backups, where you want to preserve file arrangements and integrity on one machine with another that is completely disconnected. Which, in this day and age, isn’t a bad idea. 😯

rdiffdir is part of duplicity, which is available in Debian and Arch. Tomorrow, rdiffdir’s ugly kid brother. 😉

By the way, I should mention that everything I know about rdiffdir I learned years ago from this page. Credit where credit is due. 😀