copy-pasting directories using the shell

Posted by: admin  :  Category: Debian GNU/Linux, FreeBSD, HowTo's, OS X, RHEL, Shells

Imagine that you need to copy over some files or directories to another host, but the security policy or the connectivity doesn’t allow to use standard file transfer protocols. Here’s a quick and dirty solution to overcome such restrictions.

This little trick involves the ability to access two different hosts via SSH (or telnet, or even a serial console) and a terminal client supporting to capture screen output into a file or a copy-paste buffer.

So, on the source host, go for the file/directory, and tar it up, apply some compression to it as needed, send the output to stdout and pipe it directly through base64:

tar -czpvf - some/path | base64

This will give you some output and the base64-encoded representation of the data, which may look like this:

some/path
some/path/somefile1
some/path/somefile2
some/path/somefile3
some/path/somedir
some/path/somedir/someotherfile1
H4sIAGnAGVgAA+3POw7CQAwFwD1KbkC8kfY+kUIJK+XD+VklDVBAQ7qZ5lnyK+zp+rhs83if6i2d
pW9KKXs2n7nPkVsjIuccqY8hSqRuOO2iF9uyjnPXpbnW9Vvv1/74I46M+O+RAAAAAAAAAAAA8O4J
FJs7gwAoAAA=

The “garbage” shown after the file and directory names is the base64 encoded contents from tar.
Don’t bother decoding the output above, this is just some gargabe from /dev/urandom to illustrate this example 😉

Now, copy-paste just the base64 output (or send it to a file, if your terminal client supports this).
Then, on the supposed-to-be target host, change to the directoy, where your files/directories should end up, then emter the command below:

cat|base64 -d|tar -xzpvf -

Don’t worry, it’ll “hang” on an empty line.
Now paste the buffer (or send the contents of the file captured before into the buffer).

This will looks similar to this:

cat|base64 -d|tar -xzpvf -
H4sIAGnAGVgAA+3POw7CQAwFwD1KbkC8kfY+kUIJK+XD+VklDVBAQ7qZ5lnyK+zp+rhs83if6i2d
pW9KKXs2n7nPkVsjIuccqY8hSqRuOO2iF9uyjnPXpbnW9Vvv1/74I46M+O+RAAAAAAAAAAAA8O4J
FJs7gwAoAAA=

As soon as the buffer is flushed, output will string “hang”, press CTRL-D to complete the transactions.

If done correctly, the input should be sent trough base64 to be decoded, and then passed on to tar to unpack.
You should see the file and directory names accordingly.

cat|base64 -d|tar -xzpvf -
H4sIAGnAGVgAA+3POw7CQAwFwD1KbkC8kfY+kUIJK+XD+VklDVBAQ7qZ5lnyK+zp+rhs83if6i2d
pW9KKXs2n7nPkVsjIuccqY8hSqRuOO2iF9uyjnPXpbnW9Vvv1/74I46M+O+RAAAAAAAAAAAA8O4J
FJs7gwAoAAA=
some/path
some/path/somefile1
some/path/somefile2
some/path/somefile3
some/path/somedir
some/path/somedir/someotherfile1

That’s it, a while directory tree copied without involging file transer protocols.

Of course, the base64 encoding adds some overhead, so this doesn’t work well for huge data loads as it’s limited to the console speed. However this is a very quick solution if only a few files need to be copied quickly without bothering about possible restrictions.

Ansible in 10 minutes or less

Posted by: admin  :  Category: Debian GNU/Linux, FreeBSD, HowTo's, Operating Systems, Scripting

I just remember a recent argument I had with someone about automation. It’s unbelievable, how many things are still done manually on a widespread scale, not leveraging the possibilities at all. Especially with so many frameworks available to help out, sticking to “the old way” ain’t just cool any more.

So let’s quickly look at Ansible, and how we can be up and running for even simple task automation in 10 minutes or less.
Read more…

Transform Cobalt Raq3 into a Raspberry Pi-powered Media Center

Posted by: gdelmatto  :  Category: Debian GNU/Linux, Hacks, Hardware, Operating Systems, Programming, Scripting

Anyone remember these adorable blueish 1U servers made by Cobalt Networks?

ppcobaltraq

While I was never in true love with the Cobalt OS itself, I actually liked the Cobalts Raq enclosure.
So much that I salvaged one while cleaning out a data center last summer. I decided to grant it a second live as a media center box running OSMC.
And of course it’s powered by a Respberry Pi. Nowadays there’s simply no way around those nice little boxes 😉
Read more…

Mount a dd Disk Image with Partition Table inside

Posted by: gdelmatto  :  Category: Debian GNU/Linux, Operating Systems, RHEL

After making a backup from a hard disk ta a disk image using plain old ‘dd’, I was just looking into mounting it using the Linux loopback device.

If you ‘dd’ a single partition into an image file, then this is very straight forward. But if your image file contains multiple partition partitions including the partition table itself, then you need to take additional steps.
Read more…

Debian Netinst Installer ignores network and locale preseed.cfg settings upon PXE/TFTP boot

Posted by: admin  :  Category: Debian GNU/Linux, Operating Systems

As you came along I suppose you stubled accross the same issue as me, didn’t you?

As far as I’m concerned, I can only tell that this issue rises when netbooting Debian-Installer through PXE/TFTP in combination with a preseed.cfg file, while the latter one being downloaded from a HTTP web-server.
This bevahiour has been observed with Debian Lenny and older versions and can cause quiet some headache upon setup automation if not taken into account.
Read more…

Debian GNU/kFreeBSD inside native FreeBSD jail

Posted by: admin  :  Category: Debian GNU/Linux, FreeBSD, jails

It has been some time now since development on Debian GNU/kFreeBSD started, which aims at bringing together the FreeBSD kernel with a GNU userland.

There exists a similar implementation called Gentoo GNU/kFreeBSD, although I had no time yet to review it.
Read more…

Installing Debian on a SGI Indy

Posted by: admin  :  Category: Debian GNU/Linux

After I had finally found a solution to my 13W3-VGA adapter issue the time is ready to install Debian onto the machine.
Read more…

Setting e1000 Interface Settings Permanently On Debian

Posted by: admin  :  Category: Debian GNU/Linux

Well, sometimes obvious things aren’t quiet as obvious as supposed.

After plugging in a dual Intel Pro 1000 into my Debian box and hooking it up to my ancient Cisco 1924 switch, I noticed this in the kernel logs:

Jul 31 09:10:35 localhost kernel: e1000: eth0: e1000_watchdog: NIC Link is Up 100 Mbps Half Duplex

I though it should be fairly easy set the interface speeds properly so the devices would actually talk to each other.
Read more…