copy-pasting directories using the shell

Posted by: admin  :  Category: Debian GNU/Linux, FreeBSD, HowTo's, OS X, RHEL, Shells

Imagine that you need to copy over some files or directories to another host, but the security policy or the connectivity doesn’t allow to use standard file transfer protocols. Here’s a quick and dirty solution to overcome such restrictions.

This little trick involves the ability to access two different hosts via SSH (or telnet, or even a serial console) and a terminal client supporting to capture screen output into a file or a copy-paste buffer.

So, on the source host, go for the file/directory, and tar it up, apply some compression to it as needed, send the output to stdout and pipe it directly through base64:

tar -czpvf - some/path | base64

This will give you some output and the base64-encoded representation of the data, which may look like this:

some/path
some/path/somefile1
some/path/somefile2
some/path/somefile3
some/path/somedir
some/path/somedir/someotherfile1
H4sIAGnAGVgAA+3POw7CQAwFwD1KbkC8kfY+kUIJK+XD+VklDVBAQ7qZ5lnyK+zp+rhs83if6i2d
pW9KKXs2n7nPkVsjIuccqY8hSqRuOO2iF9uyjnPXpbnW9Vvv1/74I46M+O+RAAAAAAAAAAAA8O4J
FJs7gwAoAAA=

The “garbage” shown after the file and directory names is the base64 encoded contents from tar.
Don’t bother decoding the output above, this is just some gargabe from /dev/urandom to illustrate this example 😉

Now, copy-paste just the base64 output (or send it to a file, if your terminal client supports this).
Then, on the supposed-to-be target host, change to the directoy, where your files/directories should end up, then emter the command below:

cat|base64 -d|tar -xzpvf -

Don’t worry, it’ll “hang” on an empty line.
Now paste the buffer (or send the contents of the file captured before into the buffer).

This will looks similar to this:

cat|base64 -d|tar -xzpvf -
H4sIAGnAGVgAA+3POw7CQAwFwD1KbkC8kfY+kUIJK+XD+VklDVBAQ7qZ5lnyK+zp+rhs83if6i2d
pW9KKXs2n7nPkVsjIuccqY8hSqRuOO2iF9uyjnPXpbnW9Vvv1/74I46M+O+RAAAAAAAAAAAA8O4J
FJs7gwAoAAA=

As soon as the buffer is flushed, output will string “hang”, press CTRL-D to complete the transactions.

If done correctly, the input should be sent trough base64 to be decoded, and then passed on to tar to unpack.
You should see the file and directory names accordingly.

cat|base64 -d|tar -xzpvf -
H4sIAGnAGVgAA+3POw7CQAwFwD1KbkC8kfY+kUIJK+XD+VklDVBAQ7qZ5lnyK+zp+rhs83if6i2d
pW9KKXs2n7nPkVsjIuccqY8hSqRuOO2iF9uyjnPXpbnW9Vvv1/74I46M+O+RAAAAAAAAAAAA8O4J
FJs7gwAoAAA=
some/path
some/path/somefile1
some/path/somefile2
some/path/somefile3
some/path/somedir
some/path/somedir/someotherfile1

That’s it, a while directory tree copied without involging file transer protocols.

Of course, the base64 encoding adds some overhead, so this doesn’t work well for huge data loads as it’s limited to the console speed. However this is a very quick solution if only a few files need to be copied quickly without bothering about possible restrictions.

quick-and-dirty PAM with LUA, mod_magnet and lighttpd -or- how to breach system security

Posted by: admin  :  Category: Programming, RegExp, Security, Shells

Be warned: This example serves as an illustration on how to *NOT* do it.
It’s just one of my examples I teach to apprentices at the office when it comes to learning scrips, and how important data input validation (or the absence of the same) is.
It’s also a good illustration on how attackers may break into systems to steal data or make them part of a botnet.

The given situation depicts a lighttpd server, which exposes a directory which must be protected via LDAP-managed accounts, so there is an immediate need for PAM. However, lighttpd lacks a PAM implementation. Period. There’s a very ugly and highly insecure way however …

Read more…

A backup volume switcher for Apple’s TimeMachine

Posted by: gdelmatto  :  Category: Operating Systems, OS X, Programming, Shells

So here’s another piece of code I hacked up tonight.
Since I’m roaming around with my MacBook every now and then, the need arised, that I would need to switch my TimeMachine destination volumes based on location.

So while in the office, I’d like to backup to my external USB drive there.
Being at my home office, i’d like to backup to my NAS, while on the road, I’d love to habe my external mobile drive to kick in (and yes, I know about the “mobile backup feature” of OS X Lion, but that’s not the point …)
Read more…

Trap Errors, Exit Codes and Line Numbers within a Bash script (and some output redirection, too)

Posted by: admin  :  Category: Programming, Scripting, Shells

A discussion today was about error handling in shell scripts, Bash in particular.
Well, we all know about the usual knitpicks about error handling and the possible consequences in not doing so properly 😉
Read more…

wipe out directory tree through rsync

Posted by: admin  :  Category: Bits and Bytes, Shells

To wipe out a directory tree on a rsync server (an export) which you don’t have shell access to (but proper write access of course), just use this command:

# rsync -av –delete /tmp/clear.me/ rsync://some.rsync.host/some.export/

/tmp/clear.me is an example for a local but empty directory (empty as in ‘mkdir /tmp/clear.me).

Putting the slashes at the end of both local and remote path will allow you to sync the empty directory onto the server, effectively removing everything on the receiver side.

Find And Replace Strings

Posted by: admin  :  Category: Perl, Shells

Imagine you have a directory with thousands of text files, each of which containing some keywords to be replaced.

While there’s a lot of tools out there allowing you to find and replace strings in text files, there’s always a goog reason to use the tools that you already have.

The following snippet uses find in combination with perl to achieve the task:
Read more…