copy-pasting directories using the shell

Posted by: admin  :  Category: Debian GNU/Linux, FreeBSD, HowTo's, OS X, RHEL, Shells

Imagine that you need to copy over some files or directories to another host, but the security policy or the connectivity doesn’t allow to use standard file transfer protocols. Here’s a quick and dirty solution to overcome such restrictions.

This little trick involves the ability to access two different hosts via SSH (or telnet, or even a serial console) and a terminal client supporting to capture screen output into a file or a copy-paste buffer.

So, on the source host, go for the file/directory, and tar it up, apply some compression to it as needed, send the output to stdout and pipe it directly through base64:

tar -czpvf - some/path | base64

This will give you some output and the base64-encoded representation of the data, which may look like this:

some/path
some/path/somefile1
some/path/somefile2
some/path/somefile3
some/path/somedir
some/path/somedir/someotherfile1
H4sIAGnAGVgAA+3POw7CQAwFwD1KbkC8kfY+kUIJK+XD+VklDVBAQ7qZ5lnyK+zp+rhs83if6i2d
pW9KKXs2n7nPkVsjIuccqY8hSqRuOO2iF9uyjnPXpbnW9Vvv1/74I46M+O+RAAAAAAAAAAAA8O4J
FJs7gwAoAAA=

The “garbage” shown after the file and directory names is the base64 encoded contents from tar.
Don’t bother decoding the output above, this is just some gargabe from /dev/urandom to illustrate this example 😉

Now, copy-paste just the base64 output (or send it to a file, if your terminal client supports this).
Then, on the supposed-to-be target host, change to the directoy, where your files/directories should end up, then emter the command below:

cat|base64 -d|tar -xzpvf -

Don’t worry, it’ll “hang” on an empty line.
Now paste the buffer (or send the contents of the file captured before into the buffer).

This will looks similar to this:

cat|base64 -d|tar -xzpvf -
H4sIAGnAGVgAA+3POw7CQAwFwD1KbkC8kfY+kUIJK+XD+VklDVBAQ7qZ5lnyK+zp+rhs83if6i2d
pW9KKXs2n7nPkVsjIuccqY8hSqRuOO2iF9uyjnPXpbnW9Vvv1/74I46M+O+RAAAAAAAAAAAA8O4J
FJs7gwAoAAA=

As soon as the buffer is flushed, output will string “hang”, press CTRL-D to complete the transactions.

If done correctly, the input should be sent trough base64 to be decoded, and then passed on to tar to unpack.
You should see the file and directory names accordingly.

cat|base64 -d|tar -xzpvf -
H4sIAGnAGVgAA+3POw7CQAwFwD1KbkC8kfY+kUIJK+XD+VklDVBAQ7qZ5lnyK+zp+rhs83if6i2d
pW9KKXs2n7nPkVsjIuccqY8hSqRuOO2iF9uyjnPXpbnW9Vvv1/74I46M+O+RAAAAAAAAAAAA8O4J
FJs7gwAoAAA=
some/path
some/path/somefile1
some/path/somefile2
some/path/somefile3
some/path/somedir
some/path/somedir/someotherfile1

That’s it, a while directory tree copied without involging file transer protocols.

Of course, the base64 encoding adds some overhead, so this doesn’t work well for huge data loads as it’s limited to the console speed. However this is a very quick solution if only a few files need to be copied quickly without bothering about possible restrictions.

Ansible in 10 minutes or less

Posted by: admin  :  Category: Debian GNU/Linux, FreeBSD, HowTo's, Operating Systems, Scripting

I just remember a recent argument I had with someone about automation. It’s unbelievable, how many things are still done manually on a widespread scale, not leveraging the possibilities at all. Especially with so many frameworks available to help out, sticking to “the old way” ain’t just cool any more.

So let’s quickly look at Ansible, and how we can be up and running for even simple task automation in 10 minutes or less.
Read more…

armv6 Package Builder for FreeBSD is online

Posted by: admin  :  Category: FreeBSD, HowTo's, Operating Systems, Utilities

There it finally is, The Phunsites Package Builder at http://pkgbuild.phunsites.net/.

Since my initial writings on FreeBSD on Raspberry Pi, I’ve always wanted to have a webservice, where I can just select the port I want and it’ll be packaged up in minutes.
Now, there it is. Fully automated, with a neat and (hopefully) easy to use webinterface.

Check it out. It’s free lemons! 😉

pkgbuild2016

Quick&Dirty FreeBSD on Alix (without PXE boot)

Posted by: gdelmatto  :  Category: FreeBSD, Hardware

It’s been a while since my last post and I’ve been quiet busy writing on my graduation essay.

Meanwhilst my colleague Steven donated me a somewhat dated PC Engines Alix computer. I though to put it to some good use as  a packet generator for my new network playground I’m currently building up.

Funny anectode: While googl’ing around on some docs about Alix computers, I stumled accross his 2009 original post on FreeBSD installs.

But then I read that I’d need to go through config hell for DHCP, PXE, NFS for a one-time install … Oh boy, must be kidding …

Read more…

FreeBSD on ARMv6: Cross-Compile Performance Optimization for Poudriere

Posted by: gdelmatto  :  Category: FreeBSD, Operating Systems, Programming

Important Announcements on FreeBSD-armv6 packages

While initially writing this article, I had the idea to establish a service where packages can be selected to build for armv6. As of February 2016 this service is now online.
If you just need current FreeBSD packages for armv6, this is the place to visit. Otherwise, keep on reading.

Whilst playing around with FreeBSD on Raspberry Pi, I started to dig into cross-compiling packages.

Well, if you follow the first tutorial you’ll surely notice that there is no real speed-gain, because the use of full binary emulation on a x86 host through QEMU. So this is almost as slow as if packages were natively compiled on the Raspberry Pi itself even if done on a multi-cpu Xeon powerhouse.

So let’s see how to get an actually performance gain.

Read more…

FreeBSD on the Raspberry Pi – Pt 1: Quick and Dirty Intro (also for Linux-Users)

Posted by: gdelmatto  :  Category: FreeBSD, HowTo's

Tonight I went along to fire up FreeBSD on a Raspberry Pi.
Although I’m in for Linux on a day-to-day basis, my heart truely belongs to the BSDish world.
So why not run FreeBSD on the Pi? Especially since I have an idea in mind to build a very specialized and compact access point for one of my ongoing projects.

A good starting point for this is at https://wiki.freebsd.org/FreeBSD/arm.
Read more…

Inofficial FreeBSD port for Zend Optimizer Plus

Posted by: gdelmatto  :  Category: FreeBSD, PHP

2017-03-12: A US-based company claims that their trademark is infringed by mentioning the terms “(Zend) Optimizer+ / (Zend) Optimizer Plus” on this page. In context of my writings, the use of these terms are clearly related to the well-known open source software by Rogue Wave Software (formerly Zend Technologies), as released in 2013.

While the contents of this page has become somewhat obsolete, as Zend Optimizer+/Opcache has since been integrated into the PHP programming language itself, the content is left here for reference purposes.

Again: I make it absolutely clear that this content exists as a documentation only in the scope if IT terminology and  a well-known open source software product Zend Optimizer (former trademark).

The use of the term specifically relates to the original product name, including file name references. As a reference, the original commit to the source is linked here.

“Zend Optimizer+” has been rebranded to “Zend Opcache” in the meantime.


While just in the process of doing web-server freshup on FreeBSD, I was caught by the good news that Zend Technologies have released their Zend opcode caching engine as open source.

Now it’s called Zend Optimizer+ and hosted over there at github.

As far as I have seen, it did not yet popup as a buildable port on FreeBSD’s ports tree, but that can only take little time for today.
So I quickly made up my own port which you can download here.

To build it, simply download and extract the file to /usr/ports/devel:

cd /usr/ports/devel
fetch -o- http://phaq.phunsites.net/files/2013/02/ZendOptimizerPlus.tgz | tar -xzpvf -

Then “make install” as usual:

cd /usr/ports/devel/ZendOptimizerPlus
make install

Afterwards, running “php -i” (or phpinfo from a web-accessible script file) should denote it runs “with Zend Optimizer+ 7.0.0-dev”.

Done 🙂

Install Icinga with MySQL and IDOUtils on FreeBSD

Posted by: gdelmatto  :  Category: FreeBSD, Networking, Operating Systems, Utilities

It’s not hard to install Icinga on FreeBSD, at least if you’re satisfied with default options.
Should you however require Icinga with IDOUtils and MySQL support, then you need to take additional steps.

Read more…

Fixing “Cannot find autoconf” error while building PHP extension from a FreeBSD port

Posted by: gdelmatto  :  Category: FreeBSD, Operating Systems, PHP

Today I had to add the PHP ‘bcmath’ extension to my FreeBSD host.
Actually, that’s a fairly easy one-liner:


[root@localhost ~]# cd /usr/ports/math/php5-bcmath && make clean install

However, I ended up with a crude error message:

Cannot find autoconf. Please check your autoconf installation and the
$PHP_AUTOCONF environment variable. Then, rerun this script.

Bummer! So let’s see how to fix this.
Read more…

World IPv6 Day and how to connect FreeBSD through a Teredo Tunnel

Posted by: gdelmatto  :  Category: FreeBSD, IPv6, Networking

It has finally arrived, the World IPv6 Day!

So you don’t have native IPv6 network access? No problem, just use a Teredo Tunnel to get connected.
Here’s a very short primer on doing so on FreeBSD.
Read more…