copy-pasting directories using the shell

Posted by: admin  :  Category: Debian GNU/Linux, FreeBSD, HowTo's, OS X, RHEL, Shells

Imagine that you need to copy over some files or directories to another host, but the security policy or the connectivity doesn’t allow to use standard file transfer protocols. Here’s a quick and dirty solution to overcome such restrictions.

This little trick involves the ability to access two different hosts via SSH (or telnet, or even a serial console) and a terminal client supporting to capture screen output into a file or a copy-paste buffer.

So, on the source host, go for the file/directory, and tar it up, apply some compression to it as needed, send the output to stdout and pipe it directly through base64:

tar -czpvf - some/path | base64

This will give you some output and the base64-encoded representation of the data, which may look like this:

some/path
some/path/somefile1
some/path/somefile2
some/path/somefile3
some/path/somedir
some/path/somedir/someotherfile1
H4sIAGnAGVgAA+3POw7CQAwFwD1KbkC8kfY+kUIJK+XD+VklDVBAQ7qZ5lnyK+zp+rhs83if6i2d
pW9KKXs2n7nPkVsjIuccqY8hSqRuOO2iF9uyjnPXpbnW9Vvv1/74I46M+O+RAAAAAAAAAAAA8O4J
FJs7gwAoAAA=

The “garbage” shown after the file and directory names is the base64 encoded contents from tar.
Don’t bother decoding the output above, this is just some gargabe from /dev/urandom to illustrate this example 😉

Now, copy-paste just the base64 output (or send it to a file, if your terminal client supports this).
Then, on the supposed-to-be target host, change to the directoy, where your files/directories should end up, then emter the command below:

cat|base64 -d|tar -xzpvf -

Don’t worry, it’ll “hang” on an empty line.
Now paste the buffer (or send the contents of the file captured before into the buffer).

This will looks similar to this:

cat|base64 -d|tar -xzpvf -
H4sIAGnAGVgAA+3POw7CQAwFwD1KbkC8kfY+kUIJK+XD+VklDVBAQ7qZ5lnyK+zp+rhs83if6i2d
pW9KKXs2n7nPkVsjIuccqY8hSqRuOO2iF9uyjnPXpbnW9Vvv1/74I46M+O+RAAAAAAAAAAAA8O4J
FJs7gwAoAAA=

As soon as the buffer is flushed, output will string “hang”, press CTRL-D to complete the transactions.

If done correctly, the input should be sent trough base64 to be decoded, and then passed on to tar to unpack.
You should see the file and directory names accordingly.

cat|base64 -d|tar -xzpvf -
H4sIAGnAGVgAA+3POw7CQAwFwD1KbkC8kfY+kUIJK+XD+VklDVBAQ7qZ5lnyK+zp+rhs83if6i2d
pW9KKXs2n7nPkVsjIuccqY8hSqRuOO2iF9uyjnPXpbnW9Vvv1/74I46M+O+RAAAAAAAAAAAA8O4J
FJs7gwAoAAA=
some/path
some/path/somefile1
some/path/somefile2
some/path/somefile3
some/path/somedir
some/path/somedir/someotherfile1

That’s it, a while directory tree copied without involging file transer protocols.

Of course, the base64 encoding adds some overhead, so this doesn’t work well for huge data loads as it’s limited to the console speed. However this is a very quick solution if only a few files need to be copied quickly without bothering about possible restrictions.

Ansible in 10 minutes or less

Posted by: admin  :  Category: Debian GNU/Linux, FreeBSD, HowTo's, Operating Systems, Scripting

I just remember a recent argument I had with someone about automation. It’s unbelievable, how many things are still done manually on a widespread scale, not leveraging the possibilities at all. Especially with so many frameworks available to help out, sticking to “the old way” ain’t just cool any more.

So let’s quickly look at Ansible, and how we can be up and running for even simple task automation in 10 minutes or less.
Read more…

armv6 Package Builder for FreeBSD is online

Posted by: admin  :  Category: FreeBSD, HowTo's, Operating Systems, Utilities

There it finally is, The Phunsites Package Builder at http://pkgbuild.phunsites.net/.

Since my initial writings on FreeBSD on Raspberry Pi, I’ve always wanted to have a webservice, where I can just select the port I want and it’ll be packaged up in minutes.
Now, there it is. Fully automated, with a neat and (hopefully) easy to use webinterface.

Check it out. It’s free lemons! 😉

pkgbuild2016

FreeBSD on the Raspberry Pi – Pt 1: Quick and Dirty Intro (also for Linux-Users)

Posted by: gdelmatto  :  Category: FreeBSD, HowTo's

Tonight I went along to fire up FreeBSD on a Raspberry Pi.
Although I’m in for Linux on a day-to-day basis, my heart truely belongs to the BSDish world.
So why not run FreeBSD on the Pi? Especially since I have an idea in mind to build a very specialized and compact access point for one of my ongoing projects.

A good starting point for this is at https://wiki.freebsd.org/FreeBSD/arm.
Read more…

Use Arduino Micro as ISP with ATmega on a breadboard

Posted by: gdelmatto  :  Category: Hacks, Hardware, HowTo's

Arduino is super cool, I really love that thing, but …

Who would really want to put a fully-featured Arduino board into each and every project, at least when they’re built to more like a permanent setting?
I guess nobody. Honestly, an Arduino is not expensive to buy, but to expensive, if you want to put dozens of them in place.

So, why not put just the ATmega chip, which drives every Arduino at its heart, inside and leave all the rest away?

This is what this post The RRRRRRRRRRBBA, a $3 Arduino talks about: Just provide power to the ATmega and get rid of all the bells and whistles. You can even spare the oscillator and the resistors.

Well, this is exactly what I wanted. But since I didn’t have the proper gear at hands to flash the ATmega chip, I first had to look into another topic, namely how to turn your existing Arduino into an ASP (in-system programmer).

Here’s some of the pages I came along:

Arduino ISP
Arduino to Breadboard
The Arduino Micro schematic
Arduino Leonardo as ISP
Arduino Micro as ISP and atTiny84A

But I soon found that doing so with an Arduino Micro, the only one I had available at that time, is not as straight forward as with other Arduino boards.
So I did this writeup to cover my findings on this.
Read more…

Raspberry Pi with Raspbmc: Getting around Onkyo and Philips woes

Posted by: gdelmatto  :  Category: Hardware, HowTo's

The culprit with technology is always the same: to old, to buggy, something missing here and there.
Adding something new usually urges you to add even more to get it running, so in the end, yeah, you know, why didn’t I stay with what I had before?

So to speak recently when I hooked a Raspberry Pi equipped with Raspbmc to my almost prehistoric Philips 42PF9966/10 plasma tv.
The reasing for doing so? Well, I use XBMC on a (jailbroken) Apple TV 2 for some time now and got fond of it.
Yet, for some obscure reason, the ATV2 only works properly when connected to my ultra cheap noname TV through HDMI.
With my Philips TV, the ATV2 would just refrain from working. All I’d get is a black screen and nothing else. Apple declares this being because of the connector type, because the Philips TV offers only one DVI digital input. Well, seems they didn’t know that HDMI was designed for backwards-compatibility with DVI, it uses the same protocols, but … yeah … you know … 😉

So, when I heard about upcoming XBMC supporting the Raspberry Pi, I gave it a try.
I was happy to see that it indeed worked on my Philips TV. But since I had only one digital input, which was taken up by my Bluray player, I had to put a new A/V receiver in between to get enough inputs.

So after hooking up everything to my new Onkyo receiver through HDMI, connecting the later with HDMI-to-DVI to the Philips TV, I soon found that the Raspi wouldn’t give video output. And neither would it give an audio signal. Still, the Sony BD player worked like a charm.

So I connected the Raspi directly to the Philips again, which finally gave a video signal, but only after rebooting the box. After reconnecting it to the receiver, I finally got a video singal through the receiver, but still no audio, and yet, it wouldn’t last after the next reboot.

Reading on some Raspberry Pi internal workings, I found this:

  1. The Raspberry Pi does always fall back to Composite output if no HDMI display is found during startup
  2. If the HDMI display does not support audio capability, audio output on HDMI is disabled

Well, that seemed at least to explain it.
To get around it, I had to to force the Raspberry Pi to always drive the HDMI output, regardless if a display was connected or not.

According to http://elinux.org/RPiconfig these are the settings, that need to be added to /boot/config.txt to achieve this.

hdmi_force_hotplug=1
hdmi_drive=2
hdmi_group=1

However I still had some issues getting video output to work properly. Especially, I couldn’t get stable 720p output, so I had to force it to 1080i. These symptoms went away using these settings:

config_hdmi_boost=4
hdmi_mode=5

Still I had no audio. To get this working, I had do force audio output via HDMI. Additionally, I really had to enable the “ignore edid” data mode, although I wouldn’t expect my Philips TV being a “crappy Chinese one” as stated in the docs 😉

hdmi_ignore_edid=0xa5000080
hdmi_force_edid_audio=1

After all, everything works fine now, so no reason to buy a new TV 🙂

Nagios/Icinga Plugin to check for DokuWiki Updates (v2,2015-09-05)

Posted by: gdelmatto  :  Category: HowTo's, Perl, Programming, Scripting

Nagios/Icinga can also serve to send you friendly reminders, like for example that you need to perform software updates.

Here’s my little contribution, a simple plugin to monitor a given DokuWiki site and check against the release server for any upgrades.

Just fetch the  check_dokuwiki-0.1 check_dokuwiki-0.2 tarball and extract the check_dokuwiki script to your Nagios/Icinga plugin directory.

The latest plugin update (0.2, 2015-09-06) catches up with latest updates on Dokuwikis web page, and now incorporates upstream checks using the same mechanism that Dokuwiki uses internally. Also it adds some (although not yet well tested) support for http authentication.

Please check out the cli help for more details on specific arguments:

check_dokuwiki v (nagios-plugins 1.4.15)
The nagios plugins come with ABSOLUTELY NO WARRANTY. You may redistribute
copies of the plugins under the terms of the GNU General Public License.
For more information about these matters, see the file named COPYING.
Copyright (c) 2012 Gianpaolo Del Matto

Usage:
  check_dokuwiki -H 
  check_dokuwiki [-h | --help]
  check_dokuwiki [-V | --version]

more arguments
  -a | --use-auth              enable http authentication mode
  -r name | --realm=name       use given realm with http auth
  -u user | --username=user    use given username for http auth
  -p pass | --password=pass    use given password for http auth
  -s | --use-ssl               enable SSL mode (uses TCP:443 as default, see --tcp-port
  -p num | --tcp-port num      use non-standard port
                               if not given, defaults to TCP:80 (or TCP:443 if --use-ssl is used)

   	give any valid DokuWiki hostname to fetch the 'VERSION' file from.
                         Note: HTTP AUTH is currently not supported.

Send email to nagios-users@lists.sourceforge.net if you have questions
regarding use of this software. To submit patches or suggest improvements,
send email to nagiosplug-devel@lists.sourceforge.net.
Please include version information with all correspondence (when possible,
use output from the --version option of the plugin itself).

Register the plugin with a command definition like this:

# 'check_dokuwiki' command definition
define command{
        command_name	check_dokuwiki
        command_line	/usr/local/libexec/nagios/check_dokuwiki -H $HOSTNAME
	}

Then simply add a service to one or more of your DokuWiki hosts (or hostgroups, whatever you prefer).

define service{
        use                     generic-service
        host_name               your_wiki_host_objects_list_here
        service_description     dokuwiki_version
        check_command           check_dokuwiki
        max_check_attempts      5
        check_interval          5
        retry_interval          3
        check_period            24x7
        notification_interval   0
        notification_period     24x7
        notification_options    w,c,r
        }

Restart Nagios/Icinga and you’re done.

Happy monitoring 🙂

Swisscom FTTH: 6RD mit Cisco 892F nutzen

Posted by: admin  :  Category: Cisco, HowTo's, Networking

Wie bereits aufgezeigt, kann man statt dem Centro Grande oder Centro Piccolo auch problemlos einen Cisco 892F an einem Swisscom Vivo FTTH-Anschluss betreiben.

Hat man diese Hürde erstmal geschafft, möchte man den Cisco vielleicht am Swisscom-eigenen IPv6 6RD Gateway anbinden um in den Genuss von IPv6 zu kommen.

! Wichtige Information !

Die IP-Adresse des Swisscom 6RD Relays ändert per 9. April 2013:

193.5.29.1

Read more…

Fix iTunes refraining from syncing the iPhone

Posted by: gdelmatto  :  Category: Bits and Bytes, Hardware, HowTo's, OS X

Out of a sudden iTunes struck me with this error: The iPhone “…” could not be synced because the sync session failed to start

I did web research, but did not really find a proper solution to this.
Well, there were a few which either did some funny things on Windows (I run a Mac ….), or recommended to restore the iPhone from a previous backup, removing just one app, installing just one new app through the iTunes store, and, well, some other curious and strange things.

After all, I thought to give the restore thing a try, however that changed nothing. Same message came up. So I disabled WLAN sync as to not interfere with the iPhone being hooked up by the cable already — again to no avail.

So I decided to look into the iTunes internals. There I found a primising folder within the Application Library folder called SyncServices. It was not that big, around 6 megs. But inside there where clear traces of synchronisation stuff.

My solution to the above error was then to quit iTunes, remove the SyncServices folders completely. And voila, iTunes would just start syncing my iPhone as if nothing bad had ever happened.

Interface-Statistiken in Cacti für Pirelli Router erstellen

Posted by: gdelmatto  :  Category: HowTo's, Networking, Programming

In diesem Beitrag geht es darum, Cacti Interface-Statistiken auf einem Pirelli Router über den Umweg via SSH zu erzeugen. Dies wird auf bestimmten, kastrierten Pirelli-Geräten, wie sie von bestimmten Providern ihren Kunden zur Verfügung gestellt werden, auch dringend benötigt, da die Geräte den nötigen SNMP-Support schmerzlich vermissen lassen. Das fehlende SNMP Support lässt sich auf diesen Geräten auch mit Hilfe eines Hacks, der bereits seit mehreren Jahren in einschlägigen Foren kursiert, nicht nachrüsten.

Read more…