copy-pasting directories using the shell

Posted by: admin  :  Category: Debian GNU/Linux, FreeBSD, HowTo's, OS X, RHEL, Shells

Imagine that you need to copy over some files or directories to another host, but the security policy or the connectivity doesn’t allow to use standard file transfer protocols. Here’s a quick and dirty solution to overcome such restrictions.

This little trick involves the ability to access two different hosts via SSH (or telnet, or even a serial console) and a terminal client supporting to capture screen output into a file or a copy-paste buffer.

So, on the source host, go for the file/directory, and tar it up, apply some compression to it as needed, send the output to stdout and pipe it directly through base64:

tar -czpvf - some/path | base64

This will give you some output and the base64-encoded representation of the data, which may look like this:

some/path
some/path/somefile1
some/path/somefile2
some/path/somefile3
some/path/somedir
some/path/somedir/someotherfile1
H4sIAGnAGVgAA+3POw7CQAwFwD1KbkC8kfY+kUIJK+XD+VklDVBAQ7qZ5lnyK+zp+rhs83if6i2d
pW9KKXs2n7nPkVsjIuccqY8hSqRuOO2iF9uyjnPXpbnW9Vvv1/74I46M+O+RAAAAAAAAAAAA8O4J
FJs7gwAoAAA=

The “garbage” shown after the file and directory names is the base64 encoded contents from tar.
Don’t bother decoding the output above, this is just some gargabe from /dev/urandom to illustrate this example 😉

Now, copy-paste just the base64 output (or send it to a file, if your terminal client supports this).
Then, on the supposed-to-be target host, change to the directoy, where your files/directories should end up, then emter the command below:

cat|base64 -d|tar -xzpvf -

Don’t worry, it’ll “hang” on an empty line.
Now paste the buffer (or send the contents of the file captured before into the buffer).

This will looks similar to this:

cat|base64 -d|tar -xzpvf -
H4sIAGnAGVgAA+3POw7CQAwFwD1KbkC8kfY+kUIJK+XD+VklDVBAQ7qZ5lnyK+zp+rhs83if6i2d
pW9KKXs2n7nPkVsjIuccqY8hSqRuOO2iF9uyjnPXpbnW9Vvv1/74I46M+O+RAAAAAAAAAAAA8O4J
FJs7gwAoAAA=

As soon as the buffer is flushed, output will string “hang”, press CTRL-D to complete the transactions.

If done correctly, the input should be sent trough base64 to be decoded, and then passed on to tar to unpack.
You should see the file and directory names accordingly.

cat|base64 -d|tar -xzpvf -
H4sIAGnAGVgAA+3POw7CQAwFwD1KbkC8kfY+kUIJK+XD+VklDVBAQ7qZ5lnyK+zp+rhs83if6i2d
pW9KKXs2n7nPkVsjIuccqY8hSqRuOO2iF9uyjnPXpbnW9Vvv1/74I46M+O+RAAAAAAAAAAAA8O4J
FJs7gwAoAAA=
some/path
some/path/somefile1
some/path/somefile2
some/path/somefile3
some/path/somedir
some/path/somedir/someotherfile1

That’s it, a while directory tree copied without involging file transer protocols.

Of course, the base64 encoding adds some overhead, so this doesn’t work well for huge data loads as it’s limited to the console speed. However this is a very quick solution if only a few files need to be copied quickly without bothering about possible restrictions.

([M|m]ac)?\s?OS(\sX)?: Permit ICMP redirects

Posted by: admin  :  Category: Networking, Operating Systems, OS X

So I was fighting around with that Motorola/Netopia router I’m obliged to use, because the network operator doesn’t allow hooking up a custom device.

Well, it is possible after all, as proven in the past, however, in order to use the SIP gateway of network operator (whereas the login credentials are not provided), the operator-branded router must be used. *sigh*

Here’s a somewhat high-level overview: The clients, which shall connect to the lab, are in the same subnet as the default router #1. The destination for more specific lab routes is router #2, which is in the same subnet.

+-----------+        +------------+       +-----------+       +-----------+       +-----------+
| clients   |  ----  | WiFi/Wired | ----  | router #1 | ----  | router #2 | ----  | LAB stuff |
+-----------+        +------------+       +-----------+       +-----------+       +-----------+
{                            CLIENT SUBNET                          } {      LAB SUBNETS      }

So actually, I could just add the more specific routes to any client, indicating it shall forward through router #2.
However, this is cumbersome. I wond’t want to add these routes on every client.

So I tried hacking them into the Motorola/Netopia router. I had my hard time with that, but only because it’s so silly on overly complicated … 🙁
So far so good, my clients could send ICMP echo requests towards the LAB devices, however, that was as close as I could get.

Not every client was capable in accessing everything in the LAB.

As it turned out, the Motorola/Netopia sends ICMP redirects. It does that because router #2 (a cisco, btw) is reachable via the CLIENT subnet and thus directly reachable by anyone in the same subnet.
However, ICMP redirects are somewhat non-deterministic, as the forwarding is not influenced by the router anymore. I consider it vodoo, which is why I prefer turning it off.

The only problem is that this “Netopia SOC OS” doesn’t have an equivalent to a Cisco-type “no ip redirects”-command.
Well, it’s a Linux after all, so I could turn it off by setting /proc/sys/net/ipv4/conf/*/send_redirects to 0. There is an obscure way to break out from the SOC OS shell and get a unrestricted shell:

ping 127.0.0.2;/bin/busybox telnetd -l/bin/sh -p9999

This would open a root shell on port 9999, from where the kernel setting could be changed. However, since this will get reverted whenever the router reboots due to operator updates, I would need to hack this back in. I don’t like this at all. Please, let me officially retrieve the SIP credentials to hook up my IP phone directly, so I can use a Cisco router. Pretty please!

Well, one day perhaps. Until then, I need to get it working with the least intrusive means of configuration.
So, I can’t replace the router, I can’t learn it to not send redirects.
But, if my clients, ([M|m]ac)?\s?OS(\sX)? in particular, don’t play well with ICMP redirects, let’s force them to do so.

On macOS (man, let’s blatandly change the name one more time!), this can be done via the sysctl command in the Terminal.
Query it like this:

# sudo sysctl net.inet.icmp.drop_redirect
net.inet.icmp.drop_redirect: 1

So macOS indeed drops ICMP redirects by default. Let’s change this:

# sudo sysctl net.inet.icmp.drop_redirect=0
net.inet.icmp.drop_redirect: 1 -> 0

With the new setting, connections started to work right away.

A note of caution: In my opinion ICMP redirects can be a dangerous thing, as they open the door for an attacker to influence the client’s idea of the routing table.
A client should not listen to redirects and always forward traffic towards it’s designated router. Overriding a default setting like this may be ok on a case-by-case basis, but should be strongly inspected und monitored.
If I had the choice, I surely had preferred a permanently applied setting on the router instead.

Bash Script to rip CD/DVD ISO image on OS X

Posted by: admin  :  Category: Operating Systems, OS X, Programming, Scripting

Apple’s OS X has an easy way to rip a CD/DVD image using Disk Utility program.
However, you’ll end up with a file in that is not in ISO format, thus utterly useless if you want to re-use the file for virtualization purposes or on another operating system unable to handle those .cdr files.

For a one-shot option, OS X provides everything to convert the .cdr to .iso files, which is outlined at http://imacify.com/2013/06/how-to-create-iso-disc-image-from-cddvd-in-mac-os-x/.

If you do however plan to rip a lot (and I mean, a lot!) of CDs/DVDs to ISO files on OS X, here’s a little bash script I came up with.

Read more…

Fix iTunes refraining from syncing the iPhone

Posted by: gdelmatto  :  Category: Bits and Bytes, Hardware, HowTo's, OS X

Out of a sudden iTunes struck me with this error: The iPhone “…” could not be synced because the sync session failed to start

I did web research, but did not really find a proper solution to this.
Well, there were a few which either did some funny things on Windows (I run a Mac ….), or recommended to restore the iPhone from a previous backup, removing just one app, installing just one new app through the iTunes store, and, well, some other curious and strange things.

After all, I thought to give the restore thing a try, however that changed nothing. Same message came up. So I disabled WLAN sync as to not interfere with the iPhone being hooked up by the cable already — again to no avail.

So I decided to look into the iTunes internals. There I found a primising folder within the Application Library folder called SyncServices. It was not that big, around 6 megs. But inside there where clear traces of synchronisation stuff.

My solution to the above error was then to quit iTunes, remove the SyncServices folders completely. And voila, iTunes would just start syncing my iPhone as if nothing bad had ever happened.

A backup volume switcher for Apple’s TimeMachine

Posted by: gdelmatto  :  Category: Operating Systems, OS X, Programming, Shells

So here’s another piece of code I hacked up tonight.
Since I’m roaming around with my MacBook every now and then, the need arised, that I would need to switch my TimeMachine destination volumes based on location.

So while in the office, I’d like to backup to my external USB drive there.
Being at my home office, i’d like to backup to my NAS, while on the road, I’d love to habe my external mobile drive to kick in (and yes, I know about the “mobile backup feature” of OS X Lion, but that’s not the point …)
Read more…

TimeMachine mobile backup eats up disk space if backup drive unavailable

Posted by: gdelmatto  :  Category: Operating Systems, OS X

Apple introduced a new feature called “mobile backups” with OS X Lion.
The idea behind that: Use a temporary part of the local hard drive to store backups until the backup drive is connected again.

True, this is a good feature for most. But if you can spare some backups, because you – let’s say – store your files to a dropbox folder, which is synchronized to a cloud service, then you may not need the mobile backup feature of OS X.
In fact, you may notice running out of disk space, that you could use otherwise.
Read more…

VpnInit AppleScript: Override and Restore Default VPN-Routes on OS X

Posted by: gdelmatto  :  Category: Networking, Operating Systems, OS X, Programming, Scripting, Utilities

Years ago I wrote a small script to restore local default route after connecting the RAS VPN on Windows.

Now, I made up a similar script to do the same on OS X.
Read more…

Automating MakeMKV with AppleScript

Posted by: admin  :  Category: Operating Systems, OS X, Programming, Scripting

It’s only days since I wrote about a custom script action upon inserting a DVD in OS X, allowing me have either run the DVD Player or MakeMKV to rip the DVDs.

Now I was digging around if I could do some automation on the GUI part, which indeed turned out to work … at least to a certain degree.
Read more…

Make OS X run custom actions upon DVD insert

Posted by: gdelmatto  :  Category: Operating Systems, OS X, Programming, Scripting

I’m currently ripping my complete DVD collection off to MKV (Matroska) format, so I can stream them across my home network.

So I sought a way to have my OS X give me the choice to either start DVD Player or MakeMKV upon inserting a DVD.
Read more…

Deleting old TimeMachine Backups

Posted by: gdelmatto  :  Category: Operating Systems, OS X

Are you using an external hard drive for TimeMachine backup?
Are you using it to store other data as well?
Didn’t you divide your hard drive it into a data and TimeMachine partition? (Hell No …!)
Now you’re running out of free disk space and need more room? (Hell Yes …!)

Solution #1: Buy another (bigger) hard drive.

Solution #2: Remove some old TimeMachine backups to recover some space.
Read more…