phaqphaq

After my recent article about FreeBSD GEOM mirroring on Sparc64 I was asked how to boot from a degraded array if the primary hard disk had failed.
The point was how to prevent the admin to interact with OpenBoot in that case.
I’m sure this will only apply to people not very familiar with Sun’s OpenBoot, as most other people will surely know about this.

While I’m pointing you at the solution, please have also a look at the OpenBoot Command Reference Manual.

#1 Get GEOM mirror device names

First make sure you know which devices you used for building the mirror. The command
‘gmirror list’ will show it to you. Get the consumer names for your GEOM provider. The following output has been shortened a bit.

Geom name: gm0
State: COMPLETE
Components: 2
Balance: round-robin
Consumers:
1. Name: da0
2. Name: da1

#2 Check where devices are connected to

Now you should grep through message to find where your GEOM consumers are connected to. I have shortened the output again for the relevant parts.

# dmesg | grep “da[0|1]”
da0 at sym0 bus 0 target 0 lun 0
da0: Fixed Direct Access SCSI-2 device
da1 at sym0 bus 0 target 1 lun 0
da1: Fixed Direct Access SCSI-2 device

#3 Enter OpenBoot firmware console

Shutdown your operating system to enter OpenBoot firmware console. You can even ‘STOP-A’ the machine, though I wouldn’t recommend that. If you just turned on your box and it is about to do the usual memory checks at all, it’s safe to issue ‘STOP-A’.

#4 Probe your scsi devices

If you know how the controllers and devices are attached and how they are enumerated in OpenBoot, you can skip to step 5. Otherwise issue a ‘probe-scsi-all’ command, which should give something like that:

/pci@1f,4000/scsi@2
Target 5
Unit 0 Removable Tape HP C1537A L706
Target 6
Unit 0 Removable Read Only device TOSHIBA XM5701TASUN12XCD2395

/pci@1f,4000/scsi@3
Target 0
Unit 0 Disk FUJITSU MAG3182L SUN18G 1111
Target 1
Unit 0 Disk FUJITSU MAG3182L SUN18G 1111

Look at the devices which match your output from step 2. In my example it’s the devices on scsi bus 3, target 0 and target 1.

#5 Enumerate your device aliases

Run the command ‘devalias’ to list your device aliases. Usually there should be matching device aliases for your hard disks. In my case this was (output shortened):

disk0 /pci@1f,4000/scsi@3/disk@0,0
disk1 /pci@1f,4000/scsi@3/disk@1,0

You may or may not have additional device aliases matching scsi bus 3. You may even have a device alias call simply ‘disk’ which reads the same as ‘disk1′. This depends on your configuration setup. Refer to the OpenBoot manual if you want to learn morn about it.

If the device aliases wouldn’t exist for some obscure reason, you should create them:

devalias disk0 /pci@1f,4000/scsi@3/disk@0,0
devalias disk1 /pci@1f,4000/scsi@3/disk@1,0

Make sure you don’t accidentally overwrite other device alias names, especially if you are unsure about wether you need them or not.

You could easily use other names beside ‘disk0′ or ‘disk1′. They should only be simple enough to be used in the next stept.

#6 Set your boot device priority

Now you should instruct OpenBoot about your boot device priority. Check it out first using ‘printenv boot-device’, which should return something like:

boot-device = disk net

Set it to your GEOM consumers which match to the device aliases you found in the previous steps, eg:

{1} ok setenv boot-device disk0 disk1 net
boot-device = disk0 disk1 net

Now whenever booting up your box, it should boot from the ‘disk0′ GEOM consumer (which is /dev/da0 in FreeBSD) and fail-over to ‘disk1′ (which is /dev/da1) if ‘disk0′ failed. It would even try to boot from the network when the ‘net’ device is also given as in my example.

Consider that you can set boot-device using real device names (eg. /pci@1f,4000/scsi@3/disk@0,0) though I wouldn’t recommend it at all. Use device aliases whenever possible.

Courier IMAP Login Exec (abbreviated ‘cile’) is a small patch against the popular Courier IMAP server.

It basically adds a ‘execute script on login’ feature to both pop3d and imapd.

The idea behind this is to allow automatic server side tasks to be run upon user login, eg. to check (and fix) the Maildir structure, create usage reports, expunge old mail, etc.

I have therefore created a patch which adds this functionality to Courier IMAP, which you may download and use free of charge. The patch is licensed unter the terms of the GNU GPL, version 2.

There are two releases available:

cile-generic 0.1

Take the ‘cile-generic’ package if you plan to apply this patch against the original source tarball.

cile-ports 0.1

Take the ‘cile-ports’ if you plan to apply this patch against the FreeBSD ports tree.

Both releases contain the same patches, they only differ in packageing and the way they are applied.

Any feedback is appreciated.

**** Article Obsoleted ****

This article has been obsoleted by the final release of VMware Server 1.0.
It is left here as a reference only.

**** Article Obsoleted ****

One thing I noticed when test driving VMware Server beta was the performance drop compared to the more mature ESX and GSX versions.

Of course there is nothing wrong about that considering the product still being in beta stage. It is however very hard to get an idea of the speed to be expected by the final product.

Luckily enough, VMware provides you with everything needed to circumvent this.

Of course, you shouldn’t try this if you are not willing to accept random crashes, possible lockups or other weird behaviour. VMware Inc. will most definitely deny any liability if you follow this procedure. READ: You should only try this if you are willing to take the risk. Neither blame me nor VMware if anything bad happens!

So to disable debugging mode in VMware Server beta, it’s hardly recommended to first shutdown all running services and VMs.

/etc/init.d/vmware stop

should do this. If you don’t have the VMware Tools installed it’s better to shut down your guest systems manually before stopping VMware.

Now go to the vmware lib directory at ‘/usr/lib/vmware’.

There you’ll find a directory called ‘bin-debug’. Move it to a new location, e.g.

mv /usr/lib/vmware/bin-debug /usr/lib/vmware/bin-debug.bak

Now make a symlink from the ‘bin’ directory to ‘bin-debug’:

ln -s /usr/lib/vmware/bin /usr/lib/vmware/bin-debug

The ‘bin’ directory contains the binaries stripped from debugging code. At least it did so on my installation
After creating the symlink, simply start up VMware Server:

/etc/init.d/vmware start

There you are now — without debugging mode

Btw: I’m still looking for the ‘real’ way to do this, as there must exist some command line argument, environment variable or something to put even the beta version in ‘non-debugging’ mode.

I found a reference to such a variable inside the /usr/lib/vmware/serverd/init.pl script, which says:

#27 if( -d ‘%debugdir%’ ) {
#28 $PREFIX_PATH = ‘%debugdir%’;
#29 } else {
#30 $PREFIX_PATH = ‘/usr/lib/vmware’;
#31 }

This scripts seems to be invoked by the vmware-serverd binary and the extract has definitely something to do with debugging. Though it does not specify the ‘bin-debug’ prefix in favour to the ‘bin’ directory.

Grepping the other files in /usr/bin and /usr/sbin and /etc/vmware gave me also some matches, though nothing that would of use to me, since most of them were located in binary files.
I will try to find out more on this. In the meantime, the workaround will allow you to testdrive VMware Server beta with debugging turned off. This will be of great use in getting a better idea of the actual performance rating.
A more recent article of mine covers another (the real?) way to disable debugging on VMware Server.

The initial version of this article was written in spring 2006 to cover installing VMware Server on Debian GNU/Linux 3.1 (sarge).

In the meanwhile, Debian 4.0 (etch) has been released, making it worth to review the procedure with the current VMware Server and Debian releases.

While for the initial testing a SuperMicro SuperServer 5014C-T with 2 Gigs of RAM, a Pentium 4 CPU @ 2.8 GHz and two 250 GB SATA hard drives (ICH6R, md software raid) was used, the current hardware looks quiet a bit different. Today a more powerful SuperMicro 6014 series server with 4 GiB RAM, two DualCore Xeon CPU @ 2.66GHz (VT enabled) and a 3ware 9000 storage controller (RAID10) was used.

Again Debian was bootstrapped via PXE/NetInstall, downloading everything through FTP.

After the basic install the following steps were taken to get VMware Server installed properly.

#1 Software Dependencies: X

Even if you are installing VMware Server on a host system without X, some dependencies must be fulfilled to ensure proper installation. So this is the list of packages required:

libx11-6
libxtst6
libxt6
libxrender1
libxi6

#2 Software Dependencies: db2

For the MUI package, another dependency must be installed:

libdb2 (for Debian 3.1)
libdb3 (for Debian 4.0)

This provides the libdb3.so.2 shared library.

#3 Software Dependencies: Kernel Source

Since VMware Server uses some kernel modules to provide monitoring and networking functions, you will need the kernel sources or the kernel-headers at hand.

Originally this article covered two methods, however for the sake of simplicity only the straight forward method will be lined out.
This is because the second method proved to be too error prone and is in fact only required if you happen to use a custom kernel.

Since I asked Debian to use a 2.6 kernel, I had to install the kernel headers and the kernel build environment.
The steps I took are similar if you use a 2.4 kernel on Debian 3.1.

So first I looked for the ‘linux-image’ package. I found two package names to be installed:

linux-image
linux-image-2.6

In Debian 3.1 these will be named ‘kernel-image’ instead, however they serve the same purpose.

To get the kernel headers and the build environment I looked for the ‘linux-headers-2.6.18-4′, ‘linux-headers-2.6.18-4-486′ and ‘linux-kbuild-2.6.18′ packages and installed them.
Debian 3.1 users will look for the similar packages by the name of ‘kernel-headers-xyz’ and ‘kernel-kbuild-xyz’.

Make sure you also install these packages: ‘binutils’, ‘gcc’, ‘make’, ‘libc6-dev’

After installing these packages, look at your modules directory at /lib/modules/`uname -r`.
There should be a symlink directory called ‘build’ pointing to ‘/usr/src/kernel-headers-2.6.18-4-686′.
If it would not exist or point to a different location, recreate the symlink to the proper header directory.

The kernel header directory (/usr/src/kernel-headers-2.6.18-4-686 or /lib/modules/`uname -r`/build/include) will be required during VMware installation later on.

#4 Software Dependencies: perl + CPAN

A fresh Debian installation is missing pretty much of everything. So you need to make sure that Perl is installed to including the whole base package.

Just check for the ‘perl’ and ‘perl-modules’ package and install them if required (vanilla Debian usually has only ‘perl-base’ installed).

You may find it valuable to also install some additional dependencies for CPAN first:

wget
lynx
unzip
gnupg
ncftp
ftp
curl

Now I would strongly recommend to configure CPAN on your host:

perl -MCPAN -e shell

Follow the configuration dialogs for CPAN (you *may* just press ENTER on every question as they provide you with typical default values — however it ain’t wrong to check out CPAN documentation first).

When you’re done with that, try installing the CPAN bundle. Type this in the CPAN shell:

install Bundle::CPAN

This will make sure you have the current CPAN modules at hand. It will ask you to ‘follow the dependency’ chain. Just confirm that to download and install all required modules.

As soon as it finished, try to install the MakeMaker package:

install ExtUtils::MakeMaker

This particular module is required for the VMware Perl API.

#5 Getting and Installing VMware Server

Now download the VMware packages. Use the tar files and take both the ‘VMware-server’ and the ‘VMware-mui’ package.

Right, you *can* leave the latter one away as it only provides the web-based frontend system. The essentials for running VMware are in the server package.

Untar the VMware-server tarball to a temporary location and run the vmware-install.pl script as stated by the VMware documentation.

NB: One Debian 4.0 I encountered a situation where vmware-install.pl mourned about ‘killall’ being missing. Simply installing it by means of ‘apt-get install psmisc’ solved this easily.

When asked for the ‘kernel headers’ simply provide the directory containing the kernel source or the kernel headers. You should have come along this in step #3 already.

It will now try to compile and load the kernel modules. If this step fails with an error message about failing to load the module into the kernel, double-check your kernel header and kernel source to match your currently installed kernel.
If you have met all the requirements this step should complete smoothly.

Using ‘/etc/init.d/vmware’ you can start/stop the server, though I my experience was that wouldn’t start until I rebooted the host at least once.

To install the MUI package repeat the above steps according to the setup instructions for the MUI package.

To start/stop the web ui use ‘/etc/init.d/httpd.vmware’.

#6 Accessing VMware Server

If you installed the MUI package, you should find your host system serving a page at https://IP:8333.

You need to use your ‘root’ login to access the management interface.

There you can also download the VMware console package for Linux or Windows.
You will need this to manage your VMware server as the web frontend does only allow starting/shuttind down/rebooting vm’s and status monitoring.

To connect using the VMware console just provide the server IP and the root login. In case it wouldn’t work, make sure no firewall blocks traffic from/to port 902.

#7 Securing your Host

A plain Debian installation is pretty much secure as it doesn’t run lots of fuzzy services as other distros do.

So after installing VMware you should only have port 902/tcp to listen, additionally 8222/tcp and 8333/tcp when the MUI was also installed.
If you didn’t pay attention to the mta during installation, you may end up with exim listening on port 25/tcp leaving your host as an open relay.

I usually configure it for local delivery so it will listen to port 25/tcp on localhost only.

For remote administration, you may want to install the ’ssh’ package (Debian 3.1) or ‘openssh-server’ and ‘openssh-client’ (Debian 4.0) respectively.

So all in all, you will only have a few ports open. I would stronly recommend to protect your ssh, smtp and vmware ports through iptables from unwanted access.
You may want to use the ‘ferm’ toolkit for easy but powerful creation of firewall rules.

If you are using VMware using bridged networking instead of host-only or nat, you can even think of bringing the physical host interface (eg. eth0) up without an ip address. This will reduce the risk of attacks to the host system itself. Just plugin a second NIC and attach it to a dedicated LAN (eg. non-public) for management purposes.

#8 Observations

It’s been some time that VMware Server has become a production grade product.

It has definitely matured since the beta releases and all the debugging code has gone.

Still with the availability of VT or similiar virtualization extensions native to the CPU performance impacts have become less critical than before.

Actually my experience in GEOM disk mirroring on Alpha dates back to FreeBSD 5.3, though it seems to work out perfectly after upgrading to FreeBSD 6.0.

Again, the concept is similar to setting up a mirror on intel boxes, though – again – there are some minor differences which may cause some headaches!

I’m referring once more to Ralf S. Engelschalls GEOM mirroring approach using whole disks.
As with the Sun platform there is no concept of disk slices on Alpha. Alpha is also limited to primary partitions also and lacks the absence of the “PC MBR”.

So his second method using disk slices may be adapted to primary partitions, though I did never try this on my own. For the sake of simplicity I prefer doing mirroring on whole disks pretty much the same way as physical controllers implement it.

So here is my step by step tutorial on creating a GEOM mirror on Alpha:

# assuming the following:
# first (ie. non-raid) scsi disk    : /dev/da0
# second (ie. future raid) scsi disk: /dev/da1
# geom mirror device (whole disk)   : /dev/mirror/gm0

# make sure the second disk is treated as a really fresh one
# (not really necessary, but makes procedure more deterministically 
dd if=/dev/zero of=/dev/da1 bs=512 count=79

# forget the whole PC MBR fdisk stuff, it simply doesn't exist on alpha  

# place a GEOM mirror label onto the second disk
gmirror label -v -n -b round-robin gm0 /dev/da1

# activate GEOM mirror kernel layer
# (makes the /dev/mirror/gm0 device available)
gmirror load

# place a BSD disklabel onto /dev/mirror/gm0
# (ATTENTION: consider the need to provide the full device path, eg. /dev/da1, etc
#  on some releases of FreeBSD -- there seems to be an inconsistency on how different
#  releases on different platforms handle this!)
# (NOTICE: figure out what partitions you want with "bsdlabel /dev/da0" before)
# (NOTICE: *no* partition offset is requiered at all for "a" partition!)
bsdlabel -w -B /dev/mirror/gm0           # initialize
bsdlabel -e /dev/mirror/gm0              # create custom partitions

# manually copy filesystem data from first to to second disk
# (same procedure for partitions "g", etc)
# (NOTICE: in my sample a have a single partition containing the / fs,
#  just extend as needed if you have/need more partitions for /usr, etc)
newfs -U /dev/mirror/gm0a
mount /dev/mirror/gm0a /mnt
dump -L -0 -f- / | (cd /mnt; restore -r -v -f-)

# adjust new system configuration for GEOM mirror based setup
cp -p /mnt/etc/fstab /mnt/etc/fstab.orig
sed -e 's/dev/da0/dev/mirror/gm0/g' /mnt/etc/fstab
echo 'swapoff="YES"' >>/mnt/etc/rc.conf # for 5.3-RELEASE only
echo 'geom_mirror_load="YES"' >>/mnt/boot/loader.conf

# instruct boot stage 2 loader on first disk to boot
# with the boot stage 3 loader from the second disk
# (mainly because BIOS might not allow easy booting from second disk
# or at least requires manual intervention on the console)
echo "1:da(1,a)/boot/loader" >/boot.config

# reboot system
# (for running system with GEOM mirror on second disk)
shutdown -r now

# make sure the first disk is treated as a really fresh one
# (also not really necessary, but makes procedure more deterministically 
dd if=/dev/zero of=/dev/da0 bs=512 count=79

# forget the whole PC MBR fdisk stuff, it simply doesn't exist on sparc  

# switch GEOM mirror to auto-synchronization and add first disk
# (first disk is now immediately synchronized with the second disk content)
gmirror configure -a gm0
gmirror insert gm0 /dev/da0

# wait for the GEOM mirror synchronization to complete
sh -c 'while [ ".`gmirror list | grep SYNCHRONIZING`" != . ]; do sleep 1; done'

# reboot into the final two-disk GEOM mirror setup
# (now actually boots with the MBR and boot stages on first disk
# as it was synchronized from second disk)
shutdown -r now

The equipment used in this article is a AlphaStation 500/266 with an Alpha 266 Mhz CPU, 128 Megs of RAM and two IBM DDYS-T36950N S96H 36 GB SCSI drives.
No computers have been harmed whatsoever during my experiments.