For the good of all of us

If you ever tried to take a look around an OSX install DVD unter Linux you may have been surprised by
the marvelous compression algorithms that Apple uses, because there seems to be preciously little data on those disks:

$ du -sh /media/OSX86DVD/
132K    /media/OSX86DVD/

There is no magic here, as seen by the fact that the same disk inserted into a running OSX system shows a completely different file structure with more than 4GB of data.

The explanation for this is that there are two filesystems on the DVD:

First, a normal ISO9660 CD file system, which contains some bootloader files, and not much else. This filesystem is the one that Linux (and Windows) see by default.

Behind the ISO9660 filesystem there is another one, which spans the rest of the disk. This is a complete hard disk image with it’s own partition table, and which contains the real installer data. As Linux does not expect this it does not try to access this filesystem, and the files remain invisible.

In order to get at the files in the second part of the disk some command line magic is necessary. You will need root privileges for the following operations. I assume that the DVD device is /dev/sr0, adjust as neccesary.

The Linux kernel has a nice block device mapping layer which allows us to take a slice out of an existing block device and present this slice as a new device. Even more, there is a helper tool that inspects a block device, looks for partitions and automatically creates such a mapping for each partition. This tool is called kpartx.

So we use kpartx to inspect the DVD device. The command for this is as follows:

# kpartx -a /dev/sr0
device-mapper: reload ioctl failed: Invalid argument
create/reload failed on sr0p1
device-mapper: reload ioctl failed: Invalid argument
create/reload failed on sr0p2
#

That did not work too well. For some reason the device mapper does not like CDROM/DVD devices. So we’ll have to get a bit inventive. We create a loop device which is backed by the DVD device, and use that.

# /sbin/losetup /dev/loop0 /dev/sr0
# /sbin/kpartx -av /dev/loop0 
add map loop0p1 (253:5): 0 63 linear /dev/loop0 1
add map loop0p2 (253:6): 0 9178688 linear /dev/loop0 448
#

The command found and created two partitions. The first partition is the ISO9660 file system at the beginning of the disk, the second one is the partition we are interested in. Let’s look what’s in it.

# file -s /dev/mapper/loop0p2
/dev/mapper/loop0p2: Macintosh HFS Extended version 4 data last mounted by: '10.0', created: 
Tue Oct 30 00:32:01 2007, last modified: Wed Dec 19 16:45:14 2007, last checked: Tue Oct 30 00:32:01
2007, block size: 4096, number of blocks: 1147336, free blocks: 192685
#

That looks good. We could now mount the filesystem and look around.

# mount /dev/mapper/loop0p2 /mnt
# ls /mnt
Applications  Desktop DB  dev  Install Mac OS X.app  mach_kernel  sbin    tmp  vanilla  Volumes
bin           Desktop DF  etc  Library               private      System  usr  var
#

To get rid of all this again we have to unmount the file system, destroy the mappings and release the loop device:

# umount /mnt
# kpartx -d /dev/loop0
# losetup -d /dev/loop0

Today the first half of my order arrived: the case, the mainboard and the fan. As this delivery was somewhat unexpected (the mainboard is a new model, and I had expected it a week later), I now have new hardware I can not use, due to the lack of a CPU. This is a bit embarassing, but cannot be changed right now, so I’ll start with what I have.

My apologies for the appaling quality of the pictures, but all I have in the way of digital image capture is the camera in my cell phone.

The Case

The first thing I noticed about the case is how solid it looks and feels. Contrary to the photos I had seen so far the case is not all black, but the sides and the top and bottom are a dark siverish grey. The front, however, is black. I have to say it looks quite good.
Although the front and sides are plastic, there is absolutely nothing cheap about the feeling. This is underlined by the weight of the thing. Even though there is no power supply in it (that is external), and no parts have been mounted yet, the empty case weighs over seven kilograms. Under the plasic outside panels there is a massive metal cage. It really has a no-nonsense feel to it.
The inside is full of pre-routed cables that lead to the front panel, the power distribution plane and the hard disk backplane. The case has a 20+4-pin ATX power connector plus the four pin additional CPU power connector most current boards need. Luckily, all the MSI board requires is the 20-pin ATX connector.

A pleasant surprise (aside from the colour) was that the case has a multi-format card reader already built in. From the manufacturers site I had gathered that this was an optional extra, but my case came with one included.

Also included is a CPU fan, but that is quite specific for a certain mainboard, and will not fit most other boards, so it is useless to me.

What is not included is a manual, at least I have not found any. As all of the cables are clearly labeled this is not much of a problem, though.

The Board

The mainboard comes with the usual assortment of cables (2xSATA, 2xSATA power adaptor, 44-pin IDE) and the rear panel bracket. In addition, it also contains a CPU fan, which surprised me. It also is a Coolermaster model, but not the same I ordered extra. If, as I suspect, the CPU also comes with a fan I’ll have quite enough of those things.

The board also had a pleasant surprise, this one on the bottom of the board. MSI put a CF card socket there, which the web site stated as an optional extra. I think I’ll use a CF card instead of the notebook hard disk,
as this produces less noise and heat.

The Fan

Well… it’s a fan, right? Goes on the CPU, and hopefully does not make too much noise. I can always threaten it with the other fan if it does.

This is supposed to be a documentation of my endeavour to build an OpenSolaris based storage machine for my home use. Coming from a Linux background myself it will also serve as a notebook of how to do stuff under Solaris.

History

For years I had a midi tower based system running, which was both my internet router and the local storage machine. This machine was shut down eventually, energy prices being what they are, and was replaced by an ASUS router running OpenWRT for internet access. The storage facility was not replaced, so only the hard drives on the client machines themselves were left. In the middle of last year I finally had enough of that and started looking around for a small storage appliance. I wanted something I could play around with, so being able to screw with or replace the original operating system was a must.

I ended up with the Thecus N2100, which is a small, ARM based NAS enclosure running Linux from embedded flash, and able to house two SATA hard disk drives. It’s possible to get a fully functional Debian system on it if you’re not afraid of poking around with serial ports (which I am not), so it seemed like a good choice. It was ordered together with two Seagate 7200.11 1TB drives.

It turned out pretty fast that the Thecus and the Seagate drives did not like each other a whole lot, which is probably due to the rather high spinup current that the Seagate drives need (3A on the 12V rail). This was more than the Thecus could provide, so the drives did not spin up most of the time. So the two Seagate drives were replaced with two Samsung 1TB drives, and the Seagates were banished to the shelf.

In hindsight this was probably a good thing, because a) the Seagates did not have time to fill up their log and run into the current firmare bug, and b) I had 4 1TB drives lying around, which would come in handy later.

The Thecus liked the Samsung drives a whole lot better, and the original firmare was quickly replaced with a Debian Lenny distribution. From a purely administrative standpoint all this worked very well, the distribution detects all the hardware in the system (not that there is a whole lot of it, but nonetheless), including the multi-coloured LEDs in the front panel and the fan controller.
Debian duly provided me with NFS and SAMBA based storage, a print server for the printer connected to the USB ports, and several other services.

The main problem with the Thecus was speed. The system is equipped with a 600MHz ARM processor, which sounds quite beefy, especially compared to the other NAS storage enclosures out there, which usually have less.
In reality it is not a whole lot. None of the individual subsystems are epecially fast on their own (the system has two 1GB network controllers, but trying to get 100MBit directly from memory is pushing things, and the storage controller has problems of it’s own, also limiting the possible performance). The net result of all this was that getting more than 5MB/s read or write performance was pretty much out of the question.

As this was unsatisfactory a new solution was needed. In the mean time I had played around with OpenSolaris (in the form of the bi-weekly nevada snapshots), and was quite impressed by it’s ZFS file system. So I wanted
to build the new system around this OS, in order to try it out in real life with more than just a few megabytes of test disks.

This meant using an Intel based machine, though (getting a Sparc based enclosure seemed to push my luck), so I started looking around again. Thecus offered a five disk hot-swap enclosure (the 5200(PRO)) with a 600MHz or 1.6GHz Celeron processor, Marvell-SATA-Controller and Intel Gigabit Ethernet, booting from an internal flash disk. While talking this through with several people on IRC (thanks, ofu!) it became clear that I could get more performance for the same money when building the system myself.

(Ironically, I have gotten my hands on a Thecus 5200 based system as well, so I get to have the best of both worlds. Life is great, sometimes).

I did not want a midi or mini ATX tower though, so choice was getting slim. ofu again pointed me towards the Chenbro 340 case, which has four hot swap cabable SATA bays and takes a Mini-ITX board. Finding a fitting board turned out to be somewhat complicated, as Mini-ITX boards with (at least) 4 SATA ports are rare (in theory it is possible to put a PCI card into the case using a riser card, but I did not want to go that way).
Even more, all the hardware (especially storage and network) had to be supported by Solaris. I figured the safest way to go was getting an all-Intel board (Intel SATA controller, Intel network), as these parts are known to work well.

In the end I chose the MSI IM-GM45, which has four on-board SATA ports connected to an Intel ICH9M-E controller, two Intel gigabit ethernet ports and takes an Intel Penryn processor (among others). It also has an IDE connector, which will drive the boot disk (the four SATA drives will run in a RAIDZ configuration, from which Solaris can not yet boot).
The board takes up to 4GB of DDR2 RAM, which is plenty. It also has wide selection of video outputs (which I do not need at all), and five serial ports, one of which is on the rear panel (which I do need).

The final list of parts for this project is as follows:

• Chenbro 340 case
• MSI IM-GM45 mainboard
• CoolerMaster EPN-41CSS-01-GP cooler+fan
• Intel T8100 CPU
• 4GB RAM

The hard disks are already there (the two Seagates on the shelf, and the two Samsungs in the old enclosure). These will be resused. I also have several old notebook drives lying around, one of which will be used as the boot disk.