This is supposed to be a documentation of my endeavour to build an OpenSolaris based storage machine for my home use. Coming from a Linux background myself it will also serve as a notebook of how to do stuff under Solaris.
History
For years I had a midi tower based system running, which was both my internet router and the local storage machine. This machine was shut down eventually, energy prices being what they are, and was replaced by an ASUS router running OpenWRT for internet access. The storage facility was not replaced, so only the hard drives on the client machines themselves were left. In the middle of last year I finally had enough of that and started looking around for a small storage appliance. I wanted something I could play around with, so being able to screw with or replace the original operating system was a must.
I ended up with the Thecus N2100, which is a small, ARM based NAS enclosure running Linux from embedded flash, and able to house two SATA hard disk drives. It's possible to get a fully functional Debian system on it if you're not afraid of poking around with serial ports (which I am not), so it seemed like a good choice. It was ordered together with two Seagate 7200.11 1TB drives.
It turned out pretty fast that the Thecus and the Seagate drives did not like each other a whole lot, which is probably due to the rather high spinup current that the Seagate drives need (3A on the 12V rail). This was more than the Thecus could provide, so the drives did not spin up most of the time. So the two Seagate drives were replaced with two Samsung 1TB drives, and the Seagates were banished to the shelf.
In hindsight this was probably a good thing, because a) the Seagates did not have time to fill up their log and run into the current firmare bug, and b) I had 4 1TB drives lying around, which would come in handy later.
The Thecus liked the Samsung drives a whole lot better, and the original firmare was quickly replaced with a Debian Lenny distribution. From a purely administrative standpoint all this worked very well, the distribution detects all the hardware in the system (not that there is a whole lot of it, but nonetheless), including the multi-coloured LEDs in the front panel and the fan controller.
Debian duly provided me with NFS and SAMBA based storage, a print server for the printer connected to the USB ports, and several other services.
The main problem with the Thecus was speed. The system is equipped with a 600MHz ARM processor, which sounds quite beefy, especially compared to the other NAS storage enclosures out there, which usually have less.
In reality it is not a whole lot. None of the individual subsystems are epecially fast on their own (the system has two 1GB network controllers, but trying to get 100MBit directly from memory is pushing things, and the storage controller has problems of it's own, also limiting the possible performance). The net result of all this was that getting more than 5MB/s read or write performance was pretty much out of the question.
As this was unsatisfactory a new solution was needed. In the mean time I had played around with OpenSolaris (in the form of the bi-weekly nevada snapshots), and was quite impressed by it's ZFS file system. So I wanted
to build the new system around this OS, in order to try it out in real life with more than just a few megabytes of test disks.
This meant using an Intel based machine, though (getting a Sparc based enclosure seemed to push my luck), so I started looking around again. Thecus offered a five disk hot-swap enclosure (the 5200(PRO)) with a 600MHz or 1.6GHz Celeron processor, Marvell-SATA-Controller and Intel Gigabit Ethernet, booting from an internal flash disk. While talking this through with several people on IRC (thanks, ofu!) it became clear that I could get more performance for the same money when building the system myself.
(Ironically, I have gotten my hands on a Thecus 5200 based system as well, so I get to have the best of both worlds. Life is great, sometimes).
I did not want a midi or mini ATX tower though, so choice was getting slim. ofu again pointed me towards the Chenbro 340 case, which has four hot swap cabable SATA bays and takes a Mini-ITX board. Finding a fitting board turned out to be somewhat complicated, as Mini-ITX boards with (at least) 4 SATA ports are rare (in theory it is possible to put a PCI card into the case using a riser card, but I did not want to go that way).
Even more, all the hardware (especially storage and network) had to be supported by Solaris. I figured the safest way to go was getting an all-Intel board (Intel SATA controller, Intel network), as these parts are known to work well.
In the end I chose the MSI IM-GM45, which has four on-board SATA ports connected to an Intel ICH9M-E controller, two Intel gigabit ethernet ports and takes an Intel Penryn processor (among others). It also has an IDE connector, which will drive the boot disk (the four SATA drives will run in a RAIDZ configuration, from which Solaris can not yet boot).
The board takes up to 4GB of DDR2 RAM, which is plenty. It also has wide selection of video outputs (which I do not need at all), and five serial ports, one of which is on the rear panel (which I do need).
The final list of parts for this project is as follows:
* Chenbro 340 case
* MSI IM-GM45 mainboard
* CoolerMaster EPN-41CSS-01-GP cooler+fan
* Intel T8100 CPU
* 4GB RAM
The hard disks are already there (the two Seagates on the shelf, and the two Samsungs in the old enclosure). These will be resused. I also have several old notebook drives lying around, one of which will be used as the boot disk.