Finding a New Fileserver OS: Step One - Filesystems

Written 2008-11-12

Tags:Solaris ZFS Storage OpenSolaris Standard RAID levels Linux RAID 

prev next
Recently, I started to run out of hard drive space. Previously, I was using this machine as a Linux desktop/fileserver, but as I've become more accustomed to running OSX/Windows2003/Linux2.6 on my laptop, I've lost the need for a local console.

Previous setup:
2x250Gig 7200RpM Sata drives in Raid-0
1x80Gig drive for backup

New Setup:
4x500Gig 7200RpM drives in Raid5/Raid6/ZFS/BTRFS
assuming 20MB/s = 160Mb/s
1x80Gig drive for OS

Comparisons:
FSCK Checking Time
Compression
OverHead
Data Safety
Speed ( I want 400Mb/s from the array)


ZFS
I've used ZFS with Solaris 10 before, and I must say I'm impressed. It offers dynamic compression, multi-drive mounts, and redundancy up to a level of your choosing. We used ZFS on the ACM fileserver. It handles 12 drives (18Gig, 15kRpM scsi drives) with double redundancy (3 failures must occur before data loss). The main bottleneck in this system is the 100Meg Sun HappyMeal Ethernet card, which it can effortlessly keep filled with AFS data. Also, the CPUs in this box are 400Mhz each, so I figure 4xP4s will have no trouble at all with fewer drives. What about a filesystem check? ZFS keeps things stable at all times. A full scrub is relative to the amount of data stored, not the size of the disks. Supported Operating Systems: OSX Snow Leopard, FreeBSD, Solaris, OpenSolaris, and Linux/FUSE.

BetterFS
BetterFS is supposed to replace Ext4 when it is done. It isn't done yet though. When done, it should feature multi-drive mounts (but only Raid0/Raid1 style), CopyOnWrite for everything, ACLs, and online FSCK. However, at time of writing, it isn't done yet. Supported Operating Systems: Linux.

Raid-5 with a standard filesystem
Raid-5 is a method for adding checksumming to a group of block devices. It presents you with one virtual drive consisting of many hardware drives. However, you lose 1 drive out of n for parity, so your efficiency is (n-1) drives, and there have been some problems with Raid5 scaling to very large numbers of drives. Supported Operating Systems: Linux, FreeBSD, NetBSD, Solaris, OpenSolaris, and Windows Server.

Raid-6 with a standard filesystem
Raid-6 is much like Raid-5, but you use the equivalent of two drives for parity, so your efficiency is (n-2). This also takes more cpu time to write, the Raid has to be checked in its entirety, not just the space with files on it, and still has scalability problems. Supported Operating Systems: Solaris/OpenSolaris, or any other with a hardware card.

HammerFS
HammerFS is a relatively recent filesystem for DragonFlyBSD. It's neat in the way that it supports clustering, data checksums, multidrive mounts, and dynamic rollback/versioning. However, I only have one machine, so its not the best of options. Supported Operating Systems: DragonFlyBSD.

Decisions, Decisions
Factors:
  • I don't want anything that isn't supported by at least two operating systems. So long HammerFS and BetterFS.
  • I don't want to be locked to a single model hardware card. So long Raid-6
  • I'd like Linux(liveCD rescue) and OSX support (laptop rescue)
  • Fast Fast Fast
I'm really just left with ZFS. I mean, Raid-5/EXT3 and Raid-5/JFS have worked for me in the past, but I imagine checking a 2TB array will take forever with that setup. I've seen ZFS perform in the past, so it will be the first FS I test. Now the question is: what OS do I run?
prev next