You've probably noticed that I'm a bit of a storage guy. But I'm also looking for just want I need, not necessarily more; and I want it at a good price. There are, of course, a number of highly scalable solutions in the storage space, but tend to extend up into the price range that I'm just not willing pay. IBM's DS3000 and 4000 series are good examples. The 3200 is scalable up to 48TB (4 arrays of 12 SAS/SATA drives) , the 4800 up to 16 trays of 14 drives for 224 TB. The 4800 extremely enticing, except for the cost. That leaves me looking at pulling something together myself. SAS controllers provide a great way to scale-out at low cost simply because the cards themselves support a large number of devices, something like 256 per SAS adapater versus the SATA cards which top out at around 24. The scale-out is achieved via the min-SAS external ports to cascade multiple disk enclosures. However, cascading requires the enclosure to include a fan-out adapter; basically the enclosure includes a SAS/SATA chip that handles identifiying and managing the disks in that enclosure. The typical SAS/SATA enclosure at the low end supports up to 4 chassis cascading, this gives us between 48 and 64 devices (12 or 16 disk enclosure). There are some exceptions, the SANBloc J50 and this new beast I've just found. The SANBloc is quite nice, 12 disks, supports cascading up to 7 enclosures and a reasonable price for this no-man's land between the low end, 4-chassis limit and the high end 16+.
I'll probably have to contact the company directly for pricing; none of the web shopping tools seem to know much about the DNF SPOD 16000 JBOD. The SPOD keeps the 7 chassis limit, but sports 16 drives per enclosure, bring the total capacity to 112 devices, roughly half the limit of my fav adapter; Adaptec's 51245 Unified SAS/SATA controller. My ideal setup includes this single card, direct attachment of 16 drives, plus externally another 112 devices, for a total of 128 TB raw capacity.
How does the cost compare? The IBM EXP3000 12 disk enclosure trays start at 3200 a piece, the SANBloc's are $1600, let's assume the SPOD is about $2200. That gives us $22,400 for 84 devices from IBM, $11,200 for the SANBlocs, and $15,400 for 112 devices from DNF. Breaking that down as cost per device: $267 per IBM slot, $134 per SANBloc slot, and $138 per DNF slot. Even at $2500 per DNF, that's only $156 per slot and supports 28 more devices than the SANBloc.
The other concern that I'm throwing out the window here is array performance. Each mini-SAS port has 4 300MB/s ports, for 1200 MB/s of total bandwidth . With 112 300 MB/s SATA II drives we're capable of generating 33,600 MB/s. Funneling that torrent through the 1200 MB/s link and we squeak out ~3.5% of the total bandwidth. For a much higher through put, one needs to incorperate far more Adapater cards without cascading more drives than the link can handle. However, my main use-case is total storage capacity. None of my uses for the storage will exceed the capacity of 1Gbit networking either and that makes either the SANBloc or the DNF a good deal for lower-cost highly scalable storage.
