Finally finished the Tiburon changes for the storage cluster config. Storage clusters are a bit different than computing clusters in a number of regards, not the least of those being the large RAID in the middle. In this case, the storage cluster is 8 identical JackRabbit JR5 units, each with 24 TB storage, 48 drives, 3 RAID cards, dual port QDR cards, and for our testing, we are using an SDR network (as we don’t have a nice 8 port QDR switch in house).

Henry Newman and a few other people I know are talking about RAID as being on the way out. John West pointed at this article this morning on InsideHPC. Their points are quite interesting. It boils down to this: If the time to rebuild a failed raid is comparable to the mean time between uncorrectable errors (UCE), due to reading/writing volume, then RAID as it is currently thought of, is going to need some serious rethinking.

Will try to do updates soon, and I owe someone two articles (sorry!). Add to this fighting off a cold … not a happy camper. Basically we are building an 8x JackRabbit JR5 storage cluster right now. I’ve caught a problem in Tiburon, our OS loader, in the process, and am fixing it. Tiburon is all about providing a very simple platform to enable PXE (and/or iSCSI) booting OSes to make installation/support simple.

As seen on InsideHPC, John West notes that the assets of Star-P were purchased by Microsoft today. Parsing of words is important. The phrase “acquired the assets of X” means that the IP was purchased. John points to the blog post where Kyril Faenov mentions that some of the staff will work at the Microsoft Cambridge site. This is sadly, not a great exit for Star-P. Acquiring assets usually means the choice has been to shut down the company, and auction the bits off, or find a buyer for the distressed assets and then wind down the rest of the organization that doesn’t go with the assets.

This has been bugging me for a while. Here is a simple measure of the height of the bandwidth wall. Take the size of your storage, and divide it by the maximum speed of your access to the data. This is the height of your wall, as measured in seconds. The time to read your data. The higher the wall, the more time you need to read your data. Ok, lets apply this in practice.

This is going to make a few Dell partners (Wipro et al) nervous. Sort of like the HP acquisition of EDS did. Is it possible that the service providers are going to be snapped up now to provide differentiated value in the face of declining revenues for hardware? Does this mean anything for HPC or storage? Not this particular acquisition. Perot Data Systems wasn’t/isn’t really a player in HPC to any significant degree.

* @[chris_bloke](http://twitter.com/chris_bloke) Oddly, I seem to remember my business partner working on stuff like this last year at his day job. Will ask. [in reply to chris_bloke](http://twitter.com/chris_bloke/statuses/3952863681) [#](http://twitter.com/sijoe/statuses/4005434311) Powered by Twitter Tools

We were knocked off the air around 11pm on 13-September, by a machine finally deciding to give up its ghost. A partially retired machine which happened to run scalability.org decided, finally, that it no longer wished to correctly run grub. Grub being the thing essential to booting. Like the bootloader. Yeah. It was one of those nights. I haven’t finished the figuring out why it died, and I am working on finishing restoring the services.

Scalable Informatics JackRabbit JR3 16TB storage system, 12.3TB usable. [root@jr3 ~]# df -m /data Filesystem 1M-blocks Used Available Use% Mounted on /dev/sdc2 12382376 425990 11956387 4% /data [root@jr3 ~]# df -h /data Filesystem Size Used Avail Use% Mounted on /dev/sdc2 12T 417G 12T 4% /data These tests are more to show the quite remarkable utility of the fio tool than anything else. You can probe real issues in your system (as compared to a broad swath of ‘benchmark’ tools that don’t really provide a useful or meaningful measure of anything) This is on a RAID6, so its not really optimal for for seeks.

Previously, JackRabbit JR3 units, with single RAID cards, have been hovering around 750MB/s read and write. This was our second generation unit. First generation units were about 600 MB/s +/- a bit. The third generation unit is faster. [root@jr3 ~]# dd if=/dev/zero of=/data/big.file ... 4096+0 records in 4096+0 records out 68719476736 bytes (69 GB) copied, 84.9058 seconds, 809 MB/s [root@jr3 ~]# dd if=/data/big.file of=/dev/null ... 4096+0 records in 4096+0 records out 68719476736 bytes (69 GB) copied, 66.