The recycling of an older business model using newer technology ASPs began the decade promising to reduce OPEX and CAPEX for HPC systems. They flamed out, badly, as they really didn’t meet their promise, and you had all these nasty issues of data motion, security, jurisdiction, software licenses, utilization, and compatibility. The concept itself wasn’t bad, create an external data center where you can run stuff, and pay for what you use.

Accelerators in HPC … In 2002, my business partner (he wasn’t then), showed me these Cradle SOC chips. 40 cores or something like that, on a single chip, in 2002 time frame. My comment to him was, we should figure out a way to put a whole helluva lotta them (e.g. many chips with RAM etc) onto PCI cards, with programming environments. Make them easy to use. Easy to program. We spent the next 2-3 years looking at a bunch of architectures, a bunch of chips.

The impact of markets, and government upon HPC … While the charts from top500.org are nice, they don’t tell everything that happened in this interval. We had 3 recessions, 2 major (2001 and the “Great Recession”) and 1 minor one. We had significant changes in research funding from the US federal government … a refocusing of DARPA on things less HPC specific. These elements all contributed to the trajectory within the decade.

The relentless onslaught of clusters … We are also mostly doing SMPs and MPPs then. Clusters are barely registering. See the chart and the data to get more perspective. What happened in the market was a simple alteration of the cost scale per flop. Clusters provided massive numbers of cheap cycles. Add to this that MPI has been standardized, reasonably well designed, and people were migrating codes to it. Funny, MPI on a cluster runs just as nicely as MPI on the SGI.

The death of RISC … obligatory Monty Python Holy Grail quote: old man: I’m not dead yet, I think I’ll go for a walk John Cleese: Look, you are not fooling anyone … The RISC vendors (SGI, HP, IBM, …) realized that RISC was dead, and that EPIC would be the technology that killed it. I was at SGI at the time, and disagreed that EPIC (Itanium) was going to be the killer.

[Update: 9-Jan-2010] Link fixed, thanks Shehjar! This is sort of another itch I need to scratch. Please bear with me. This is a long read, and I am breaking it up into multiple posts so you don’t have to read this as a huge novel in and of itself. Many excellent blogs and news sites are giving perspectives on 2009. Magazine sites are talking about the hits in HPC over the last year in computing, storage, networking.

dstat output on a 10GB pbzip2 compressed file being uncompressed … with pbzip2. ----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system-- usr sys idl wai hiq siq| read writ| recv send| in out | int csw 49 3 42 5 0 0| 35M 95M|3168B 6978B| 0 28k|1575 1198 52 4 39 5 0 0| 39M 188M|2508B 5230B| 0 80k|1769 1475 51 4 40 5 0 0| 19M 206M|4686B 9390B| 0 0 |2396 2240 42 4 48 5 0 0| 31M 158M|3054B 5360B| 0 16k|1820 2025 50 5 40 5 0 0| 37M 115M|2640B 5360B| 0 104k|1731 1564 38 4 50 8 0 0| 24M 105M|3102B 6270B| 0 0 |1639 1477 ^C Run … don’t walk … to get pbzip2.

As it turns out, a good friend was on that Northwest flight. I won’t identify him (though I know he reads this blog occasionally). What happened to him has made me think of my own responses in such a scenario. But it has also made me question the TSA’s kneejerk ineffective new guidelines. Especially in light of the potential accuracy of this report, if true, suggests that real security measures ought to be taken.

Two days ago, some nutjob wanted to blow up an airplane 20 minutes out from the airport I live 15 minutes from. Said nutjob is apparently in a hospital in Ann Arbor, again, 20 minutes away from us. Today, the TSA is closing the barn doors after all the horses have left. I am going to follow their train of illogic to its inevitable conclusion. At some point in time in the future, our TSA won’t allow planes to take off and land until all passengers are in a chemically induced coma.

A quick and dirty test on a JackRabbit Flash machine. This is a machine going to a proof of concept project soon. A simple 8k random read against 256GB data spread out over 4 volumes. 1 machine. 48GB ram. Open up the throttle on the stock config. Engines aren’t running at full speed, but its a baseline test. random: (groupid=0, jobs=256): err= 0: pid=18717 read : io=262144MB, bw=733052KB/s, iops=91631, runt=366189msec clat (usec): min=296, max=184778, avg=2777.