Posts
JR5: Marathon run
For those benchmark pr0n viewers … here is Scalable Informatics Inc. JackRabbit JR5 unit with 48 drives. Simple benchmark. How long does it take you to write 1TB. How about 524 seconds?
[root@jr5 ~]# !498 mpirun -np 8 ./io-bm.exe -n 1024 -f /data/file -w -d -s -v -------------------------------------------------------------------------- [[8861,1],3]: A high-performance Open MPI point-to-point messaging module was unable to find any relevant network interfaces: Module: OpenFabrics (openib) Host: jr5.scalableinformatics.com Another transport will be used instead, although this may result in lower performance.
Posts
A silly bug in io-bm
It wasn’t enough to impact results, but it was enough to cause questioning my results (and sanity). Part of the IO operation is having N processes write to 1 file. To make this happen correctly, each process has to compute their offset into the file, and start operations from there. There is a seek involved. Now if I am smart, it won’t be lseek(file_descriptor,0,SEEK_SET); Nope … that would be wrong (the zero).
Posts
Getting Inc.ed
Somewhat of a play on getting inked, the day job is now an Inc. rather than an LLC. The reasons for the change are many, ranging from taxes, to important things like raising capital. Yeah, I know, we are at a 15 year low for VC investments. But this is a capital intensive business, and we are growing, and have a need. So, this is one option for pursuing the capital we need.
Posts
rethinking git vs hg
I thought git was just getting better than mercurial. I found that basic git operations are easy, but others appear to be harder than I like (setting up remote repositories). Just tried the same thing with hg that I tried (and failed to do) with git. Worked like a charm.
landman@pgda-100:~/target$ hg push ssh://mmmmq@mmmmq/hg/target pushing to ssh://mmmmq@mmmmq/hg/target searching for changes remote: adding changesets remote: adding manifests remote: adding file changes remote: added 1 changesets with 5 changes to 5 files hg can use ssh, which looks like it makes lots of stuff simple.
Posts
Talking about high performance ...
… the Blue Angels are doing hard banks right above my house right now, as part of the Willow Run Thunder over Michigan airshow. We went to it yesterday, but live close enough that we see them on their far turns. Two minutes ago they were literally right over my house, doing a hard bank. Have pictures and videos. Most impressive! These guys and gals have some serious high performance hardware … I have to admit, some of my favorite parts were when they did their high speed passes, and then opened up with afterburners on the climb out.
Posts
... and HP snarfs up Ibrix ...
See here Whats Dell going to do now? Ibrix is Dell’s favorite parallel file system for clusters.
Posts
More JackRabbit 96TB OpenSolaris system
even more JackRabbit OpenSolaris 2009.06 goodness.
landman@pgda-100:~$ ssh scalable@192.168.1.74 Password: Last login: Fri Jul 17 13:32:57 2009 Sun Microsystems Inc. SunOS 5.11 snv_111b November 2008 scalable@jr5-96TB:~$ su Password: scalable@jr5-96TB:~# zpool create tank c7t1d0 c10t2d0 c11t3d0 scalable@jr5-96TB:~# zpool status tank pool: tank state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 c7t1d0 ONLINE 0 0 0 c10t2d0 ONLINE 0 0 0 c11t3d0 ONLINE 0 0 0 errors: No known data errors scalable@jr5-96TB:~# df -h /tank Filesystem size used avail capacity Mounted on tank 70T 19K 70T 1% /tank 70TB of ZFS on JR5-96Tn.
Posts
JR5 96TB running OpenSolaris 2009.06
Scalable Informatics JackRabbit OpenSolaris pr0n for those disposed to such things. Remember, this is a 2.5+ GB/s machine (e.g. measured bandwidth on RAID6, not “theoretical bandwidth” with a RAID0). From what I can see, this is the largest (capacity) and fastest (raw and usable) performance OpenSolaris machine on the market. 8 Nehalem cores, 72 GB RAM, 96TB raw, 72TB usable in 5U, for under $60k USD, with 10GbE and/or Infiniband. There does appear to be some strangeness upon setup … Solaris was not willing to run on a > 2TB disk.
Posts
When debugging ... make sure you have evidence before you point fingers
A 128 core cluster, user is experiencing ‘strange’ delays in their application. It is an MPI code, we found problems in previously. I have to admit that at first I was amused when someone blamed the MPI stack, claiming it was broken, and did so by demonstrating that they didn’t know how to use an MPI stack. Their test case, literally copied from from an online course (possibly even our course materials), was the MPI-HelloWorld fortran example.
Posts
OT: Bad ideas ... part II
Again, not a partisan issue … this approach to fund something by penalizing one group is a very bad idea. Cigarette and alcohol taxes are in place and as high as they are, specifically to change behavior. Making something far more expensive does materially impact the way people behave. It causes them to perform more cost-benefit analyses. And they change their behaviors. I am quite certain that those who designed this legislation did not anticipate that by significantly increasing taxes on businesses, that this would impact … I dunno … employment?