BYOG (Build Your Own Grid)! Over the past few weeks I’ve been working with a couple of different companies to install, test, and document how to install and run Sun Grid Engine using Joyent Accelerators as the basic Infrastructure as a Service (IaaS) building blocks Sun Grid Engine as the Platform as a Service (PaaS) install on that infrastructure.

The Grid Engine project is an open source community effort to facilitate the adoption of distributed computing solutions. Sponsored by Sun Microsystems and hosted by CollabNet, the Grid Engine project provides enabling distributed resource management software for wide ranging requirements from compute farms to gridcomputing. A video is available that introduces grid computing and the Grid Engine project.

The great news is that it works wonderfully. You can very easily start small with Sun Grid Engine node or two and then scale on-demand by adding more nodes or making your nodes larger depending on the resources you need. I have to thank Bernard Li, whom also works on the very cool Ganglia project, for helping me by contributing a how to install guide which I transformed into a wiki document. Mark Mayo, one of our resident guru’s here at Joyent said about SGE, “the thing I like about SGE is that it’s a reliable, proven batching and queueing system. Once you set it up, it can be handy for all sorts of async processing needs not just traditional HPC / financial batching”

Here is a basic how to document for getting things up and running.

How to Install Sun Grid Engine 6.2 on Joyent Accelerators

In the tests that were done in the labs it was Blast simulations.

Basic Local Alignment Search Tool, or BLAST, is an algorithm for comparing primary biological sequence information, such as the amino-acid sequences of different proteins or the nucleotides of DNA sequences. A BLAST search enables a researcher to compare a query sequence with a library or database of sequences, and identify library sequences that resemble the query sequence above a certain threshold.

Blast is just one example of many types of tools that can be run on an SGE compute grid enabled by Joyent IaaS. I won’t even pretend to be an expert in all the wide and varied uses for SGE but I do know a couple of people that do know and I’m excited about the very interesting ideas they have in mind.

Personally, I find this an exciting use of Joyent’s unique and extremely flexible On-Demand Cloud Computing Infrastructure and the Joyent Accelerator as a core technology enabling the rapid deployment of a Sun Grid Engine Compute Grid.

For further reading, here are some resources where you will be able to follow up if this is of interest to you.

Resources:

Joyent Wiki How To Install
http://wiki.joyent.com/all-accelerators:kb:sge

Grid Engine Project Home
http://gridengine.sunsource.net/

Sun Grid Engine Wikipedia Entry
http://en.wikipedia.org/wiki/Sun_Grid_Engine