Java and the Sun Grid
Sun put the Sun Grid Compute Utility online this week. Good news for all of us with big compute jobs. Perfect for domains like music information retrieval where a single analysis pass on a large music collection could take years of CPU time. Instead of waiting years for the job to complete, the job can be put on the grid and completed in a weekend. Good Stuff.
I spent some time a few months ago figuring out what it would take to get our music analysis software to run on the grid. The main question I needed to answer was "What do I need to do to my big hunk of java code to get it to run well on the Sun Grid so that I could take advantage of all of resources of the Grid" The answer, I discovered, was that there were no Java APIs for writing software for the grid and getting my Java app to run on the grid was going to be a bit of work.
The good news is that there's a group in Sun that is trying to fix this, to make sure that Java works and works well on the Sun Grid. This week these folks have issued an early access release of the 'Compute Server' (catchy name, isn't it?). The goal for the compute server is to create an environment for Java programmers so that they can easily use the Sun Grid Compute utility. The compute server provides an extremely simple API for a distributing computation across a large number of CPUs and gathering feedback and results from the computation. The compute server is integrated with NetBeans, so deploying a job to the grid is part of the development work flow. Here's a diagram that shows how the process works:
Remember, this is an early access release. The goal is to get feedback from developers. You can download the early access compute server and look at the code (there's some good examples including a grid-enabled program that calculates the digits of PI in just a couple of hundred lines of code).
I've had the lucky opportunity to talk with the developers building the compute server and have been quite impressed. These are some really smart programmers with lots of experience building distributed systems in Java. I have high hopes for the compute server.