Customer

In March 2000, Blackstone Computing (now Argentys) of Worcester, Massachusetts approached Team345 to help them build a high-performance Unix/Linux clustering system for accelerating applications for biotech and EDA customers. Clustering systems allow a racked set of low-cost computers (nodes) running Linux, Sun Solaris™ or another Unix-like OS to work together to perform tasks in supercomputer-like speeds. Blackstone needed a highly scalable (up to 1000 cluster nodes), high performance, open system that would provide enterprise-wide scheduling capabilities, strong administration and management reporting features, and allow for easy integration of biotech and EDA data and applications.

Solution

A highly skilled Team345 team of engineers designed and developed a cluster solution for Blackstone that easily met their requirements. The solution was a three-tiered Java™ J2EE system that made extensive use of the model-view-controller and observer patterns. The middle or business tier was implemented with EJBs using Borland's J2EE AppServer running on a dedicated portal machine. Entity beans were used to access data objects that changed infrequently or not at all. A custom transaction-aware in-memory-only database was designed and implemented to allow high-throughput for changes in frequently accessed data objects. Session beans wrapped all modification access to the entity beans and in-memory database. The data tier made use of either Oracle or PostgreSQL. While those were the only two databases tested with the system, it was designed to allow any database with a JDBC level 4 driver to be substituted with little additional effort.

The presentation tier had several facets: First, a management console provided a gui via a Java Swing based interface for easy monitoring, reporting and modification of cluster settings. Second, a simple command-line interface provided a set of quick and easy tools for knowledgeable users. Several other server components were developed to form a cohesive cluster solution. First, a resource-based user/group-allocatable scheduler for scheduling tasks across cluster nodes was essential to the system. The scheduler component used a complex scheduling algorithm to allocate tasks across cluster nodes based on resource availability and user/group cluster allocation. Second, an XML IP socket-based open interface (very similar to a webservices interface) was developed that facilitated communication with non-Java third-party applications and resource managers. Third, a cluster node "agent" or daemon was created that controls jobs executing on a given cluster node and reports utilization statistics on the given node back to the business tier.

Event notification was a large component of the architected system. The presentation layer, and many back-end server components needed close to realtime notification when cluster data objects changed state. A system based on the Observer pattern was developed, using the multicast Java package created by Team345 as its foundation, that allowed the middle business tier to multicast packets of data indicating what changed. Observers such as the management console, the scheduler, XML interface and the command-line tools receive desired packets without having to establish a point-to-point connection with the business tier server - leaving the server free to perform tasks necessary to keep the cluster running quickly and smoothly, and freeing both the applications and the server from resource intensive polling-based queries.

Results

The system was named PowerCloud™, and the first publicly available version was released in November of 2001. The system was developed for and tested on RedHat™ Linux and Sun Solaris for the business and data tiers, and RedHat Linux, Sun Solaris and Microsoft Windows™ for the presentation layer. Enhancements and upgrades to the original system continued since that time with the release of a web-based interface for submitting and monitoring jobs, and culminating in version 3 of the product released in October of 2002. Blackstone's customers have enjoyed significantly higher throughput with their scientific applications along with vastly better administration and monitoring as a result of PowerCloud. Up to a 20-fold increase in throughput in bioinformatics applications has been demonstrated in actual customer environments. Team345 has provided ongoing services to Blackstone since March of 2000.

Solaris is a registered trademark of Sun Microsystems, Inc.; Windows is a registered trademark of Microsoft, Inc.; RedHat is a registered trademark of RedHat, Inc.; PowerCloud is a registered trademark of Argentys.
Java and all Java-based marks are trademarks or registered trademarks of Sun Microsystems, Inc. in the U.S. and other countries