A Multi-platform Job Submission System for Epidemiological Simulation Models

Abstract

In the current era of computing, the emergence of middle-ware software has resulted into a high level of software abstraction which has contributed to the diffusion of distributed computing. Distributed frameworks are systems comprised of many computing resources that are coupled together to perform one or more tasks. In a scientific grid-like environment, there are many different applications which require vast computing resources for simulation, execution, or analysis. Some of the applications require faster results, while other rely on more resources for execution. A configurable job submission system helps in distributing the tasks among the resources depending on their preference, priority and usage in a distributed environment.

We implemented a job submission system using JavaSpace which can be used for simulating jobs according to their priority. Two different types of brokers are used to monitor the JavaSpace. These brokers can either be used to compute results for tasks that require faster results using binaries installed on a stand-alone system or they can be used to submit the task to a BOINC system for computation. The flexibility to submit tasks as per their priority would help users to get faster results. Upon completion of the tasks, the brokers updates the jobs and transfers the results for further processing thus completing a cycle of retrieving a job, computing the data and transferring the results back to the user.

The generic nature of the framework makes it very simple to add new services which can perform a variety of tasks, making the system highly modular and easily extensible. Multiple brokers doing the same or different tasks can run on the same or different systems allowing the users to make efficient use of their resources. The brokers can be configured to detect the existing system and start monitoring jobs of different types. The framework can be used to transfer the results or detect any failures while execution and report it back to the user. The simple modular design and strong high-level JavaSpace API make it very easy to develop new systems for the framework.