Publication: Job Allocation Strategies for Multiple High Performance Computing Clusters
All || By Area || By Year| Title | Job Allocation Strategies for Multiple High Performance Computing Clusters | Authors/Editors* | J. Qin and M. Bauer |
|---|---|
| Where published* | Cluster Computing |
| How published* | Journal |
| Year* | 2010 |
| Volume | |
| Number | |
| Pages | to appear |
| Publisher | |
| Keywords | resource management, job co-allocation, job scheduling, HPC clusters, performance evaluation |
| Link | |
| Abstract |
To more effectively use a network of high performance computing clusters, allocating multi-process jobs across multiple connected clusters becomes an attractive possibility. This allocation process entails dividing the processes of a job among several clusters, which we refer to as co-allocation. Co-allocation offers the possibility of more efficient use of computer resources, reduced turn-around time and computations using numbers of processes larger than processes on any single cluster. In order to realize these possibilities, effective co-allocation, ultimately, depends on the inter-cluster communication cost. In this paper, we introduce a scalable co-allocation strategy called the Maximum Bandwidth Adjacent cluster Set (MBAS) strategy. The strategy makes use of two thresholds to control allocation: one to control the limit on bandwidth on usable inter-cluster communication links and another to control how jobs are split. A simulator that can simulate the dynamic behavior of jobs running across multiple clusters was developed and used to examine the performance of the MBAS co-allocation strategy. Our results indicate that by adjusting the thresholds for link level control and chunk size control in splitting jobs, the MBAS co-allocation strategy can significantly improve both user satisfaction and system utilization. |
Back to page 28 of list