DEVELOPMENT OF HYBRID MODEL FOR JOB SCHEDULING AND RESOURCE ALLOCATION IN COMPUTATIONAL CLUSTERS

Abstract

Enabling technologies in high speed communication and global process scheduling have pushed clusters of computers into the mainstream as general-purpose high performance computing systems. A cluster is a type of parallel or distributed computer system, which consists of a collection of inter-connected stand-alone computers working together as a single integrated computing resource. For effective and efficient utilization of cluster resources, there is the need to use a cluster manager that optimizes the way jobs are scheduled and computing resources are allocated so as to minimize waiting time of jobs as well as maximize the cluster resources. Thus, satisfying users and cluster owners. Existing researches have used several algorithms ranging from First-Come-First-Served (FCFS) Algorithm to Backfilling Algorithm. However, there are lots of resource gaps leading to inefficient usage of clusters. This research combines two favourable algorithms namely predictive scheduling algorithm and combinational scheduling algorithm to optimize usage of cluster resources. A detailed analysis of usage characteristics of an existing cluster was carried out, the characteristics were used to predict future resources utilization using Kalman filter theory. In addition, usage of cluster resources was further improved upon by applying combinational backfilling strategy. Result showed that jobs submitted to cluster nodes with hybrid model completed at the fastest rate and attained the least average job waiting time with highest utilization of resources when compared with FCFS Extensible Argonne Scheduling sYstem (EASY) backfilling algorithms. Remarkably, there was 56% efficiency and effectiveness of the hybrid model which was higher than the existing algorithms.