An Approach to Optimise Resource Provision with Energy-awareness in Datacentres by Combating Task Heterogeneity

IEEE Cloud workloads are increasingly heterogeneous such that a single Cloud job may encompass one to several tasks, and tasks belonging to the same job may behave distinctively during their actual execution. This inherent task heterogeneity imposes increased complexities in achieving an energy efficient management of the Cloud jobs. The phenomenon of a few proportions of tasks characterising increased resource intensity within a given job usually lead the providers to over-provision all the encompassed tasks, resulting in majority of the tasks incurring an increased proportions of resource idleness. To this end, this paper proposes a novel analytics framework which integrates a resource estimation module to estimate the resource requirements of tasks a priori, a straggler classification module to classify tasks based on their resource intensity, and a resource optimisation module to optimise the level of resource provision depending on the task nature and various runtime factors. Performance evaluations conducted both theoretically and through practical experiments prove that the proposed methodology performs better than the compared statistical resource estimation methods and existing models of straggler mitigation, and further demonstrate the effectiveness of the proposed methodology in achieving energy conservation by postulating appropriate level of resource provisioning for task execution.