An Experimental Performance Evaluation of Autoscaling Algorithms for Complex Workflows

Simplifying the task of resource management and scheduling for customers, while still delivering complex Quality-of-Service (QoS), is key to cloud computing. Many autoscaling policies have been proposed in the past decade to decide on behalf of cloud customers when and how to use the elastic features of clouds. However, in prior work many of these policies are not compared to each other, and instead are often compared only to static provisioning or to a predefined QoS target. This reduces the ability of cloud customers and of cloud operators to choose and deploy a suitable autoscaling policy. In our work, we conduct an experimental performance evaluation of autoscaling policies, using as application model workflows, a commonly used formalism for automating resource management for applications with well-defined yet complex structure. We present a detailed comparative study of general, state-of-the-art, generic autoscaling policies, along with two new workflow-specific auto- scalers. To understand which policy is the best, we also conduct various forms of pairwise and group comparisons, and report both individual and aggregated metrics.