Impact of Workloads on Software Performance Modeling

This technical report investigates the effects of workload variation on the performance of configurable software systems. It presents a systematic empirical study involving 25,258 configurations from nine real-world software systems to understand how workload and configuration interact to influence performance. The research highlights that traditional performance models, which typically rely on single workload measurements, may not generalize effectively across different workloads. The findings indicate that workloads can induce significant performance variations and interact with configuration options in complex, often non-monotonous ways. This challenges existing assumptions in transfer-learning techniques used for performance prediction. The report emphasizes the necessity of considering workload characteristics when developing performance models to enhance their reliability and representativeness. Additionally, it discusses the implications of these interactions for performance estimation and the potential need for more comprehensive modeling approaches that account for workload variability.