WONDERS: Integrating WOW, PONDER, and SCALE for Enhanced Scheduling Performance — Unleashing Data-Aware and Resource-Sizing Synergies

Zusammenfassung

Scientific workflows are an important tool for new discoveries. Recent Scientific Workflow Management Systems (SWMSs), such as Nextflow, put a huge focus on the portability of workflows. Portability encompasses replacing both the target infrastructure and the input dataset.
The more portable systems become, the more the importance of automatic adaptation and optimization increases. Extensive research is conducted in the field of automatically adapting workflows. Strategies to optimize the execution of scientific workflows are often evaluated in simulations, and only for the individual proposed strategy. Accordingly, it is unclear how strategies with different goals affect each other.
In this work, we fuse three strategies to optimize workflow execution. First, WOW, an approach that focuses on location-aware scheduling. Second, PONDER, an approach that predicts task memory consumption and sizes the tasks accordingly. And, third, SCALE, an approach to predict task CPU and size it accordingly. We test all three approaches together and investigate their synergies. Our results show that the whole is greater than the sum of its parts. When combined into WONDERS, WOW, PONDER, and SCALE, the reduction in makespan increases by up to 67.4% more than the sum of their individual decreases.