Keywords: Parallelization, Resource-Aware Scheduling, Hyperparameter Tuning, Embedded Systems
Webpages:
http://sfb876.tu-dortmund.de/SPP/sfb876-a3.html We present a resource-aware scheduling strategy for parallelizing
R applications on heterogeneous architectures, like those commonly found in mobile devices. Such devices typically consist of different processors with different frequencies and memory sizes, and are characterized by tight resource and energy restrictions. Similar to the
parallel package that is part of the
R distribution, we target problems that can be decomposed into independent tasks that are then processed in parallel. However, as the
parallel package is not resource-aware and does not support heterogeneous architectures, it is ill-suited for the kinds of systems we are considering.
The application we are focusing on is parameter tuning of machine learning algorithms. In this scenario, the execution time of an evaluation of a parameter configuration can vary heavily depending on the configuration and the underlying architecture. Key to our approach is a regression model that estimates the execution time of a task for each available processor type based on previous evaluations. In combination with a scheduler allowing to allocate tasks to specific processors, we thus enable efficient resource-aware parallel scheduling to optimize the overall execution time.
We demonstrate the effectiveness of our approach in a series of examples targeting the ARM big.LITTLE architecture, an architecture commonly found in mobile phones.
References ARM. 2017. “big.LITTLE Technology.”
https://www.arm.com/products/processors/technologies/biglittleprocessing.php.
Helena Kotthaus, Ingo Korb. 2017. “TraceR: Profiling Tool for the R Language.” Department of Computer Science 12, TU Dortmund University.
https://github.com/allr/traceR-installer.
Kotthaus, Helena, Ingo Korb, and Peter Marwedel. 2015. “Performance Analysis for Parallel R Programs: Towards Efficient Resource Utilization.” 01/2015. Department of Computer Science 12, TU Dortmund University.
Richter, Jakob, Helena Kotthaus, Bernd Bischl, Peter Marwedel, Jörg Rahnenführer, and Michel Lang. 2016. “Faster Model-Based Optimization Through Resource-Aware Scheduling Strategies.” In
LION10, 267–73. Springer International Publishing.