Hyperparameters employed by deep learning methods play a substantial role in the performance and reliability of these methods in practice. Unfortunately, finding performance-optimizing hyperparameter settings is a notoriously difficult task. Hyperparameter search methods typically have limited production-strength implementations or do not target scalability within a highly parallel machine, portability across different machines, experimental comparison between different methods, and tighter integration with workflow systems. In this talk, we present DeepHyper, a Python package that provides a common interface for the implementation and study of scalable hyperparameter search methods. It leverages an asynchronous model-based search methods that consist of sampling a small number of input hyperparameter configurations and progressively fitting surrogate models over the input-output space until exhausting a user-defined budget of evaluations. It leverages the Balsam workflow system to hide the complexities of running large numbers of hyperparameter configurations in parallel on high-performance computing systems.

Speaker

Prasanna Balaprakash, Argonne National Laboratory