We propose trust-region and direct-search frameworks for potentially large-scale stochastic derivative-free optimization by introducing the STARS and StoDARS algorithms. While STARS achieves scalability by minimizing random models that approximate the objective in low-dimensional affine subspaces thus significantly reducing per-iteration costs in terms of function evaluations, these goals are achieved by StoDARS through the exploration of the decision space by means of poll directions generated in random subspaces. These subspaces and their dimension are chosen via Johnson--Lindenstrauss transforms such as those obtained from Haar-distributed orthogonal random matrices. The quality of the subspaces and sets of poll directions, as well as the accuracies of estimates and models used by the algorithms are required to hold with sufficiently high, but fixed, probabilities. Convergence and complexity results are obtained for both methods using martingale theory. In particular, by leveraging the ability of StoDARS to generate a dense set of poll directions, its almost sure convergence to Clarke stationary points is established. Moreover, the analysis of second-order behavior of the well-known mesh adaptive direct-search algorithms using a second-order-like extension of the Rademacher's theorem-based definition of the Clarke subdifferential (so-called generalized Hessian) is extended to the StoDARS framework, making it the first in a stochastic direct-search setting to the best of our knowledge.

Bio: Kwassi Joseph Dzahini joined Argonne National Laboratory (USA) in 2022 as a postdoctoral appointee. Before receiving his Ph.D. in applied mathematics from the Department of Mathematics and Industrial Engineering at Polytechnique Montréal (Canada), he obtained his M.Sc. in applied mathematics from Lille 1 University of Science and Technology (France) and his B.Sc. in mathematics from Université de Lomé (Togo). For his Master's studies in France, Kwassi Joseph Dzahini was awarded two scholarships by the "Laboratory of Excellence -- European Centre for Mathematics, Physics and their Interactions". During his PhD in Canada, he was also awarded the highly competitive scholarship for doctoral studies, by "Fonds de Recherche du Québec - Nature et Technologies" (FRQNT). Most recently, he was awarded the best poster prize at the 2023 SIAM conference on computational science and engineering in The Netherlands. During his Ph.D., he worked on direct-search methods for stochastic blackbox optimization. Initially supervised at Argonne by Stefan Wild, he is currently working under the supervision of Paul Hovland.

See all upcoming talks at https://www.anl.gov/mcs/lans-seminars