Stochastic optimization on Riemannian manifolds

Fong, Robert Simon (2020). Stochastic optimization on Riemannian manifolds. University of Birmingham. Ph.D.

Fong2020PhD.pdf
Text - Accepted Version
Restricted to Repository staff only until 31 December 2026.
Available under License All rights reserved.
Download (1MB) | Request a copy

Abstract

Recent developments of geometry in information theory gave rise to two contemporary branches of optimization theories: \textit{optimization over Riemannian manifolds} and the \textit{Information Geometric interpretation of stochastic optimization over Euclidean spaces}. Inspired by the geometrical insights of these two emerging fields of optimization theory, this thesis studies Stochastic Derivative-Free Optimization (SDFO) algorithms over Riemannian manifolds from a geometrical perspective.

We begin our discussions by investigating the Information (statistical) Geometrical structure of probability densities over Riemannian manifolds. This establishes a geometrical framework for Riemannian SDFO algorithms incorporating both the statistical geometry of the decision space and the Riemannian geometry of the search space.

Equipped with the geometrical framework, we return to optimization methods over Riemannian manifolds. Riemannian adaptations of SDFO in the literature use search information generated within the normal neighbourhoods around search iterates. While this is natural due to the local Euclidean property of Riemannian manifolds, the search information remains local. We address this restriction using only the intrinsic geometry of Riemannian manifolds. In particular, we construct an evolving sampling mixture distribution for generating non-local search populations on the manifold, which is done using the aforementioned geometrical framework.

We propose a generalized framework for adapting SDFO algorithms on Euclidean spaces to Riemannian manifolds, which encompasses known methods such as Riemannian Covariance Matrix Adaptation Evolutionary Strategies (Riemannian CMA-ES). We formulate and propose a novel algorithm -- Extended RSDFO. The Extended RSDFO is compared to Riemannian Trust-Region method, Riemannian CMA-ES and Riemannian Particle Swarm Optimization in a set of multi-modal optimization problems over a variety of Riemannian manifolds.

Type of Work:

Thesis (Doctorates > Ph.D.)

Award Type:

Doctorates > Ph.D.

Supervisor(s):