Inertial Methods with Viscous and Hessian driven Damping for Non-Convex Optimization - GREYC image
Pré-Publication, Document De Travail Année : 2024

Inertial Methods with Viscous and Hessian driven Damping for Non-Convex Optimization

Résumé

In this paper, we aim to study non-convex minimization problems via second-order (in-time) dynamics, including a non-vanishing viscous damping and a geometric Hessian-driven damping. Second-order systems that only rely on a viscous damping may suffer from oscillation problems towards the minima, while the inclusion of a Hessian-driven damping term is known to reduce this effect without explicit construction of the Hessian in practice. There are essentially two ways to introduce the Hessian-driven damping term: explicitly or implicitly. For each setting, we provide conditions on the damping coefficients to ensure convergence of the gradient towards zero. Moreover, if the objective function is definable, we show global convergence of the trajectory towards a critical point as well as convergence rates. Besides, in the autonomous case, if the objective function is Morse, we conclude that the trajectory converges to a local minimum of the objective for almost all initializations. We also study algorithmic schemes for both dynamics and prove all the previous properties in the discrete setting under proper choice of the step-size.
Fichier principal
Vignette du fichier
Convergence_ISEHD_ISIHD_nonconvex (25).pdf (1.04 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04651420 , version 1 (17-07-2024)
hal-04651420 , version 2 (18-07-2024)
hal-04651420 , version 3 (22-07-2024)

Identifiants

  • HAL Id : hal-04651420 , version 3

Citer

Rodrigo Maulen-Soto, Jalal Fadili, Peter Ochs. Inertial Methods with Viscous and Hessian driven Damping for Non-Convex Optimization. 2024. ⟨hal-04651420v3⟩
65 Consultations
28 Téléchargements

Partager

More