Generative Adversarial Networks (GANs) are a generative framework with a notorious reputation for instability. Despite significant work in attempting to improve stability, training remains extremely difficult in practice. Nearly all GAN optimization methods are built on either simultaneous (Sim-GDA) or alternating (Alt-GDA) gradient descent-ascent, where the generator and discriminator are updated either at the same time iteratively or in a fixed pattern. In this paper, we prove for simple GANs, for which training had been proven non-convergent under Sim-GDA and Alt-GDA, that our newly introduced training method is Lyapunov-stable. We then design a novel oracle-guided GDA training strategy called Dynamic-GDA that leverages generalized analogs of the properties exhibited in the simple case. We also prove that in contrast to Sim/Alt-GDA, GANs with Dynamic-GDA achieve Lyapunov-stable training with non-infinitesimal learning rates. Empirically, we show Dynamic-GDA improves convergence orthogonally to common stabilizing techniques on 8 classes of GAN models and 7 different data sets.

Creator

• DeOliveira, Joshua

Contributors

• Mangoubi, Oren
• Rundensteiner, Elke A.

Degree

• MS

Unit

• Data Science

Publisher

• Worcester Polytechnic Institute

Identifier

• etd-121894

Keyword

• Deep Learning
• GAN
• Optimization
• Gradient Descent-Ascent

Advisor

• Rundensteiner, Elke A.

Defense date

• 2023-10-05

Year

• 2024

Date created

• 2024-04-26

Resource type

• Thesis

Source

• etd-121894

Rights statement

• In Copyright