Learning Nonlinear Dynamics: Variational Autoencoders

Abstract

Differential Equations (PDEs) exhibiting behaviors over multiple time-scales arise in many physics and engineering problems. Capturing behaviors on a target time-scale can involve challenges in analysis and formulation of efficient numerical methods for simulation. As an alternative, we consider data-driven approaches based on recent developments in machine learning for capturing system behaviors. We develop reduced order models (ROMs), aiming to simplify high dimensional dynamics by learning a low dimensional representation of the relevant physics. This is done with Variational Autoencoders (VAEs), a machine learning algorithm which has demonstrated the ability to disentangle latent features in prior applications in speech recognition and image processing. Here we formulate VAEs for physical systems with inductive biases to learn good latent variables for capturing dynamic evolution and provide predictions of physical behaviors. We show how our methods work on Burgers' Equation, a non-linear PDE for fluid flow which exhibits shock waves. Comparisons are made between our learned features and those of analytic approaches. We discuss potential application areas where our machine learning methods can complement traditional approaches.