Adjoint-accelerated Bayesian Inference

Bayesian Inference provides a probabilistic framework that is well suited to Machine Learning of model parameters from data. We specify any number of candidate models, their parameters, and their prior probability distributions. When data arrives, we calculate (i) the most likely parameter values, (ii) their posterior probability distributions, (iii) the marginal likelihood of each model. This combines how well each model fits the data with how much each parameter space collapses when the data arrive. This penalizes (i) models that do not fit the data and (ii) models that fit the data but whose parameters require excessively delicate tuning to do so. 

 

Bayesian inference is usually prohibitively expensive, but its cost is greatly reduced if all distributions are taken to be Gaussian. This is often reasonable and can always be checked a posteriori. This allows the optimal parameter values to be found cheaply with gradient-based optimization and their posterior uncertainties and marginal likelihoods to be calculated instantly with Laplace's method. This requires calculation of the gradients of each model's outputs with respect to its parameters, which is achieved cheaply with adjoint methods at first and (optionally) second order. 

 

I will outline Bayesian inference, Laplace's method, the acceleration due to adjoint methods, and Bayesian experimental design. I will demonstrate this with assimilation of 3D Flow-MRI data, model selection in thermoacoustics, and Bayesian identification of nonlinear dynamics.