A hybrid physics informed neural network model for patient specific phonation simulation

This research aims to develop a new AI-enabled data-assimilation computational framework that enables seamless integration of multimodal experimental/clinical data and high-fidelity subject-specific modeling of human/animal vocal systems to provide accurate, realistic, robust, efficient and reliable simulations of individual vocal system. Toward this, we designed a novel hybrid physics informed neural network(PINN) based differentiable learning algorithm that integrates a recurrent neural network model of 3D continuum soft tissue with a differentiable fluid solver to infer the 3D flow-induced vocal dynamics and other physical quantities from high speed  videoendoscopy. The effectiveness and merit of the proposed algorithm is demonstrated in subject-specific voice production problems by using synthetic data from a canine VF model and in-vivo experimental data of pigeon VFs.  Results revealed that the algorithm successfully reconstructed the full three-dimensional motion of vocal fold, as well as estimation of other features such as flow rate and acoustic signals, which are difficult to be measured experimentally.