The brain outside the lab: Exploring the neural basis of long-term, naturalistic human behaviors*

Much of our knowledge about neural computation in humans has been informed by data collected through carefully controlled experiments in laboratory settings, but understanding the brain in action requires exploration of large, naturalistic neural data recorded outside structured tasks. I will talk about our work analyzing neural processing in long-term brain recordings acquired in a task-free, naturalistic setting. Our data set consists of large-scale human intracranial brain recordings (ECoG) augmented with video, audio and depth camera recordings, all simultaneously and continuously monitoring a subject over several days to weeks. Importantly, unlike the majority of previous neural data used to train neural decoders, here the subjects are not instructed to perform specific tasks but are simply behaving as they wish in their hospital rooms. Motivated by the size of the dataset and substantial variety between individual subjects, our scalable computational approach circumvents tedious manual annotation and fine tuning of parameters. I will talk about data-driven models we developed to explore this long-term, naturalistic data using perspectives from dynamical systems, unsupervised clustering, and multimodal deep neural networks.