FriSem

Eric Schulz, Harvard University, Cognitive Scientist and Data Science Postdoctoral Fellow in Sam Gershman's Computational Cognitive Neuroscience Lab

Title: Using structure to explore efficiently

Abstract: Many types of intelligent behavior can be framed as a search problem, where an individual must explore a vast set of possible actions, while carefully balancing the exploration-exploitation dilemma. Under finite search horizons, optimal solutions are normally unobtainable. Yet humans and other animals regularly manage to solve these problems gracefully. How do they accomplish this? We propose that two simple principles can explain this behavior: generalization over features and uncertainty-guided exploration. Together these form a model that learns from past observations to generalize to similar options and seeks out uncertainty eagerly in order to gain more information about the search space. This model can be used to predict participant's search behavior in a complex multi-armed bandit task. Its parameter estimates can also be used to gain meaningful insights into developmental differences in generalization an directed exploration. Furthermore, this model opens up a theoretical bridge between traditional models of generalization and modern theories of generalization in neuroscience such as the Successor Representation. Finally, we can use this model to describe customers' purchasing decisions in a large-scale data set (1.6 million orders) of online food delivery purchases. I will end by describing ongoing work that puts this model to a test in a multi-armed bandit task with rats, where we find similar principles influencing animals' motor variability.