Learning intuitive physics: a computational investigation

Abstract

Intuitive physics is the ability to reason about three-dimensional objects, their dynamic interactions, and responses to forces. The development of intuitive physical knowledge in humans has been extensively catalogued by developmental psychologists in order to study the "origins of knowledge." It is at the center of a debate, parallel to that of language, between nativists and empiricists. Nativists have demonstrated evidence of physical intuition as early as 2.5 months of age (by some accounts even younger) suggesting domain-specific constraints on learning. Empiricists have shown (for some domains) that general-purpose learning principles, instantiated via artificial neural networks, are capable of producing the stage-like progression of rule-like behavior seen in infants. However, methodological limitations abound on both sides, impeding progress on this dialog. Nativists have experimental limits on probing newborns and young infants due to perceptual and attentional constraints. Empiricists have only been able to build learning systems for gross simplifications of the physical world, abstracting sensory information to physical quantities and learning based on that. The main contribution of this thesis is to capitalize on advances in deep learning to build models that can actually learn intuitive physics from visual data. In doing so, this work unlocks a necessary component for mapping the mechanisms of intuitive physical knowledge: any model that seeks to provide a mechanistic account of learning phenomenon must first be able to learn that phenomenon. We adopt the Violation-of-Expectation paradigm used in developmental psychology to probe infant physical intuition and build a synthetic dataset to probe physical knowledge in deep neural networks. In Chapter 2, we build a model with a domain-general architecture and find it only capable of narrow generalization of intuitive physics knowledge. In Chapter 3, we build a model with an object-centric inductive bias (reflecting hypothesized domain-specific constraints) and find it is capable of learning generalized physical concepts with as little as 28 hours of visual experience in our synthetic 3D environment. Furthermore, we ablate the architecture and find the object-centric components are necessary for the rich generalization. Finally, Chapter 4 briefly elaborates potential mechanisms for learning necessary components of the object-centric model: object discovery and tracking.