Modeling the Physical World: Perception, Learning, and Control

NeurIPS Workshop, December 7, 2018, Montreal, Canada

News: Slides are now available.

Invited Talks

Peter Battaglia (DeepMind) Jeannette Bohg (Stanford) Chelsea Finn (Google/UCB) Leslie Kaelbling
Zico Kolter
Jitendra Malik (FAIR/UCB) Emo Todorov
Dan Yamins


Despite recent progress, AI is still far from achieving common-sense scene understanding and reasoning. A core component of this common sense is a useful representation of the physical world and its dynamics that can be used to predict and plan based on how objects interact. This capability is universal in adults, and is found to a certain extent even in infants. Yet despite increasing interest in the phenomenon in recent years, there are currently no models that exhibit the robustness and flexibility of human physical reasoning.

There have been many ways of conceptualizing models of physics, each with their complementary strengths and weaknesses. For instance, traditional physical simulation engines have typically used symbolic or analytic systems with “built-in” knowledge of physics, while recent connectionist methods have demonstrated the capability to learn approximate, differentiable system dynamics. While more precise, symbolic models of physics might be useful for long-term prediction and physical inference; approximate, differentiable models might be more practical for inverse dynamics and system identification. The design of a physical dynamics model fundamentally affects the ways in which that model can, and should, be used.

This workshop will bring together researchers in machine learning, computer vision, robotics, computational neuroscience, and cognitive psychology to discuss artificial systems that capture or model the physical world. It will also explore the cognitive foundations of physical representations, their interaction with perception, and their applications in planning and control. There will be invited talks from world leaders in the fields, presentations and poster sessions based on contributed papers, and a panel discussion.


Jiajun Wu Kelsey Allen Kevin Smith Jessica Hamrick
Emmanuel Dupoux Marc Toussaint Joshua Tenenbaum