On Wednesday, September 18th, 2019 as part of IBM Research's
AI week we will be hosting the first workshop on Neuro-Symbolic Computation and Machine Common Sense (NSC-MCS).
Neuro-symbolic methods are relatively new in the DL community and at present, they are largely driven by the findings at the intersection of the fields of child-psychology, generative modelling and neuroscience suggesting symbol manipulation may be at the core of human common sense. This workshop will bring researchers from the aforementioned fields together with the highly diverse set of researchers in the wider fields of representation learning and reasoning in order to advance discussion and promote collaborations across its various subdomains, with a special focus on Neuro-symbolic AI and Neuro-Symbolic Computing and Machine Common Sense.
Schedule
9:00-9:15 AM
Welcome and Opening Remarks
09:15-09:45 AM
Invited talk: T.S. Jayram "Can External Memory Help in Visual and Symbolic Reasoning?"
09:45-10:15 AM
Invited talk: Tomer Ullman "Computational models of core intuitive physics (and some psychology)"
Daniel Yamins is a cognitive computational neuroscientist at Stanford University, where he's an assistant professor of Psychology and Computer Science,
a faculty scholar at the Wu Tsai Neurosciences Institute, and an affiliate of the Stanford Artificial Intelligence Laboratory.
His research group focuses on reverse engineering the algorithms of human cognition, both to learn both about how our brains work and build more effective
artificial intelligence systems. He is especially interested in how brain circuits for sensory information processing and decision making arise via
the optimization of high-performing cortical algorithms for key behavioral tasks. He received his AB and PhD degrees from Harvard University,
was a postdoctoral research at MIT, and has been a visiting researcher at Princeton University and Los Alamos National Laboratory.
He is a recipient of an NSF Career Award, the James S. McDonnell Foundation award in Understanding Human Cognition, the Sloan Research Fellowship,
and is a Simons Foundation Investigator.
Rebecca Saxe is an associate investigator of the McGovern Institute and the John W. Jarve (1978) Professor in Brain and Cognitive Sciences at MIT.
She obtained her Ph.D. from MIT and was a Harvard Junior Fellow before joining the MIT faculty in 2006. She was awarded tenure in 2011.
Saxe was chosen In 2012 as a Young Global Leader by the World Economic Forum, and she received the 2014 Troland Award from the National Academy of Sciences.
Rebecca Saxe studies human social cognition, using a combination of behavioral testing and brain imaging technologies.
Kate Saenko is an Associate Professor of Computer Science at Boston University and a consulting professor for the MIT-IBM Watson AI Lab.
She leads the Computer Vision and Learning Group at BU, is the founder and co-director of the Artificial Intelligence Research (AIR) initiative,
and member of the Image and Video Computing research group. Kate received a PhD from MIT and did her postdoctoral training at UC Berkeley and Harvard.
Her research interests are in the broad area of Artificial Intelligence with a focus on dataset bias, adaptive machine learning, learning for image and
language understanding, and deep learning.
Josh Tenenbaum is a professor of Computational Cognitive Science in the Department of Brain and Cognitive Sciences at MIT, and a member of the Computer Science and Artificial Intelligence Laboratory (CSAIL).
He studies learning, reasoning and perception in humans and machines, with the twin goals of understanding human intelligence in computational terms and bringing computers closer to human capacities.
T.S. Jayram (Jayram Thathachar) is a research scientist in the Machine Intelligence Team at IBM Research AI. His work in deep learning focuses on cognitively inspired memory-augmented neural network architectures with
a view towards visually grounded language learning and reasoning with memory. This was inspired partly by his past work on the role of memory in theoretical computer science where he has made fundamental
contributions to both algorithms in data science as well as demonstrating their limitations using information theory. For these contributions, which includes starting two new areas namely, Information
Complexity in theoretical computer science and Index Coding in information theory, he was awarded the IBM Research Accomplishment (Outstanding) in the Science category (along with David Woodruff) in 2012.
He was also an invited speaker at the ACM PODS conference in 2010. In the past he has worked on time-space tradeoffs for branching programs; part of this work won the Machtey Award in FOCS 1994.
Akash Srivastava is a research scientist at the MIT-IBM lab in Cambridge, MA where he works on building machines with child-like common-sense and
intuitive physics using probabilistic modelling and Bayesian inference. Before this, he was a PhD student in the Informatics Forum at
University of Edinburgh where he worked with Dr Charles Sutton and Dr Michael U. Gutmann on variational inference for generative models using deep learning.
Tomer Ullman is a cognitive scientist interested in common-sense reasoning, and building computational models for explaining high-level cognitive processes
and the acquisition of new knowledge by children and adults. In particular, he is focused on how children and adults come to form intuitive theories of agents
and objects, and providing both a functional and algorithmic account of how these theories are learned.
Tomer D. Ullman is a postdoctoral associate in the Computational Cognitive Science group at MIT, and the Lab for Developmental Studies at Harvard.
He studies people's common-sense reasoning about physics and psychology, using computational models and behavioral experiments with children and adults.
His research is funded by the Center for Brains, Minds and Machines.
Jiajun Wu is a Ph.D. student in Electrical Engineering and Computer Science at Massachusetts Institute of Technology.
He received his B.Eng. from Tsinghua University in 2014. His research interests lie in the intersection of computer vision, machine learning, robotics,
and computational cognitive science.
His research has been recognized through the IROS Best Paper Award on Cognitive Robotics and fellowships from Facebook, Nvidia, Samsung, Baidu, and Adobe,
and his work has been covered by major media outlets including CNN, BBC, WIRED, and MIT Tech Review.
ThreeDWorld Tutorial
Researching Intelligence using Virtual Worlds
ThreeDWorld (TDW) is a simulation framework that uses state-of-the-art video game technology to generate photo-realistic scenes in a virtual world. TDW’s flexible, general design allows researchers to collect next-generation experimental data and generate large-scale datasets for training AI systems with complete control over data generation and full access to all associated generative parameters.
Tutors
Seth Alter
Seth Alter is the back-end programmer and API developer for TDW. He is an experienced game designer and developer, specializing in python, Unity 3D development and procedural content creation. He is also the co-organizer of Boston Indies (a local monthly meetup group for independent game developers).
Jeremy Schwartz
Jeremy Schwartz is the Project Lead for TDW. He is a veteran interactive technology developer and CGI production manager,
who has worked for many years in the video game industry. While at Electronic Arts and Acclaim Entertainment he ran studio
operations for motion capture and audio/video post-production, and managed 3D animation and tool-development teams. Jeremy
also spent several years as a technology analyst with Forrester Research, and has worked for design consultancies and
interactive agencies developing both digital and hybrid physical/digital interactive experiences.
ThreeDWorld Use Cases
Use Case I
The aim of our project is to understand the underlying neural computations involved in intuitive physics, i.e., common sense
about physical interactions. We present video stimuli of physical events unfolding over time to rhesus macaques and compare
the brain response (with fMRI and fMRI-guided electrophysiology) to intentional interactions versus purely physical interactions.
Crucially, by using TDW to create parametric animations we will achieve unprecedented control over relevant physical properties
(like magnitude and direction of force), which we can subsequently decode from neural activity using computational models.
Speaker
Ronald Alvarez
Ronald is an Electrical Engineering and Mathematics student at Mercer University. He works on developing computational
models of planning and decision-making with the Tenenbaum lab at MIT, investigating neural correlates of intuitive physics
with the Freiwald lab at Rockefeller University, and control theory with the Thitsa lab at Mercer.
Use Case II
Computer Vision researchers have limited control over various dimensions such as object size, angle, etc. in
large-scale photorealistic datasets. Using TDW, we can generate extremely flexible and fully controlled datasets
that allow for training models with transfer performances approaching those of photographic datasets such as ImageNet.
Speaker
Martin Schrimpf
Martin completed his Bachelor and Master degrees in Software Engineering in Germany at TUM, LMU and UNA,
conducting his Master’s Thesis on recurrent visual processing at Harvard with Gabriel Kreiman.
He has founded the non-profit startup Integreat and worked at Salesforce Research on deep learning for NLP
with Richard Socher before joining MIT’s Brain and Cognitive Sciences program for his PhD. There, he is
advised by James DiCarlo and works on evaluating and building neural network models of brain processing.
Use Case III
Humans can readily interpret the sounds made when objects collide,
scrape, and clatter against each other. We know little about how humans
can make such inferences, nor can we replicate their success with
machine hearing algorithms. We are building a perceptually inspired
generative model of contact sounds, which can be interfaced with TDW to
simulate sounds made by everyday objects. We can use these sounds to
study human perception of natural sounds.
Speaker
James Traer
James was formally trained as a physicist, and during his PhD work
(Univ. of California, San Diego) he studied sound generation by ocean
waves and developed algorithms to map the ocean and the Earth's crust
with ambient sound. In 2013 he joined the
McDermott Laboratory for Computational Audition (MIT) as a
post-doctoral researcher. He has since studied how humans infer physical
attributes of the world (e.g. source distance, room size, object
material, force of impact) from the sounds made by everyday objects as
they collide, scrape and deform.
Registration
Registration to the Neuro-Symbolic Computing and Machine Common Sense workshop is open!
In order to register please use Eventbride website.
