Tutorial Speakers

Dr. Nicolas Heess
Research Scientist at DeepMind, London

Deep reinforcement learning for control - algorithms and architectures
Reinforcement learning algorithms in combination with powerful function approximators such as neural networks have achieved a number of impressive results in a number of challenging domains such as Atari games, Starcraft, chess, or Go. The growing interest in (deep) RL has led to significant progress but also to a large space of algorithms to choose from. In my talk I will review some of the principles underlying modern policy search methods and discuss several of the ones that are widely used in practice, with a particular focus on algorithms suitable for high-dimensional continuous action spaces. I will also discuss some of the special challenges arising when RL is applied to motor control tasks and present some applications especially to the control of simulated physical characters.

Deep reinforcement learning for control - algorithms and architectures

Reinforcement learning algorithms in combination with powerful function approximators such as neural networks have achieved a number of impressive results in a number of challenging domains such as Atari games, Starcraft, chess, or Go. The growing interest in (deep) RL has led to significant progress but also to a large space of algorithms to choose from. In my talk I will review some of the principles underlying modern policy search methods and discuss several of the ones that are widely used in practice, with a particular focus on algorithms suitable for high-dimensional continuous action spaces. I will also discuss some of the special challenges arising when RL is applied to motor control tasks and present some applications especially to the control of simulated physical characters.

Biography
Nicolas Heess is a Research Scientist at DeepMind, London. He is interested in questions related to artificial intelligence and machine learning, perception, motor control, and robotics. One of his long-term goals is to develop algorithms and architectures that enable embodied agents to learn to intelligently reason about and interact with their physical environment and other agents. He has worked on the theory and applications of reinforcement learning and control, unsupervised learning, probabilistic models, and inference. His current research focuses on the application of these methods at the intersection of perception and control with a special interest in the acquisition, representation and adaptation of sensorimotor skills. Prior to joining DeepMind Nicolas was a postdoctoral researcher at the Gatsby Unit (UCL) working with Yee Whye Teh and David Silver. He did his PhD under the supervision of Chris Williams at the University of Edinburgh and also paid several extended visits to Microsoft Research (Cambridge, UK) where he worked with John Winn and others.

Biography

Nicolas Heess is a Research Scientist at DeepMind, London. He is interested in questions related to artificial intelligence and machine learning, perception, motor control, and robotics. One of his long-term goals is to develop algorithms and architectures that enable embodied agents to learn to intelligently reason about and interact with their physical environment and other agents. He has worked on the theory and applications of reinforcement learning and control, unsupervised learning, probabilistic models, and inference. His current research focuses on the application of these methods at the intersection of perception and control with a special interest in the acquisition, representation and adaptation of sensorimotor skills. Prior to joining DeepMind Nicolas was a postdoctoral researcher at the Gatsby Unit (UCL) working with Yee Whye Teh and David Silver. He did his PhD under the supervision of Chris Williams at the University of Edinburgh and also paid several extended visits to Microsoft Research (Cambridge, UK) where he worked with John Winn and others.

^ Back to Top