Autonomous learning has been a promising direction in control and
robotics for more than a decade since learning models and controllers
from data allows us to reduce the amount of engineering knowledge that
is otherwise required. Due to their flexibility, autonomous
reinforcement learning (RL) approaches typically require many
interactions with the system to learn controllers. However, in real
systems, such as robots, many interactions can be impractical and time
consuming. To address this problem, current learning approaches
typically require task-specific knowledge in form of expert
demonstrations, pre-shaped policies, or specific knowledge about the
underlying dynamics. In the first part of the talk, we follow a different approach and speed
up learning by efficiently extracting information from sparse data. In
particular, we learn a probabilistic, non-parametric Gaussian process
dynamics model. By explicitly incorporating model uncertainty into
long-term planning and controller learning our approach reduces the
effects of model errors, a key problem in model-based learning. Compared
to state-of-the art RL our model-based policy search method achieves
an unprecedented speed of learning. We demonstrate its applicability to
autonomous learning in real robot and control tasks. In the second part of my talk, we will discuss an alternative method for
learning controllers based on Bayesian Optimization, where it is no
longer possible to learn models of the underlying dynamics. We
successfully applied Bayesian optimization to learning controller
parameters for a bipedal robot, where modeling the dynamics is very
difficult due to ground contacts. Using Bayesian optimization, we
sidestep this modeling issue and directly optimize the controller
parameters without the need of modeling the robot's dynamics. |