ICRA’19 Workshop: Learning Legged Locomotion

• 众所周知，腿式机器人很难控制。机器学习的最新进展表明了自动设计鲁棒灵活的运动控制器的前景。然而，这些基于学习的方法大多局限于仿真或简单的硬件平台。将这些基于学习的控制方法应用于腿式机器人仍然存在许多挑战，包括：仿真和实体之间的差距、安全探索、连续数据收集、数据高效学习算法、实验评估和硬件鲁棒性。

• 这次研讨会将双腿机器人领域和机器学习/强化学习领域的专家汇聚在一起，共同讨论用学习的方法来控制腿式机器人的最新技术和进展。

• Legged robots are notoriously difficult to control. Recent progress in machine learning has shown promises to design robust and agile locomotion controllers automatically. However, most of these learning-based methods are limited to simulation or to simple hardware platforms. Many challenges remain in bringing these learning-based control approaches to real legged robots, including the reality gap, safe exploration, continuous data collection, data-efficient learning algorithms, experimental evaluation, and hardware robustness.

• This workshop brings together experts in the fields of legged robotics and machine learning/reinforcement learning to discuss the state-of-the-art and challenges in learning-based control of legged robots.

ICML‘19 Workshop: Multi-Task and Lifelong Reinforcement Learning

• 近年来，强化学习在很多方面都取得了重大进展，它使得智能体能够完成复杂的任务，如Atari游戏、机器人操纵、虚拟角色运动控制和围棋。这些成功来自于强化学习的核心形式化：从零开始学习单任务策略或价值函数。然而，由于学习效率和目标设定等方面的问题，强化学习很难扩展到许多实际的现实问题中。最近，越来越多的研究开始利用多个强化学习任务中的结构和信息来更有效地学习复杂行为。这包括：

• 课程学习和终身学习。该方法需要学习一系列的任务，并利用它们之间的共享结构来实现知识转移。
• 目标条件下的强化学习。该方法利用所提供目标空间的结构来更快地学习许多任务。
• 元学习方法。该方法旨在学习可以快速学习新任务的高效学习算法。
• 分层强化学习。该方法可能需要具有共享结构的子目标或子任务的组合。
• 多任务和终身强化学习有可能改变传统强化学习的范式，提供更加实用和多样化的监督信息来源，同时帮助克服与强化学习相关的许多挑战，如探索、样本效率和信用分配。然而，多任务和终身强化学习领域目前仍然很年轻，它们需要研究者在问题形式化、算法和理论进展以及更好的基准和评估方面有更多的进展。

• 这次研讨会主要关注多任务和终身强化学习的算法和理论基础，以及与构建多任务智能体和终身学习基准相关的实际挑战。我们的目标是聚集研究不同问题领域（如游戏、机器人、语言等）、不同优化方法（深度学习、进化算法、基于模型的控制等）和不同形式（如上所述）的研究人员，共同来讨论多任务和终身强化学习中的前沿进展、开放问题和有意义的后续步骤。

• Significant progress has been made in reinforcement learning, enabling agents to accomplish complex tasks such as Atari games, robotic manipulation, simulated locomotion, and Go. These successes have stemmed from the core reinforcement learning formulation of learning a single policy or value function from scratch. However, reinforcement learning has proven challenging to scale to many practical real world problems due to problems in learning efficiency and objective specification, among many others. Recently, there has been emerging interest and research in leveraging structure and information across multiple reinforcement learning tasks to more efficiently and effectively learn complex behaviors. This includes:

• curriculum and lifelong learning, where the problem requires learning a sequence of tasks, leveraging their shared structure to enable knowledge transfer
• goal-conditioned reinforcement learning techniques that leverage the structure of the provided goal space to learn many tasks significantly faster
• meta-learning methods that aim to learn efficient learning algorithms that can learn new tasks quickly
• hierarchical reinforcement learning, where the reinforcement learning problem might entail a compositions of subgoals or subtasks with shared structure
• Multi-task and lifelong reinforcement learning has the potential to alter the paradigm of traditional reinforcement learning, to provide more practical and diverse sources of supervision, while helping overcome many challenges associated with reinforcement learning, such as exploration, sample efficiency and credit assignment. However, the field of multi-task and lifelong reinforcement learning is still young, with many more developments needed in terms of problem formulation, algorithmic and theoretical advances as well as better benchmarking and evaluation.

• The focus of this workshop will be on both the algorithmic and theoretical foundations of multi-task and lifelong reinforcement learning as well as the practical challenges associated with building multi-tasking agents and lifelong learning benchmarks. Our goal is to bring together researchers that study different problem domains (such as games, robotics, language, and so forth), different optimization approaches (deep learning, evolutionary algorithms, model-based control, etc.), and different formalisms (as mentioned above) to discuss the frontiers, open problems and meaningful next steps in multi-task and lifelong reinforcement learning.