DeepLoco: Dynamic Locomotion Skills Using Hierarchical Deep Reinforcement Learning

Transactions on Graphics (Proc. ACM SIGGRAPH 2017)

Xue Bin Peng(1)    Glen Berseth(1)    KangKang Yin(2)    Michiel van de Panne(1)

(1)University of British Columbia    (2)National University of Singapore

---

Abstract

Learning physics-based locomotion skills is a difficult problem, leading to solutions that typically exploit prior knowledge of various forms. In this paper we aim to learn a variety of environment-aware locomotion skills with a limited amount of prior knowledge. We adopt a two-level hierarchical control framework. First, low-level controllers are learned that operate at a fine timescale and which achieve robust walking gaits that satisfy stepping-target and style objectives. Second, high-level controllers are then learned which plan at the timescale of steps by invoking desired step targets for the low-level controller. The high-level controller makes decisions directly based on high-dimensional inputs, including terrain maps or other suitable representations of the surroundings. Both levels of the control policy are trained using deep reinforcement learning. Results are demonstrated on a simulated 3D biped. Low-level controllers are learned for a variety of motion styles and demonstrate robustness with respect to forcebased disturbances, terrain variations, and style interpolation. High-level controllers are demonstrated that are capable of following trails through terrains, dribbling a soccer ball towards a target location, and navigating through static or dynamic obstacles.

Paper: [PDF]       Code: [GitHub]       Media: [Popular Mechanics]

Videos

Bibtex

@article{
	2017-TOG-deepLoco,
	author = {Peng, Xue Bin and Berseth, Glen and Yin, Kangkang and Van De Panne, Michiel},
	title = {DeepLoco: Dynamic Locomotion Skills Using Hierarchical Deep Reinforcement Learning},
	journal = {ACM Trans. Graph.},
	issue_date = {July 2017},
	volume = {36},
	number = {4},
	month = jul,
	year = {2017},
	issn = {0730-0301},
	pages = {41:1--41:13},
	articleno = {41},
	numpages = {13},
	url = {http://doi.acm.org/10.1145/3072959.3073602},
	doi = {10.1145/3072959.3073602},
	acmid = {3073602},
	publisher = {ACM},
	address = {New York, NY, USA},
	keywords = {locomotion skills, motion control, physics-based character animation},
}