Terrain-Adaptive Locomotion Skills Using Deep Reinforcement Learning

Transactions on Graphics (Proc. ACM SIGGRAPH 2016)

Xue Bin Peng    Glen Berseth    Michiel van de Panne

University of British Columbia

---

Abstract

Reinforcement learning offers a promising methodology for developing skills for simulated characters, but typically requires working with sparse hand-crafted features. Building on recent progress in deep reinforcement learning (DeepRL), we introduce a mixture of actor-critic experts (MACE) approach that learns terrain-adaptive dynamic locomotion skills using high-dimensional state and terrain descriptions as input, and parameterized leaps or steps as output actions. MACE learns more quickly than a single actor-critic approach and results in actor-critic experts that exhibit specialization. Additional elements of our solution that contribute towards efficient learning include Boltzmann exploration and the use of initial actor biases to encourage specialization. Results are demonstrated for multiple planar characters and terrain classes.

Paper: [PDF]       Code: [GitHub]

Videos

Bibtex

@article{
	2016-TOG-deepRL,
	author = {Peng, Xue Bin and Berseth, Glen and van de Panne, Michiel},
	title = {Terrain-adaptive Locomotion Skills Using Deep Reinforcement Learning},
	journal = {ACM Trans. Graph.},
	issue_date = {July 2016},
	volume = {35},
	number = {4},
	month = jul,
	year = {2016},
	issn = {0730-0301},
	pages = {81:1--81:12},
	articleno = {81},
	numpages = {12},
	url = {http://doi.acm.org/10.1145/2897824.2925881},
	doi = {10.1145/2897824.2925881},
	acmid = {2925881},
	publisher = {ACM},
	address = {New York, NY, USA},
	keywords = {physics-based characters, reinforcement learning},
}