Xue Bin (Jason) Peng

I'm an Assistant Professor at Simon Fraser University (SFU) and a Research Scientist at NVIDIA. I received a Ph.D. from UC Berkeley, advised by Professor Sergey Levine and Professor Pieter Abbeel. Prior to that, I received an M.Sc from the University of British Columbia, advised by Professor Michiel van de Panne. My work lies in the intersection between computer graphics and machine learning, with a focus on reinforcement learning for motion control of simulated characters. I have previously worked for Sony, Google Brain, OpenAI, Adobe Research, Disney Research, Microsoft (343 Industries), and Capcom.

Prospective students: Information is available here.

---

Publications

  — 2024 —

	MaskedMimic: Unified Physics-Based Character Control Through Masked Motion Inpainting Chen Tessler, Yunrong Guo, Ofir Nabati, Gal Chechik, Xue Bin Peng ACM Transactions on Graphics (Proc. SIGGRAPH Asia 2024) [Project page] [Paper]
	Interactive Character Control with Auto-Regressive Motion Diffusion Models Yi Shi, Jingbo Wang, Xuekun Jiang, Bingkun Lin, Bo Dai, Xue Bin Peng ACM Transactions on Graphics (Proc. SIGGRAPH 2024) [Project page] [Paper]
	SuperPADL: Scaling Language-Directed Physics-Based Control with Progressive Supervised Distillation Jordan Juravsky, Yunrong Guo, Sanja Fidler, Xue Bin Peng ACM SIGGRAPH 2024 [Project page] [Paper]
	Flexible Motion In-betweening with Diffusion Models Setareh Cohan, Guy Tevet, Daniele Reda, Xue Bin Peng, Michiel van de Panne ACM SIGGRAPH 2024 [Project page] [Paper]
	Generating Human Interaction Motions in Scenes with Text Control Hongwei Yi, Justus Thies, Michael J. Black, Xue Bin Peng, Davis Rempe European Conference on Computer Vision (ECCV 2024) [Project page] [Paper]
	Multi-Track Timeline Control for Text-Driven 3D Human Motion Generation Mathis Petrovich, Or Litany, Umar Iqbal, Michael J. Black, Gul Varol, Xue Bin Peng, Davis Rempe CVPR Workshop on Human Motion Generation (CVPR Workshop 2024) [Project page] [Paper]
	Trajeglish: Traffic Modeling as Next-Token Prediction Jonah Philion, Xue Bin Peng, Sanja Fidler International Conference on Learning Representations (ICLR 2024) [Project page] [Paper]

  — 2023 —

	Learning Physically Simulated Tennis Skills from Broadcast Videos Haotian Zhang, Ye Yuan, Viktor Makoviychuk, Yunrong Guo, Sanja Fidler, Xue Bin Peng, Kayvon Fatahalian ACM Transactions on Graphics (Proc. SIGGRAPH 2023) Best Paper Honourable Mention [Project page] [Paper]
	Synthesizing Physical Character-Scene Interactions Mohamed Hassan, Yunrong Guo, Tingwu Wang, Michael Black, Sanja Fidler, Xue Bin Peng ACM SIGGRAPH 2023 [Project page] [Paper]
	CALM: Conditional Adversarial Latent Models for Directable Virtual Characters Chen Tessler, Yoni Kasten, Yunrong Guo, Shie Mannor, Gal Chechik, Xue Bin Peng ACM SIGGRAPH 2023 [Project page] [Paper]
	Video Prediction Models as Rewards for Reinforcement Learning Alejandro Escontrela, Ademi Adeniji, Wilson Yan, Ajay Jain, Xue Bin Peng, Ken Goldberg, Youngwoon Lee, Danijar Hafner, Pieter Abbeel Neural Information Processing Systems (NeurIPS 2023) [Project page] [Paper]
	Creating a Dynamic Quadrupedal Robotic Goalkeeper with Reinforcement Learning Xiaoyu Huang, Zhongyu Li, Yanzhen Xiang, Yiming Ni, Yufeng Chi, Yunhao Li, Lizhi Yang, Xue Bin Peng, and Koushil Sreenath IEEE International Conference on Intelligent Robots and Systems (IROS 2023) [Project page] [Paper]
	Learning and Adapting Agile Locomotion Skills by Transferring Experience Laura Smith, J. Chase Kew, Tianyu Li, Linda Luu, Xue Bin Peng, Sehoon Ha, Jie Tan, Sergey Levine Robotics: Science and Systems (RSS 2023) [Project page] [Paper]
	RoboPianist: Dexterous Piano Playing with Deep Reinforcement Learning Kevin Zakka, Philipp Wu, Laura Smith, Nimrod Gileadi, Taylor Howell, Xue Bin Peng, Sumeet Singh, Yuval Tassa, Pete Florence, Andy Zeng, Pieter Abbeel Conference on Robot Learning (CoRL 2023) [Project page] [Paper]
	Trace and Pace: Controllable Pedestrian Animation via Guided Trajectory Diffusion Davis Rempe, Zhengyi Luo, Xue Bin Peng, Ye Yuan, Kris Kitani, Karsten Kreis, Sanja Fidler, Or Litany Conference on Computer Vision and Pattern Recognition (CVPR 2023) [Project page] [Paper]
	Robust and Versatile Bipedal Jumping Control through Reinforcement Learning Zhongyu Li, Xue Bin Peng, Pieter Abbeel, Sergey Levine, Glen Berseth, Koushil Sreenath Robotics: Science and Systems (RSS 2023) [Project page] [Paper]
	GenLoco: Generalized Locomotion Controllers for Quadrupedal Robots Gilbert Feng, Hongbo Zhang, Zhongyu Li, Xue Bin Peng, Bhuvan Basireddy, Linzhu Yue, Zhitao Song, Lizhi Yang, Yunhui Liu, Koushil Sreenath, Sergey Levine Conference on Robot Learning (CoRL 2023) [Project page] [Paper]

  — 2022 —

	Unsupervised Reinforcement Learning with Contrastive Intrinsic Control Michael Laskin, Hao Liu, Xue Bin Peng, Denis Yarats, Aravind Rajeswaran, Pieter Abbeel Neural Information Processing Systems (NeurIPS 2022) [Project page] [Paper]
	PADL: Language-Directed Physics-Based Character Control Jordan Juravsky, Yunrong Guo, Sanja Fidler, Xue Bin Peng ACM SIGGRAPH Asia 2022 [Project page] [Paper]
	Adversarial Motion Priors Make Good Substitutes for Complex Reward Functions Alejandro Escontrela, Xue Bin Peng, Wenhao Yu, Tingnan Zhang, Atil Iscen, Ken Goldberg, Pieter Abbeel IEEE International Conference on Intelligent Robots and Systems (IROS 2022) [Project page] [Paper]
	Hierarchical Reinforcement Learning for Precise Soccer Shooting Skills using a Quadrupedal Robot Yandong Ji, Zhongyu Li, Yinan Sun, Xue Bin Peng, Sergey Levine, Glen Berseth, Koushil Sreenath IEEE International Conference on Intelligent Robots and Systems (IROS 2022) [Project page] [Paper]
	ASE: Large-Scale Reusable Adversarial Skill Embeddings for Physically Simulated Characters Xue Bin Peng, Yunrong Guo, Lina Halper, Sergey Levine, Sanja Fidler ACM Transactions on Graphics (Proc. SIGGRAPH 2022) [Project page] [Paper]
	Legged Robots that Keep on Learning: Fine-Tuning Locomotion Policies in the Real World Laura Smith, J. Chase Kew, Xue Bin Peng, Sehoon Ha, Jie Tan, Sergey Levine IEEE International Conference on Robotics and Automation (ICRA 2022) [Project page] [Paper]

  — 2021 —

	Deep Reinforcement Learning for Modeling Human Locomotion Control in Neuromechanical Simulation Seungmoon Song, Łukasz Kidziński, Xue Bin Peng, Carmichael Ong, Jennifer Hicks, Sergey Levine, Christopher G. Atkeson, Scott L. Delp Journal of NeuroEngineering and Rehabilitation 2021 [Project page] [Paper]
	Offline Meta-Reinforcement Learning with Advantage Weighting Eric Mitchell, Rafael Rafailov, Xue Bin Peng, Sergey Levine, Chelsea Finn International Conference on Machine Learning (ICML 2021) [Project page] [Paper]
	AMP: Adversarial Motion Priors for Stylized Physics-Based Character Control Xue Bin Peng, Ze Ma, Pieter Abbeel, Sergey Levine, Angjoo Kanazawa ACM Transactions on Graphics (Proc. SIGGRAPH 2021) [Project page] [Paper]
	Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robots Zhongyu Li, Xuxin Cheng, Xue Bin Peng, Pieter Abbeel, Sergey Levine, Glen Berseth, Koushil Sreenath IEEE International Conference on Robotics and Automation (ICRA 2021) [Project page] [Paper]

  — 2020 —

	Learning Agile Robotic Locomotion Skills by Imitating Animals Xue Bin Peng, Erwin Coumans, Tingnan Zhang, Tsang-Wei Edward Lee, Jie Tan, Sergey Levine Robotics: Science and Systems (RSS 2020) Best Paper Award [Project page] [Paper]
	Reinforcement Learning with Competitive Ensembles of Information-Constrained Primitives Anirudh Goyal, Shagun Sodhani, Jonathan Binas, Xue Bin Peng, Sergey Levine, Yoshua Bengio International Conference on Learning Representations (ICLR 2020) [Project page] [Paper]

  — 2019 —

	Reward-Conditioned Policies Aviral Kumar, Xue Bin Peng, Sergey Levine arXiv Preprint 2019 [Project page] [Paper]
	On Learning Symmetric Locomotion Farzad Adbolhosseini, Hung Yu Ling, Zhaoming Xie, Xue Bin Peng, Michiel van de Panne ACM SIGGRAPH Conference on Motion, Interaction, and Games (MIG 2019) [Project page] [Paper]
	Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learning Xue Bin Peng, Aviral Kumar, Grace Zhang, Sergey Levine arXiv Preprint 2019 [Project page] [Paper]
	MCP: Learning Composable Hierarchical Control with Multiplicative Compositional Policies Xue Bin Peng, Michael Chang, Grace Zhang, Pieter Abbeel, Sergey Levine Neural Information Processing Systems (NeurIPS 2019) [Project page] [Paper]
	Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow Xue Bin Peng, Angjoo Kanazawa, Sam Toyer, Pieter Abbeel, Sergey Levine International Conference on Learning Representations (ICLR 2019) [Project page] [Paper]

  — 2018 —

	SFV: Reinforcement Learning of Physical Skills from Videos Xue Bin Peng, Angjoo Kanazawa, Jitendra Malik, Pieter Abbeel, Sergey Levine ACM Transactions on Graphics (Proc. SIGGRAPH Asia 2018) [Project page] [Paper]
	DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills Xue Bin Peng, Pieter Abbeel, Sergey Levine, Michiel van de Panne ACM Transactions on Graphics (Proc. SIGGRAPH 2018) [Project page] [Paper]
	Sim-to-Real Transfer of Robotic Control with Dynamics Randomization Xue Bin Peng, Marcin Andrychowicz, Wojciech Zaremba, Pieter Abbeel IEEE International Conference on Robotics and Automation (ICRA 2018) [Project page] [Paper]

  — 2017 —

	DeepLoco: Developing Locomotion Skills Using Hierarchical Deep Reinforcement Learning Xue Bin Peng, Glen Berseth, KangKang Yin, Michiel van de Panne ACM Transactions on Graphics (Proc. SIGGRAPH 2017) [Project page] [Paper]
	Learning Locomotion Skills Using DeepRL: Does the Choice of Action Space Matter? Xue Bin Peng, Michiel van de Panne ACM SIGGRAPH / Eurographics Symposium on Computer Animation 2017 Best Student Paper Award [Project page] [Paper]

  — 2016 —

Terrain-Adaptive Locomotion Skills Using Deep Reinforcement Learning Xue Bin Peng, Glen Berseth, Michiel van de Panne ACM Transactions on Graphics (Proc. SIGGRAPH 2016) [Project page] [Paper]

  — 2015 —

Dynamic Terrain Traversal Skills Using Reinforcement Learning Xue Bin Peng, Glen Berseth, Michiel van de Panne ACM Transactions on Graphics (Proc. SIGGRAPH 2015) [Project page] [Paper]

---

Thesis

	Ph.D. Thesis Acquiring Motor Skills Through Motion Imitation and Reinforcement Learning University of California, Berkeley 2021 [Project page] [Thesis]
	M.Sc. Thesis Developing Locomotion Skills with Deep Reinforcement Learning University of British Columbia 2017 [Project page] [Thesis]