GPC: Large-Scale Generative Pretraining for Transferable Motor Control
ACM SIGGRAPH 2026
Yi Shi (1, 2)Yifeng Jiang (1)Chen Tessler (1)Xue Bin Peng (1, 2)
(1) NVIDIA(2) Simon Fraser University
Abstract
Developing controllers capable of completing a wide range of tasks in a
natural and life-like manner is a key challenge in enabling practical
applications of physics-based character animation. In this work, we
introduce Generative Pretrained Controllers (GPC), which leverage
tokenization and next-token modeling to create general-purpose,
reusable generative controllers from large-scale motion datasets. Our
framework utilizes end-to-end reinforcement learning to jointly
optimize a "motion vocabulary", modeled via Finite Scalar Quantization
(FSQ), along with a corresponding control policy that can map the
discrete codes to physics-based controls. After the "codebook" has been
learned, the underlying structure of this large vocabulary is modeled by
training a GPT-style autoregressive transformer, leading to a powerful
generative controller that generates controls for a physically simulated
character by performing next-token prediction. Once the generative
controller has been trained, we propose a suite of adaptation techniques
for finetuning the controller for new downstream tasks. Our proposed
framework greatly simplifies the training process compared to previous
tokenized methods, and achieves a 99.98% success rate in reproducing a
vast corpus of motion clips. The generative controller exhibits a variety
of natural emergent behaviors, such as responsive behaviors to perturbations
and recovery behaviors after falling. This results in highly robust general
purpose controllers for a variety of downstream applications.
@inproceedings{
shi2026gpc,
title={GPC: Large-Scale Generative Pretraining for Transferable Motor Control},
author={Shi, Yi and Jiang, Yifeng and Tessler, Chen and Peng, Xue Bin},
booktitle={SIGGRAPH '26 Conference Proceedings},
year={2026}
}
