CALM: Conditional Adversarial Latent Models for Directable Virtual Characters

ACM SIGGRAPH 2023

Chen Tessler (1)    Yoni Kasten (1)    Yunrong Guo (1)    Shie Mannor (1, 2)    Gal Chechik (1, 3)    Xue Bin Peng (1, 4)

(1) NVIDIA    (2) Technion - Israel Institute of Technology    (3) Bar-Ilan University    (4) Simon Fraser University

---

Abstract

In this work, we present Conditional Adversarial Latent Models (CALM), an approach for generating diverse and directable behaviors for user-controlled interactive virtual characters. Using imitation learning, CALM learns a representation of movement that captures the complexity and diversity of human motion, and enables direct control over character movements. The approach jointly learns a control policy and a motion encoder that reconstructs key characteristics of a given motion without merely replicating it. The results show that CALM learns a semantic motion representation, enabling control over the generated motions and style-conditioning for higher-level task training. Once trained, the character can be controlled using intuitive interfaces, akin to those found in video games.

Paper: [PDF]       Code: [GitHub]       Webpage: [Link]       Preprint: [arXiv]

Bibtex

@inproceedings{
	tessler2023calm,
	author={Tessler, Chen and Kasten, Yoni and Guo, Yunrong and Mannor, Shie and Chechik, Gal and Peng, Xue Bin},
	title = {CALM: Conditional Adversarial Latent Models for Directable Virtual Characters},
	year = {2023},
	isbn = {9798400701597},
	publisher = {Association for Computing Machinery},
	address = {New York, NY, USA},
	url = {https://doi.org/10.1145/3588432.3591541},
	doi = {10.1145/3588432.3591541},
	booktitle = {ACM SIGGRAPH 2023 Conference Proceedings},
	keywords = {reinforcement learning, animated character control, adversarial training, motion capture data},
	location = {Los Angeles, CA, USA},
	series = {SIGGRAPH '23}
}