Robotics: Science and Systems XIX

Robotic Table Tennis: A Case Study into a High Speed Learning System

David B D'Ambrosio, Navdeep Jaitly, Vikas Sindhwani, Ken Oslund, Peng Xu, Nevena Lazic, Anish Shankar, Tianli Ding, Jonathan Abelian, Erwin Coumans, Gus Kouretas, Thinh Nguyen, Justin Boyd, Atil Iscen, Reza Mahjourian, Vincent Vanhoucke, Alex Bewley, Yuheng Kuang, Michael Ahn, Deepali Jain, Satoshi Kataoka, Omar E Cortes, Pierre Sermanet, Corey Lynch, Pannag R Sanketi, Krzysztof Choromanski, Wenbo Gao, Juhana Kangaspunta, Krista Reymann, Grace Vesom, Sherry Q Moore, Avi Singh, Saminda W Abeyruwan, Laura Graesser


We present a deep-dive into a real-world robotic learning system that, in previous work, was shown to be capable of hundreds of table tennis rallies with a human and has the ability to precisely return the ball to desired targets. This system puts together a highly optimized perception subsystem, a high-speed low-latency robot controller, a simulation paradigm that can prevent damage in the real world and also train policies for zero-shot transfer, and automated real world environment resets that enable autonomous training and evaluation on physical robots. We complement a complete system description, including numerous design decisions that are typically not widely disseminated, with a collection of studies that clarify the importance of mitigating various sources of latency, accounting for training and deployment distribution shifts, robustness of the perception system, sensitivity to policy hyper-parameters, and choice of action space. A video demonstrating the components of the system and details of experimental results can be found at



    AUTHOR    = {David B D'Ambrosio AND Navdeep  Jaitly AND Vikas Sindhwani AND Ken Oslund AND Peng Xu AND Nevena Lazic AND Anish Shankar AND Tianli Ding AND Jonathan Abelian AND Erwin Coumans AND Gus Kouretas AND Thinh Nguyen AND Justin Boyd AND Atil Iscen AND Reza Mahjourian AND Vincent Vanhoucke AND Alex Bewley AND Yuheng Kuang AND Michael Ahn AND Deepali Jain AND Satoshi Kataoka AND Omar E Cortes AND Pierre Sermanet AND Corey Lynch AND Pannag R Sanketi AND Krzysztof Choromanski AND Wenbo Gao AND Juhana Kangaspunta AND Krista Reymann AND Grace Vesom AND Sherry Q Moore AND Avi Singh AND Saminda W Abeyruwan AND Laura Graesser}, 
    TITLE     = {{Robotic Table Tennis: A Case Study into a High Speed Learning System}}, 
    BOOKTITLE = {Proceedings of Robotics: Science and Systems}, 
    YEAR      = {2023}, 
    ADDRESS   = {Daegu, Republic of Korea}, 
    MONTH     = {July}, 
    DOI       = {10.15607/RSS.2023.XIX.006}