Robotics: Science and Systems XI

Autonomy Infused Teleoperation with Application to BCI Manipulation

Katharina Mulling, Arun Venkatraman, Jean-Sebastien Valois, John Downey, Jeffrey Weiss, Shervin Javdani, Martial Hebert, Andrew Schwartz, Jennifer Collinger, Andrew Bagnell


Robot teleoperation systems face a common set of challenges including latency, low-dimensional user commands, and asymmetric control inputs. User control with Brain-Computer Interfaces (BCIs) exacerbates these problems through especially noisy and erratic low-dimensional motion commands due to the difficulty in decoding neural activity. We introduce a general framework to address these challenges through a combination of computer vision, user intent inference, and arbitration between the human input and autonomous control schemes. Adjustable levels of assistance allow the system to balance the operator's capabilities and feelings of comfort and control while compensating for a task's difficulty. We present experimental results demonstrating significant performance improvement using the shared-control assistance framework on adapted rehabilitation benchmarks with two subjects implanted with intracortical brain-computer interfaces controlling a seven degree-of-freedom robotic manipulator as a prosthetic. Our results further indicate that shared assistance mitigates perceived user difficulty and even enables successful performance on previously infeasible tasks. We showcase the extensibility of our architecture with applications to quality-of-life tasks such as opening a door, pouring liquids from containers, and manipulation with novel objects in densely cluttered environments.



    AUTHOR    = {Katharina Mulling AND Arun Venkatraman AND Jean-Sebastien Valois AND John Downey AND Jeffrey Weiss AND Shervin Javdani AND Martial Hebert AND Andrew Schwartz AND Jennifer Collinger AND Andrew Bagnell}, 
    TITLE     = {Autonomy Infused Teleoperation with Application to BCI Manipulation}, 
    BOOKTITLE = {Proceedings of Robotics: Science and Systems}, 
    YEAR      = {2015}, 
    ADDRESS   = {Rome, Italy}, 
    MONTH     = {July},
    DOI       = {10.15607/RSS.2015.XI.039}