Robotics: Science and Systems XVII
Manipulator-Independent Representations for Visual Imitation
Yuxiang Zhou, Yusuf Aytar, Konstantinos BousmalisAbstract:
Imitation learning is an effective tool for robotic learning tasks where specifying a reinforcement learning (RL) reward is not feasible or where the exploration problem is particularly difficult. Imitation; typically behavior cloning or inverse RL; derive a policy from a collection of first-person action-state trajectories. This is contrary to how humans and other animals imitate: we observe a behavior; even from other species; understand its perceived effect on the state of the environment; and figure out what actions our body can perform to reach a similar outcome. In this work; we explore the possibility of third-person visual imitation of manipulation trajectories; only from vision and without access to actions; demonstrated by embodiments different to the ones of our imitating agent. Specifically; we investigate what would be an appropriate representation method with which an RL agent can visually track trajectories of complex manipulation behavior —non-planar with multiple-object interactions— demonstrated by experts with different embodiments. We present a way to train manipulator-independent representations (MIR) that primarily focus on the change in the environment and have all the characteristics that make them suitable for cross-embodiment visual imitation with RL: domain-invariant; temporally smooth; and actionable. We show that with our proposed method our agents are able to imitate; with complex robot control; trajectories from a variety of embodiments and with significant visual and dynamics differences; e.g. simulation-to-reality gap.
Bibtex:
@INPROCEEDINGS{Zhou-RSS-21, AUTHOR = {Yuxiang Zhou AND Yusuf Aytar AND Konstantinos Bousmalis}, TITLE = {{Manipulator-Independent Representations for Visual Imitation}}, BOOKTITLE = {Proceedings of Robotics: Science and Systems}, YEAR = {2021}, ADDRESS = {Virtual}, MONTH = {July}, DOI = {10.15607/RSS.2021.XVII.002} }