Robotics: Science and Systems XXII

Learning to Localize Reference Trajectories in Image-Space for Visual Navigation

Finn Lukas Busch, Matti Vahs, Quantao Yang, Jesús Gerardo Ortega Peimbert, Yixi Cai, Jana Tumova, Olov Andersson

Abstract:

We present LoTIS, a model for visual navigation that provides robot-agnostic image-space guidance by localizing a reference RGB trajectory in the robot’s current view, without requiring camera calibration, poses, or robot-specific training. Instead of predicting actions tied to specific robots, we predict the image-space coordinates of the reference trajectory as they would appear in the robot’s current view. This creates robot-agnostic visual guidance that easily integrates with local planning. Consequently, our model's predictions provide guidance zero-shot across diverse embodiments. By decoupling perception from action and learning to localize trajectory points rather than imitate behavioral priors, we enable a cross-trajectory training strategy that learns robust invariance to viewpoint and camera changes. We outperform state-of-the-art methods by 20-50 percentage points in success rate on forward navigation, and paired with a local planner we achieve 94-98% success rate across diverse sim and real environments. Furthermore, we achieve over 5× improvements on challenging tasks where baselines fail, such as backward traversal. The system is straightforward to use: we show how even a video from a handheld phone camera directly enables different robots to navigate to any point on the trajectory.

Download:

Bibtex:

  
@INPROCEEDINGS{BuschF-RSS-26, 
    AUTHOR    = {Finn Lukas Busch AND Matti Vahs AND Quantao Yang AND Jesús Gerardo Ortega Peimbert AND Yixi Cai AND Jana Tumova AND Olov Andersson}, 
    TITLE     = {{Learning to Localize Reference Trajectories in Image-Space for Visual Navigation}}, 
    BOOKTITLE = {Proceedings of Robotics: Science and Systems}, 
    YEAR      = {2026}, 
    ADDRESS   = {Sydney, Australia}, 
    MONTH     = {July}, 
    DOI       = {10.15607/RSS.2026.XXII.131} 
}