How do humans navigate? We navigate with almost exclusive visual sensing and coarse floor plans. To reach a destination, we demonstrate a diverse set of skills, such as obstacle avoidance, and we use tools, such as buses and elevators, to traverse through different locations. All these are not yet possible for robots. In this work, we target three technical challenges in real-world visual navigation: (i) partial observability, (ii), multimodal behaviours, and (iii) visual complexity. Our strategy is to design proper structural components in the neural network-based controller that allows it to learn the required skills from data. Our controller enable robust navigation without high-definition map or expensive Lidar sensors. Thus, it is not only of research value but also opens up engineering opportunities.

The learned controller is currently deployed on our Boston Dynamics Spot robot, which has been navigating in the vicinity of SoC for a cumulative total of 150km.

Welcome to cite our paper:
B. Ai, W. Gao, Vinay, and D. Hsu. Deep Visual Navigation under Partial Observability. In International Conference on Robotics and Automation (ICRA), 2022.