Towards Dense Visual SLAM with Deep Neural Networks

Abstract

Simultaneous localization and mapping (SLAM) are classic research problems in computer vision. Many real applications further require dense SLAM with a monocular camera, i.e. recovering the depth of each pixel of a single moving camera. Solving this problem is crucial for obstacle avoidance in robotics and scene understanding in AR/VR. This talk presents our recent progresses on this challenging problem with deep neural networks. We first review the classic structure-from-motion (SfM) formulation to design a differentiable Gauss-Newton algorithm, which enables an end-to-end deep neural network to solve both camera pose and dense scene depth from input images. The network further learns shape priors in terms of linear subspaces to facilitate depth estimation at textureless regions. With this end-to-end formulation, we then introduce a gated recurrent unit (GRU) to enhance the iterative optimization by predicting a solution updater according to the feature space landscape of an intermediate solution. The GRU optimizer achieves better accuracy and efficiency than many previous optimization methods. In the next, we employ a neural implicit function with a differentiable renderer to encode the scene, which enforces strong shape priors can even predict maps of unobserved regions. This novel scene representation leads to promising results and outperforms several strong baselines of RGB and RGB-D SLAM.

Speaker

Dr Ping Tan will join the Hong Kong University of Science and Technology (HKUST) as a Professor. Earlier, he was the Head of the XR Lab at Alibaba DAMO Academy, and an Associate Professor at the Simon Fraser University (SFU) in Canada and the National University of Singapore (NUS) in Singapore. Dr. Tan obtained his PhD degree from HKUST in 2007, and Master and Bachelor degrees both from Shanghai Jiao Tong University (SJTU) in 2003 and 2000 respectively. His research interests mainly focus on 3D computer vision and computer graphics. He serves as an Associate Editor at the IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) and International Journal of Computer Vision (IJCV).