Md Ishat-E-Rabban

Md Ishat-E-Rabban

SDE at Intel | M.Sc. in CS from UMD College Park | Expert in Robotic Planning, Deep Learning, and Geometric Computing | Highly Proficient in C++ and Python

Object Detection using Pseudo-LiDAR

Keywords: Stereo Depth Estimation, Point Clouds, 3D Object Detection.

Source Code: Github

In this project, we implement a novel pipeline for 3D object detection from stereo images using the Pseudo-LiDAR framework. The basic Pseudo-LiDAR pipeline takes as input a pair of images and uses a stereo depth estimation network to compute the depth map. Then the depths of the pixels are back-projected in the 3D space to create a point cloud, which is called Pseudo-LiDAR. Finally, a 3D object detection model is employed to identify the 3D objects in the Pseudo-LiDAR point cloud. The pipeline is illustrated in the following figure.

A key characteristic of both Pseudo-LiDAR is that any existing stereo depth estimation model and object detection model can be incorporated into the pipeline. In the original version, Pseudo-LiDAR used the state-of-the-art depth estimator, PSM-Net, and the then best-performing object detectors, AVOD, for object detection.

In this project, we integrate a more recent and efficient 3D object detector, called SFA3D, into the pseudo-LiDAR pipeline. We test the performance on the KITTI dataset. Our initial findings suggest that the integration of SFA3D reduces the number of false positives compared to the existing pseudo-LiDAR variants as can be seen in the above figure. The point clouds with and without depth correction are shown in the right and left columns, respectively, with the ground truth in the middle.

Papers