Skip to main content

VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection

Citations: 1,323 | Paper Published: 18 Jun 2018 | Institutions: Apple Inc. | Authors: Yin Zhou, Oncel Tuzel

Abstract in 2 Lines

The paper proposes a 3D detection network called VoxelNet that can accurately detect objects in LiDAR point clouds without the need for manual feature engineering. VoxelNet divides the point cloud into equally spaced 3D voxels and transforms a group of points within each voxel into a unified feature representation, which is then connected to a region proposal network to generate detections.

Contributions​

  • Proposing VoxelNet, a generic 3D detection network that unifies feature extraction and bounding box prediction into a single stage, end-to-end trainable deep network.
  • Introducing the voxel feature encoding (VFE) layer that transforms a group of points within each voxel into a unified feature representation.
  • Demonstrating that VoxelNet outperforms the state-of-the-art LiDAR based 3D detection methods by a large margin on the KITTI car detection benchmark.
  • Showing that VoxelNet achieves highly encouraging results in detecting pedestrians and cyclists from LiDAR point cloud.

Practical Implications​

  • VoxelNet can be used for accurate detection of objects in 3D point clouds, which is useful in many applications such as autonomous navigation, housekeeping robots, and augmented/virtual reality.
  • VoxelNet removes the need for manual feature engineering, which can save time and effort in developing LiDAR-based 3D detection systems.
  • VoxelNet can operate directly on sparse 3D points and capture 3D shape information effectively, which can improve the accuracy of object detection.
  • VoxelNet's efficient implementation benefits from point cloud sparsity and parallel processing on a voxel grid, which can improve the speed of object detection.

Methods​

Methodology
  • Voxelization: The point cloud is divided into equally spaced 3D voxels.
  • Voxel Feature Encoding (VFE) layer: A group of points within each voxel is transformed into a unified feature representation.
  • Region Proposal Network (RPN): The volumetric representation of the point cloud is connected to a RPN to generate detections.
  • Training: The network is trained end-to-end using a combination of classification and regression losses.
  • Evaluation: The performance of the network is evaluated on the KITTI car detection benchmark.

Data​

The data used in this paper is the KITTI dataset, which is a benchmark for LiDAR-based 3D object detection. The dataset consists of LiDAR point clouds, camera images, and ground truth annotations for cars, pedestrians, and cyclists. The authors used less than 4000 training point clouds to train their network and introduced three different forms of data augmentation to reduce overfitting.

Results​

Performance Analysis
  • For car detection, VoxelNet achieves 83.23% average precision (AP) on the moderate level and 79.02% AP on the hard level of the KITTI benchmark, outperforming the state-of-the-art methods by a large margin.
  • For pedestrian detection, VoxelNet achieves 44.43% AP on the moderate level and 34.01% AP on the hard level, which is also a significant improvement over the state-of-the-art methods.
  • For cyclist detection, VoxelNet achieves 54.73% AP on the moderate level and 45.87% AP on the hard level, which is again a significant improvement over the state-of-the-art methods.

Limitations​

  • The proposed method is designed for LiDAR-based 3D object detection and may not be directly applicable to other types of sensors or modalities.
  • The proposed method requires a large amount of computational resources and may not be suitable for real-time applications on low-power devices.

Table of Contents