3dmatch Toolbox

3DMatch is a research codebase that uses a 3D ConvNet to learn local geometric descriptors from RGB-D reconstructions. It addresses the challenge of matching local features on noisy, low-resolution, and incomplete depth data, outperforming traditional histogram-based methods like FPFH and Spin Images. The tool is designed for computer vision researchers and robotics engineers who need to establish correspondences between partial 3D scans for applications such as 3D reconstruction, object model alignment, and mesh surface correspondence. 3DMatch extracts local 3D patches from RGB-D reconstructions, converts them into a volumetric truncated distance function (TDF) representation, and trains a 3D ConvNet to map similar patches to nearby points in feature space. The unsupervised training leverages millions of correspondence labels from existing RGB-D reconstructions without manual annotation. The project includes benchmarks for keypoint matching, geometric registration, and RGB-D reconstruction, along with pre-trained models and training/testing code on GitHub. The descriptor generalizes across different tasks and spatial scales, as demonstrated in the Amazon Picking Challenge and mesh surface correspondence tasks. Compared to handcrafted descriptors, 3DMatch consistently achieves lower error rates on benchmark leaderboards. However, it is a research tool, not a polished product, requiring significant expertise to deploy.