Treffer: PGC-NeRF: probabilistic and geometric constraint-driven neural radiance fields for enhanced 3D reconstruction.

In recent years, NeRF represents a significant advance in the field of 3D reconstruction. By inputting two-dimensional (2D) images, three-dimensional (3D) scenes can be reconstructed. Compared with object scenes, urban scenes lack a surround view. In urban scenes, due to the random selection of rays during training and the lack of a surround view, there are errors in the network’s prediction of density. A critical aspect of NeRF is the probability density function derived from ray sampling points, which quantifies the contribution of each point along the ray to depth estimation. Depth estimation is accumulated from densities, resulting in inaccurate depth estimation. The surface roughness appears during the 3D mesh reconstruction through point clouds. To address this issue, this study proposes Probabilistic and Geometric Constraint-Driven Neural Radiance Fields (PGC-NeRF). In this paper, when predicting density, the KL (Kullback-Leibler) divergence is used to constrain the probability density function solving the error of density prediction. When estimating the depth, the extracted flatness features are used to constrain the geometric structure of the local mesh patches solving the surface roughness. The experimental results show that PGC-NeRF not only improves the accuracy of image rendering, but also can generate flat 3D mesh surfaces, with particularly notable enhancements on planar features such as roofs and roads that are common in urban drone surveys. The code will be made publicly available at https://github.com/hardworkingcompiler/PGC-NeRF.git. [ABSTRACT FROM AUTHOR]