A survey on surface reconstruction based on 3D Gaussian splatting

3D Gaussian splatting

A 3D Gaussian is defined by parameters including position, opacity, a 3D covariance matrix, and color. All of these parameters are learnable and continuously optimized through backpropagation. The position of a 3D Gaussian can be represented as:

(1) $$\mathrm{G}(\mathrm{x})={\mathrm{e}}^{-\frac{1}{2}{x}^T{\sum }^{-1}x}$$where $\sum$ is the covariance matrix that controls the size and orientation of the 3D Gaussian distribution.

During the rendering process, the 3D Gaussian is first projected from the 3D space onto a 2D Gaussian. Given an observation transformation W and a 3D covariance matrix $\sum$, the corresponding 2D covariance matrix ${\sum }^{\mathrm{\prime }}$ is computed using a projection matrix

(2) $${\sum }^{\mathrm{\prime }}=\mathrm{J}\mathrm{W}\sum {\mathrm{W}}^{\mathrm{T}}{\mathrm{J}}^{\mathrm{T}}$$where W is the observation transformation matrix, and J is the Jacobian matrix obtained from the affine approximation of the projection transformation. Subsequently, the Gaussians within the same pixel block are sorted, and alpha blending is applied to these 2D Gaussians to compute the color of each pixel.

(3) $$\mathrm{G}({\mathrm{x}}^{\mathrm{\prime }})=\sum _{i\in N}{c}_i{\sigma }_i\prod _{j=1}^{i-1}\left(1-{\sigma }_j\right),{\sigma }_i={\alpha }_i{G}_i^{\mathrm{\prime }}(x^{\mathrm{\prime }})$$where $x^{\mathrm{\prime }}$ is the pixel position on the image plane, and ${\alpha }_i$ is the opacity of the i-th Gaussian.

Contextual concepts

Several techniques and research areas are closely related to 3D GS and are briefly introduced below.

Survey methodology

The aim of this survey is to systematically summarize surface reconstruction methods based on 3D GS, along with relevant datasets and applications. Due to the novelty of this technology, we consulted three electronic databases: the Computer Vision Foundation Open Access Database, arXiv.org, and Web of Science. A comprehensive literature search was conducted using the keywords 3D Gaussian splatting, 3D GS, mesh, and surface. We systematically assess the collected articles by considering the innovativeness and representativeness of scene representation, optimization strategies, and surface extraction methods, using these criteria to guide the selection and filtering of articles. Although it is challenging to cover all the methods related to surface reconstruction based on 3D GS, we have tried our best to include the main methods and provided comprehensive descriptions.

Scene representation

Accurately fitting 3D Gaussians to the model surface is a critical step when initializing a scene using 3D GS. The input for 3D GS includes multi-view imagery and camera pose data, while the output is typically a mesh or point cloud. This section outlines the key characteristics and motivations behind surface reconstruction techniques based on 3D GS. Depending on whether the target scenes involve motion, they can be categorized as static or dynamic. For static scenes, factors such as model size, quantity, and geometric complexity must be considered. In contrast, for dynamic scenes, modeling deformation over time introduces additional complexity, making surface structure extraction significantly more challenging.

Optimization process of 3D Gaussians

Due to the substantial differences between the rendered images of an initialized scene and real-world images, 3D Gaussians must be optimized during training to ensure accurate surface fitting to the 3D model. During this process, it is often necessary to consider the attributes of 3D Gaussians and incorporate regularization constraints into the loss function to guide learning in the desired direction. In practice, additional constraints are frequently designed to account for the specific characteristics of a given scene. Table 1 summarizes commonly used optimization constraints for geometric surface reconstruction based on 3D GS.

Surface structure extraction

In 3D reconstruction using 3D Gaussians, highly realistic real-time rendering is achieved by fitting Gaussian points to pixel-wise image contributions when rendering scenes. In addition, the geometric surface of a 3D model can be constructed by leveraging the attributes of the 3D Gaussian set. Depending on whether the scene contains static or dynamic models, surface extraction methods can be categorized into static model surface extraction and deformable model surface extraction.

For static scenes, methods such as Poisson reconstruction (Kazhdan & Hoppe, 2013) and the TSDF method (Curless & Levoy, 1996) are commonly employed to extract surface structures from 3D models represented implicitly within the scene (Lin & Li, 2024; Zhang et al., 2024a; Dai et al., 2024; Wolf, Zhang & Feng, 2024). When Poisson reconstruction is applied, Gaussian noise around the model surface can lead to floating artifacts. To mitigate this, SuGaR (Guédon & Sugar, 2024) introduces a regularization term that constrains 3D Gaussians to fit the surface more tightly. Meshes are subsequently extracted using Poisson reconstruction applied to the rendered depth maps. High-quality (Dai et al., 2024) proposes a volume-cutting strategy that aggregates Gaussian planes with Poisson reconstruction and then applies them to the fused depth maps for surface extraction. OMEGAS (Wang, Yu & Fang, 2024) incorporates semantic information and diffusion models to refine segmentation and enhance geometric detail before applying Poisson reconstruction to extract the surface. While these methods significantly improve reconstruction quality, they also entail long training times and high computational costs.

In addition to leveraging depth information, DN-Splatter (Turkulainen, Wang & Feng, 2024) improves 3D Gaussian attributes by combining priors on depth and surface normals, resulting in better alignment with real-world geometry and enabling more precise reconstruction of indoor scenes. To eliminate dependency on Gaussian normals, GauStudio (Ye, Zhang & Liu, 2024) employs a volumetric fusion method (Vizzo et al., 2022) that integrates median depth values from RGB-D cameras into the mesh, significantly enhancing both reconstruction quality and efficiency. While Poisson reconstruction can become unstable in regions with specular reflections or highlights, adaptive TSDF fusion techniques extract model surfaces in a coarse-to-fine manner, reducing memory usage and improving geometric detail. RaDe-GS (Zhang et al., 2024a) enhances the expressive capacity of 3D Gaussians by applying rasterization techniques to compute depth and normal maps, although TSDF has thus far been applied only to low-resolution voxel grids in large-scale scenes. GS2Mesh (Wolf, Zhang & Feng, 2024) advances TSDF meshing by incorporating stereo matching algorithms to construct RGB-D structures, integrating multiple views into a triangular mesh using TSDF to extract the geometric surface of the mode.

Inspired by neural implicit surface volume rendering techniques, several methods use volume rendering to optimize implicit surfaces. NeuSG (Chen, Bao & Wang, 2023) and 3DGSR (Lyu, Wang & Sun, 2024) employ volume rendering to learn neural implicit surfaces by configuring the attributes and spatial distribution of 3D Gaussians and applying unified optimization and surface constraints. These methods extract surfaces using Poisson reconstruction on Gaussians whose normals are derived from an implicit SDF field. Octree-GS (Li et al., 2024) reconstructs both the SDF and radiance field using volume rendering, encoding them within an octree structure to jointly optimize the SDF and 3D Gaussians. The Marching Cubes algorithm (Lewiner et al., 1987) is then applied to extract the zero isosurface and generate a surface mesh. These approaches simplify the modeling process by optimizing implicit surface representations via volume rendering. However, they are primarily suited for extracting foreground objects (Rosu & Permuto, 2023) and demonstrate low efficiency in capturing fine-grained geometric details and reconstructing background regions. To address this issue, GOF (Yu et al., 2024c) proposes a mesh extraction method based on tetrahedral grids. It determines the contribution of Gaussians to volume rendering by calculating intersection points between rays and Gaussians, evaluating opacity along the ray, and constructing level sets for geometry extraction. In this approach, the centers and corner points of 3D Gaussian bounding boxes are treated as vertices of the tetrahedral grid. After assessing the opacity values of the tetrahedra, the marching tetrahedra algorithm (Shen et al., 2021) is applied to extract the surface mesh. Although GOF enables detailed surface reconstruction for both foreground objects and background regions, 3D Gaussian-based methods continue to suffer from depth estimation errors, which compromise geometric consistency across multiple views.

In dynamic scenes, extracting deformable model surfaces while preserving geometric detail during motion remains a significant challenge. Traditional deformation representations rely on Laplacian coordinates (Sorkine et al., 2004; Sorkine & Alexa, 2007), Poisson equations (Yu et al., 2004), or cage-based techniques (Zhang, Zheng & Cai, 2020). Regularization or data modality constraints are often required to extract deformable surface geometry from dynamic monocular inputs (Yang et al., 2021; Xu & Harada, 2022; Yuan et al., 2022). Three-dimensional (3D) GS has emerged as a powerful approach for surface extraction in dynamic scenes due to its efficient reconstruction quality and real-time rendering capabilities. DG-Mesh (Liu, Su & Wang, 2024) treat 3D Gaussians as oriented point clouds. After generating a mesh using DPSR (Peng et al., 2021) and the Marching Cubes algorithm, the deformable Gaussians are aligned with the mesh surfaces to construct a dynamic Gaussian mesh. GaMeS (Waczyńska et al., 2024) initializes the mesh using neural geometry (Takikawa et al., 2021) or FLAME (Li et al., 2017) and parameterizes the Gaussian components at mesh vertices to align them with the surface. Although Gaussian deformation is achieved by manipulating the mesh, artifacts may emerge when large mesh areas are formed. Mesh-GS (Gao et al., 2024a) employs explicit geometric priors to initialize 3D Gaussians via mesh reconstruction. It guides both segmentation and adaptive refinement of mesh surfaces through Gaussian rendering, while mesh segmentation concurrently informs Gaussian segmentation. These explicit mesh constraints help regularize Gaussian distributions, prevent deformation artifacts, and enable efficient, high-quality reconstruction. GSDeformer (Huang & Yu, 2024) extends cage-based deformation techniques to 3D Gaussians by converting Gaussian representations into occupancy grids and calculating morphological closure. Segmenting contours and constructing a cage mesh achieve simpler and faster deformations than GaMeS, although they do not support real-time performance.

Currently, multiple methods exist for extracting model surface structures based on 3D Gaussian representations. However, due to the inherent limitations in the shape and spatial distribution of 3D Gaussians, the resulting surfaces may exhibit discontinuities or distortions that compromise reconstruction quality. Consequently, a key research challenge in this area is to accurately represent fine geometric features using 3D Gaussians and to minimize errors in surface structure extraction.

Real scene datasets

Real-scene datasets are essential resources in 3D reconstruction research, as they capture complex real-world environments and conditions. These datasets enable researchers to develop more accurate and generalizable models, thereby advancing reconstruction technologies across multiple domains.

The Tanks and Temples dataset (Knapitsch et al., 2017) is a video-sequence-based 3D reconstruction dataset acquired using industrial-grade laser scanners in real-world settings. It comprises 14 scenes, including single-object models such as “Horse” and “Lighthouse,” and large-scale indoor and outdoor environments like “Courtroom” and “Temple.” The dataset supports both camera localization and dense reconstruction tasks, encouraging time-dense sampling to enhance fidelity and realism. The DTU dataset (Aanæs et al., 2016) was captured in a controlled laboratory environment and is a large-scale multi-view stereo dataset with highly accurate camera trajectories. It includes 80 sets of 3D models captured using a structured light scanner and a calibrated camera system. To study the influence of lighting conditions on surface geometry, the dataset provides seven distinct illumination settings, enabling research on how lighting affects multi-view stereo performance, particularly in scenes with reflective or textured surfaces. The Replica dataset (Straub et al., 2019) contains 18 photorealistic 3D indoor scenes composed of dense meshes, high-dynamic-range textures, and rich semantic and instance-level annotations. The scenes were captured using custom-built RGB-D systems and included temporal variations in furniture arrangements, supporting research on semantic mapping, object permanence, and dynamic indoor reconstruction. The RGBD-SLAM Benchmark dataset (Sturm et al., 2012) comprises 39 sequences from two scenarios, including color images, depth images, and detailed camera trajectory information. It introduces two evaluation metrics and associated tools for benchmarking the performance of visual SLAM and odometry systems on RGB-D data.

The ScanNet dataset (Dai et al., 2017) is a large-scale indoor scene dataset that includes semantic annotations for objects and regions within each scene. It supports a range of tasks in 3D scene understanding, including object recognition, semantic segmentation, and scene parsing. Building on this, the ScanNet++ dataset (Yeshwanth et al., 2023) leverages advanced scanning equipment to capture high-precision indoor models. It includes accurate geometric structures, detailed texture information, and spatial relationships between objects. These features support more complex tasks such as object detection, pose estimation, and path planning, enabling robust development and evaluation of 3D scene understanding algorithms. Real-scene datasets offer advantages such as high-fidelity, stable imagery, and accurate laser-based scanning, enabling objective documentation of real-world environments that reflect their physical characteristics and inherent structural regularities. However, they also present limitations, including high data acquisition costs, smaller dataset sizes, and relatively homogeneous scene content. Moreover, because the data are collected in uncontrolled, real-world conditions, ambient lighting variability can introduce noise and reconstruction errors that may adversely affect the accuracy of 3D model generation.

Synthetic scene datasets

Synthetic datasets generated through computer simulations or algorithmic rendering are characterized by their low cost, scalability, and consistently high quality, making them essential in 3D reconstruction research.

The Mip-NeRF360 dataset (Barron et al., 2022) models unbounded 3D scenes using nonlinear scene parametrization, real-time distillation, and orientation distortion regularization. It comprises nine complex scenes featuring intricate objects and backgrounds, enabling multi-view synthesis and detailed depth map generation. This makes it particularly suitable for reconstructing large, complex, and unbounded environments. The Blended-MVS dataset (Yao et al., 2020) includes a wide variety of scenes—ranging from urban areas and buildings to small-scale objects—and provides image sets for training, validation, and testing. It supports novel view synthesis by generating depth maps and color images from unstructured camera trajectories, enhancing the generalization capabilities of 3D reconstruction networks.

The NeRF Synthetic dataset (Mildenhall et al., 2022) contains eight scenes depicting objects such as chairs, drum kits, and hot dogs. Each scene offers 100 images across training, validation, and testing sets, making it well-suited for evaluating deep learning-based reconstruction algorithms on object-centric tasks. The Matrix City dataset (Li et al., 2023a), built with Unity Engine 5, provides high-resolution images of large-scale urban environments. It includes aerial data from two city maps spanning 28 km², comprising 67,000 aerial views and 452,000 street-level images. This dataset supports configurable environmental parameters—such as lighting, weather, people, and vehicles—and includes depth, normals, and decomposed BRDF materials, making it ideal for city-scale neural rendering and 3D reconstruction research. Synthetic datasets allow the simulation of scenes that are difficult to observe or capture in real-world conditions, thereby enriching dataset diversity and supporting broader generalization scenarios. However, simulations inevitably introduce discrepancies in how real-world phenomena are represented. These inconsistencies can be amplified during model training, leading to reduced generalization performance.

Both synthetic and real-world datasets offer critical support for 3D reconstruction research. Table 3 summarizes the performance metrics of several surface reconstruction methods across three benchmark datasets. Specifically, reconstruction speed was evaluated on the DTU dataset; the F1 score was used to assess accuracy on the Tanks and Temples dataset; and peak signal-to-noise ratio (PSNR) was employed to measure reconstruction quality in dynamic scenes using the D-NeRF dataset (Pumarola et al., 2021). All reported experimental results are derived from original publications, ensuring transparency and reproducibility. While real-world datasets enhance model robustness and realism, their time-consuming collection and annotation processes limit their use for rapid prototyping and algorithm iteration. In contrast, synthetic datasets reduce reliance on manual data acquisition and support large-scale data generation. Each dataset type has distinct advantages and complementary strengths. Joint training with both types can improve performance in diverse 3D reconstruction tasks. Future research will benefit from high-resolution, large-scale, and semantically enriched datasets with fine-grained geometric labels to support increasingly complex reconstruction objectives.

Automatic driving

The applications of 3D GS in autonomous driving are diverse and impactful. Constructing geometric models of roads, buildings, and obstacles is critical for enabling autonomous vehicles to accurately interpret their environments, thereby improving navigation safety and operational reliability. As illustrated in Fig. 2, DriveX (Yang et al., 2024) present a novel framework that integrates generative priors into the reconstruction of the driving scene from single-trajectory recorded videos, addressing limitations in extrapolating novel views beyond a recorded trajectory. Road surface reconstruction based on Gaussian Splatting (RoGS) (Feng et al., 2024) uses 2D Gaussians to reconstruct road surfaces at large scale. These reconstructions support lane detection and automatic annotation, contributing to more precise path planning in real-time scenarios. TCLC-GS (Zhao, Huang & Fang, 2024) combines the strengths of LiDAR and camera sensors by aligning 3D Gaussians with mesh structures, generating rich geometric and color representations of urban street scenes. This integration enhances environmental perception and accelerates real-time rendering, thereby improving the situational awareness and safety of autonomous systems. DrivingForward (Tian et al., 2024) introduces a feed-forward GS framework that jointly trains a pose estimation network, a depth prediction network, and a Gaussian rendering network. This system predicts both the appearance and position of 3D Gaussians from surround-view inputs, enabling the reconstruction of dynamic driving environments from flexible viewpoints.

Digital man

The use of digital human models spans diverse domains, including education, entertainment, and virtual reality. GauFace (Qin et al., 2024) employs geometric priors and constrained optimization to construct a clean and structured Gaussian representation, enabling high-fidelity, real-time facial interactions on platforms such as computers and mobile devices. This method supports efficient animation and physics-based rendering of facial assets. MoSca (Lei, Wu & He, 2024) develops animatable digital human avatars from both multi-view and monocular video inputs. By initializing 3D Gaussians on mesh surfaces, the system reconstructs detailed geometric models of the human body, achieving realistic lighting and interactive rendering, thereby extending the application of digital humans in virtual environments. As illustrated in Fig. 3, the proposed framework (Nguyen et al., 2025) synergistically combines 3D Gaussian Splatting with adaptive light-field shading to achieve real-time photorealistic rendering of dynamic facial animations, while maintaining sub-millisecond latency through optimized neural computation pipelines. MeshAvatar (Chen et al., 2025) extracts explicit triangular meshes from implicit SDF fields to represent digital avatars. It combines this geometry with physics-based rendering to decompose and reconstruct both shape and texture components, enabling flexible editing and manipulation of digital human models.

Large-scale scene reconstruction

Large-scale scene reconstruction is essential for applications such as digital city modeling, road planning, and immersive virtual environments. These use cases require not only photorealistic rendering but also high-precision geometric modeling. Real-time Gaussian SLAM (RTG-SLAM) (Peng et al., 2024) introduces a method that utilizes both transparent and opaque 3D Gaussians to reduce the total number of Gaussians needed for accurate surface representation. This approach improves the realism of rendered scenes without compromising geometric fidelity and enables real-time reconstruction of diverse real-world environments, including corridors, storerooms, residences, and offices. As shown in Fig. 4, Vast Gaussian (Lin et al., 2024) adopts a divide-and-conquer strategy to partition large-scale scenes into smaller spatial cells, distributing training views and point clouds across these subregions. This method supports parallel optimization and seamless merging, enabling efficient 3D reconstruction and real-time rendering of complex environments such as urban structures, water networks, and transportation infrastructure. In contrast to techniques that rely on Structure-from-Motion (SfM) for initial point cloud generation, Gaussian Pro (Cheng et al., 2024) reconstructs scenes using predefined geometric shapes and applies block matching to initialize 3D Gaussians with precise positions and orientations. This method is particularly effective for optimizing large-scale reconstructions involving untextured or low-textured surfaces.

3D editing

3D editing refers to the reconstruction, modification, and refinement of 3D models and is widely applied in domains such as game development, film production, industrial design, and architectural planning. Editing and optimizing 3D models enhance their reusability and adaptability across varied applications. Direct (Lin & Li, 2024) introduces a differentiable marching algorithm to learn a mesh representation from which 3D Gaussians are generated. By manipulating this mesh, users can flexibly control both the appearance and geometry of 3D models. SC-GS (Huang et al., 2024b) enables motion editing by adjusting sparse control points, allowing for rigid pose deformations while preserving high-fidelity rendering quality. As illustrated in Fig. 5, Mani-GS (Gao et al., 2024b) proposes an adaptive triangular shape-aware Gaussian binding method that supports large-scale deformations, localized adjustments, and soft-body simulations for enhanced model expressiveness. Inspired by 3D GS techniques, Tetrahedron (Gu et al., 2024) introduces tetrahedron splatting for AI-driven modeling, enabling the generation and manipulation of 3D content from text prompts. This approach significantly broadens the creative and functional scope of 3D editing workflows.