From cca26b6c96c11ba9299f41f845d48d35d1396fdd Mon Sep 17 00:00:00 2001 From: zhangyu Date: Wed, 29 Nov 2023 20:02:38 +0800 Subject: [PATCH] wip --- docs/reading/papers/202311-sugar.md | 49 +++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+) create mode 100644 docs/reading/papers/202311-sugar.md diff --git a/docs/reading/papers/202311-sugar.md b/docs/reading/papers/202311-sugar.md new file mode 100644 index 00000000..be7f0c80 --- /dev/null +++ b/docs/reading/papers/202311-sugar.md @@ -0,0 +1,49 @@ +--- +keywords: gaussian splatting +--- + +# SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Rendering + +precise and extremely fast mesh extraction from 3D Gaussian Splatting + +## Contributions + +- a regularization term that encourages the gaussians to align well with the surface of the scene +- a method that exploits this alignment to extract a mesh from the Gaussians using Poisson reconstruction, which is fast, scalable, and preserves details, in contrast to the Marching Cubes algorithm usually applied to extract meshes from Neural SDFs +- an optional refinement strategy that binds gaussians to the surface of the mesh, and jointly optimizes these Gaussians and the mesh through Gaussian splatting rendering + +--- + +the gaussians are unstructured + +Gaussian Splatting performs densification in order to capture details of the scene with highfidelity... This results in a density function that is close to zero almost everywhere(?), and the Marching Cubes algorithm(?) fails to extract proper level sets(?) of such a sparse density function even with a fine voxel grid + +Instead, Poisson reconstruction algorithm, scalable + +## Related works + +light fields(?) Their work emphasized the importance of efficiently traversing volumetric data to produce realistic images(?) +Traditional mesh-based IBR methods +Volumetric IBR methods - NeRF +Hybrid IBR methods +Point-based IBR methods + +## Methods + +regularization term: derive an SDF from the gaussians, assuming (needs to reword) +- well spread (the density of any position can be well approximated by only considering the closest gaussian) +- flat (one of scaling factors close to 0) +- opaque (alpha = 1) + +computing a slightly different loss relying on an SDF rather than on density further increases the alignment of gaussians with the surface of the scene +the zero-crossings(?) of the Signed Distance Function + +We also add a regularization term to encourage the normals of SDF ¯f and the normals of SDF ˆf to also be similar(?) + +mesh extraction: ... + +refinement: instantiate a set of new thin 3D gaussians from each mesh triangle. The gaussians have only 2 learnable scaling factors and only 1 learnable 2D rotation + +## Results + +slightly(?) worse than vanilla 3DGS and Mip-NeRF360