-
StochasticSplats: Stochastic Rasterization for Sorting-Free 3D Gaussian Splatting
Authors:
Shakiba Kheradmand,
Delio Vicini,
George Kopanas,
Dmitry Lagun,
Kwang Moo Yi,
Mark Matthews,
Andrea Tagliasacchi
Abstract:
3D Gaussian splatting (3DGS) is a popular radiance field method, with many application-specific extensions. Most variants rely on the same core algorithm: depth-sorting of Gaussian splats then rasterizing in primitive order. This ensures correct alpha compositing, but can cause rendering artifacts due to built-in approximations. Moreover, for a fixed representation, sorted rendering offers little…
▽ More
3D Gaussian splatting (3DGS) is a popular radiance field method, with many application-specific extensions. Most variants rely on the same core algorithm: depth-sorting of Gaussian splats then rasterizing in primitive order. This ensures correct alpha compositing, but can cause rendering artifacts due to built-in approximations. Moreover, for a fixed representation, sorted rendering offers little control over render cost and visual fidelity. For example, and counter-intuitively, rendering a lower-resolution image is not necessarily faster. In this work, we address the above limitations by combining 3D Gaussian splatting with stochastic rasterization. Concretely, we leverage an unbiased Monte Carlo estimator of the volume rendering equation. This removes the need for sorting, and allows for accurate 3D blending of overlapping Gaussians. The number of Monte Carlo samples further imbues 3DGS with a way to trade off computation time and quality. We implement our method using OpenGL shaders, enabling efficient rendering on modern GPU hardware. At a reasonable visual quality, our method renders more than four times faster than sorted rasterization.
△ Less
Submitted 31 March, 2025;
originally announced March 2025.
-
3D Gaussian Splatting as Markov Chain Monte Carlo
Authors:
Shakiba Kheradmand,
Daniel Rebain,
Gopal Sharma,
Weiwei Sun,
Jeff Tseng,
Hossam Isack,
Abhishek Kar,
Andrea Tagliasacchi,
Kwang Moo Yi
Abstract:
While 3D Gaussian Splatting has recently become popular for neural rendering, current methods rely on carefully engineered cloning and splitting strategies for placing Gaussians, which can lead to poor-quality renderings, and reliance on a good initialization. In this work, we rethink the set of 3D Gaussians as a random sample drawn from an underlying probability distribution describing the physic…
▽ More
While 3D Gaussian Splatting has recently become popular for neural rendering, current methods rely on carefully engineered cloning and splitting strategies for placing Gaussians, which can lead to poor-quality renderings, and reliance on a good initialization. In this work, we rethink the set of 3D Gaussians as a random sample drawn from an underlying probability distribution describing the physical representation of the scene-in other words, Markov Chain Monte Carlo (MCMC) samples. Under this view, we show that the 3D Gaussian updates can be converted as Stochastic Gradient Langevin Dynamics (SGLD) updates by simply introducing noise. We then rewrite the densification and pruning strategies in 3D Gaussian Splatting as simply a deterministic state transition of MCMC samples, removing these heuristics from the framework. To do so, we revise the 'cloning' of Gaussians into a relocalization scheme that approximately preserves sample probability. To encourage efficient use of Gaussians, we introduce a regularizer that promotes the removal of unused Gaussians. On various standard evaluation scenes, we show that our method provides improved rendering quality, easy control over the number of Gaussians, and robustness to initialization.
△ Less
Submitted 12 February, 2025; v1 submitted 15 April, 2024;
originally announced April 2024.
-
Accelerating Neural Field Training via Soft Mining
Authors:
Shakiba Kheradmand,
Daniel Rebain,
Gopal Sharma,
Hossam Isack,
Abhishek Kar,
Andrea Tagliasacchi,
Kwang Moo Yi
Abstract:
We present an approach to accelerate Neural Field training by efficiently selecting sampling locations. While Neural Fields have recently become popular, it is often trained by uniformly sampling the training domain, or through handcrafted heuristics. We show that improved convergence and final training quality can be achieved by a soft mining technique based on importance sampling: rather than ei…
▽ More
We present an approach to accelerate Neural Field training by efficiently selecting sampling locations. While Neural Fields have recently become popular, it is often trained by uniformly sampling the training domain, or through handcrafted heuristics. We show that improved convergence and final training quality can be achieved by a soft mining technique based on importance sampling: rather than either considering or ignoring a pixel completely, we weigh the corresponding loss by a scalar. To implement our idea we use Langevin Monte-Carlo sampling. We show that by doing so, regions with higher error are being selected more frequently, leading to more than 2x improvement in convergence speed. The code and related resources for this study are publicly available at https://ubc-vision.github.io/nf-soft-mining/.
△ Less
Submitted 29 November, 2023;
originally announced December 2023.
-
Online Optimization for Randomized Network Resource Allocation with Long-Term Constraints
Authors:
Ahmed Sid-Ali,
Ioannis Lambadaris,
Yiqiang Q. Zhao,
Gennady Shaikhet,
Shima Kheradmand
Abstract:
In this paper, we study an optimal online resource reservation problem in a simple communication network. The network is composed of two compute nodes linked by a local communication link. The system operates in discrete time; at each time slot, the administrator reserves resources for servers before the actual job requests are known. A cost is incurred for the reservations made. Then, after the c…
▽ More
In this paper, we study an optimal online resource reservation problem in a simple communication network. The network is composed of two compute nodes linked by a local communication link. The system operates in discrete time; at each time slot, the administrator reserves resources for servers before the actual job requests are known. A cost is incurred for the reservations made. Then, after the client requests are observed, jobs may be transferred from one server to the other to best accommodate the demands by incurring an additional transport cost. If certain job requests cannot be satisfied, there is a violation that engenders a cost to pay for each of the blocked jobs. The goal is to minimize the overall reservation cost over finite horizons while maintaining the cumulative violation and transport costs under a certain budget limit. To study this problem, we first formalize it as a repeated game against nature where the reservations are drawn randomly according to a sequence of probability distributions that are derived from an online optimization problem over the space of allowable reservations. We then propose an online saddle-point algorithm for which we present an upper bound for the associated K-benchmark regret together with an upper bound for the cumulative constraint violations. Finally, we present numerical experiments where we compare the performance of our algorithm with those of simple deterministic resource allocation policies.
△ Less
Submitted 3 April, 2024; v1 submitted 24 May, 2023;
originally announced May 2023.
-
Preimplantation Blastomere Boundary Identification in HMC Microscopic Images of Early Stage Human Embryos
Authors:
Shakiba Kheradmand,
Parvaneh Saeedi,
Jason Au,
John Havelock
Abstract:
We present a novel method for identification of the boundary of embryonic cells (blastomeres) in Hoffman Modulation Contrast (HMC) microscopic images that are taken between day one to day three. Identification of boundaries of blastomeres is a challenging task, especially in the cases containing four or more cells. This is because these cells are bundled up tightly inside an embryo's membrane and…
▽ More
We present a novel method for identification of the boundary of embryonic cells (blastomeres) in Hoffman Modulation Contrast (HMC) microscopic images that are taken between day one to day three. Identification of boundaries of blastomeres is a challenging task, especially in the cases containing four or more cells. This is because these cells are bundled up tightly inside an embryo's membrane and any 2D image projection of such 3D embryo includes cell overlaps, occlusions, and projection ambiguities. Moreover, human embryos include fragmentation, which does not conform to any specific patterns or shape. Here we developed a model-based iterative approach, in which blastomeres are modeled as ellipses that conform to the local image features, such as edges and normals. In an iterative process, each image feature contributes only to one candidate and is removed upon being associated to a model candidate. We have tested the proposed algorithm on an image dataset comprising of 468 human embryos obtained from different sources. An overall Precision, Sensitivity and Overall Quality (OQ) of 92%, 88% and 83% are achieved.
△ Less
Submitted 14 October, 2019;
originally announced October 2019.
