Filtering Gaussian Splats 🫟

⏱️ 3 min read

Maintainer

We’ve been tinkering with Gaussian-splats for a while, and today we’re releasing our small, but useful tool SplatMesh — a Wolfram Language paclet for rendering (with spark.js), and basic editing splats right inside your WLJS or Mathematica notebook workflow.

⚠️ Accurate rendering is not possible in Mathematica, but only preview as point clouds

TL;DR

In this short post I’ll walk through:

import a noisy Scaniverse capture,
define a cylindrical region of interest,
filter splats with a single predicate (functions receive [index, center, scales, quaternion, opacity, color]),
preview as 3D points (Mathematica-friendly ✨),
and export the result.

Download original notebook

SplatMesh is a small Wolfram Language paclet for importing, manipulating, and exporting Gaussian splatting meshes (.spz files). It provides a compact in-memory representation of splats (centers, scales, orientations, color and opacity data) using SPZ format, extends default Import, Export expressions, allows bulk operations, simple transformations, and a lightweight preview using Wolfram Standard Library.

Load the library

if you run it from its examples directory

PacletDirectoryLoad[NotebookDirectory[]];

or using LPM

LPMRepositories[{
    "Github" -> "https://github.com/JerryI/wl-splatmesh" -> "master"
}];

or directly as a paclet (not recommended)

PacletInstall["https://github.com/JerryI/wl-splatmesh/releases/download/v1.0.0/CoffeeLiqueur__SplatMesh-1.0.0.paclet"]

Let's import a messy scene scanned with Scaniverse App:

mesh = Import[FileNameJoin[{"attachments", "Untitled scan-663.spz 2"}], "SPZ"]

(*VB[*)BoxForm`temporalStorage$228770(*,*)(*"1:eJxdVMeO41YAW6QAQb4i2dPMOIBldQfYg3ovluRnSYs9qDw1q1nNkn4+8eSYC0ESPBEg/4w7J/v5y5cv4+8vENJy6gZQwqf7z8t0+zqaDDgW398+APv94/2Pt4+/Pt7fPr6e/obqLoJ0tpqduWApuUSP2wFYvaftN/t2ubrjvc14RU42Ky7vyoU+BUyhG9HzIlFUk6F8dtePnRxtXpwCDT05CDI4K29W+7pl+Covg1TdsQohrGudGZWpdL2/BFqShRghXI8J9Ki1cDocvfWuICOSq1h9wW4UL3AgFtt4oXl0aUzJLaXgkaYXc+qTcMxLiTq1wXpR0Wi79Zo4KFodW7LoCKigi1F9Q+I4dZ2OFDEO0pt8alL+aJ7Lq43lVEquYz70M2asq5Xpm7SfCiuIUcfD5A2H2OEpmg0wMrvrvOUAyE10xJY6SnC9F0vAT5th78KiqIGGF0ydz+5u43rQIr5gifk+P7GavgHp1KCCAjgJ8lZY7bNzMYJ2mRoBcd0QMfNd8whbyRfFRZUx39pBwelu7bYTgW9DWz1CxuYPkHOpmG2RMTdcE8WZBuO93mBNcXZsKX/cVYIfEII1kQc2xd3cUGSzujkzAmbrQNKZx9LuqJl1pUE5rVmQ0PJig26zHWpnr6HVEvqkV71VL4VNwlCEjFrDuD3Q7TyzdG6NtomRIGSfbDjdYsu02NXHN7KU8jFAEEpuM+9WTrE7n5IaqWT/SeVcsTBjIwcKuqdy7lyijjlrz+fceUGs3DmCosc+SW0fPnLfv2LX9VIxnBerKLqcxbJLRvaIDK0Ilf2ZFGpFKwZ4ls3ShvXoaz0OHq5XpQ/AltNhKvhTG/NaRve1Xp0rRfS1iOhp/ab1HHk998NGAMBwFDK62rUD8kxuPrpK0qRhtQYOZNA5js3cNWn3vaCuAoQs106pTdCAvvR5oBY020+MTO6i6k+Adm26to/sUaMVvWY4wxDYQy3dZWiodMLujuwcN2bfGb6ZA36729xyEIFL8KaNRMomkzdhw4+qUskrPVNteeYl1rqK2yOLFMFs5QLN9JJWxScyqmne2nRaK7R9ZmXd8zl3bOBFPsDpLJyxHBeVw2IYdgN1duwvLRa+ehSI1s618QqMMP/27evnvn8A9uP9R/bT5y/88gJnrqH72yeBUWq19faf6w0z/F/m80RcWMNkiuIajr++5GtVI/wX8Rtslg=="*)(*]VB*)

Next step is to render it and mask parts of interest with a simple cylinder primitive:

Graphics3D[Scale[{mesh, Cylinder[{{0.0053`,-0.0088`,0.0077`},{0.0017`,0.0902`,0.0119`}}, 0.7/30.0]}, 100], "OrthographicCameraZoom"->5]

tip

Use helper tools such as drag gizmo from the command palette. Apply it on the selected

{x,y,z}

Create a region function for faster processing:

f = RegionMember[Cylinder[{{0.0053`,-0.0088`,0.0077`},{0.0017`,0.0902`,0.0119`}}, 0.7/30.0]]

RegionMemberFunction[(*VB[*) Cylinder[{{0.0053, -0.0088, 0.0077}, {0.0017, 0.0902, 0.0119}}, 0.02333333333333333], 3, Function[{\[FormalA]}, 0 <= 0.08726351763700724 - 0.36616623947272064*\[FormalA][[1]] + 10.06957158549982*\[FormalA][[2]] + 0.4271939460515075*\[FormalA][[3]] <= 1 && 1.*\[FormalA][[1]]^2 + 0.0031165212290093134*\[FormalA][[2]]^2 + \[FormalA][[2]]*(0.00032224829507956297 - 0.08469604751778252*\[FormalA][[3]]) + \[FormalA][[1]]*(-0.009984864706313705 + 0.0725966121580993*\[FormalA][[2]] + 0.00307985627337391*\[FormalA][[3]]) + 0.9995233555767397*\[FormalA][[3]]^2 <= 0.00045509120729551124 + 0.01615430813228716*\[FormalA][[3]]] (*,*)(*"1:eJy9kMtLw0AQxuML1ItQEMFTBa+BmKR2400xRaEgJNqzm85sWNgH7APb/97drmBv3rx8zHzfzPBjbgbdsaMsy+x1kCe9WWgjPx+NoWpE6L2U1GyDzQ7izEWQhdHKtQpW3DhPRQome0G7wbV3dBDY3wZ7VpA5ZU2ZE0bneU3K+5yQuyavq6YqGlbPygLYYTxyHGTJrUvd1R7PD8erQ7ni+NVfhqyVAwJwNU6BS1SWa/UwTavnMQbutInTiTAef9ECUncW5KNbPuNaA/bRqn4ZOh/QT2OBFN6U2O7cd+Pxb7LJbm0MMP+IZU/i+6mw+A11xXS7"*)(*]VB*)]

Filter splats 🫟

Note that the filtering function accepts

[index, center, scales, quaternions, opacity, color]

filtered = SplatMeshFilter[mesh, (f[#2])&];
Graphics3D[filtered, "OrthographicCameraZoom"->26]

(*VB[*)(FrontEndRef["a0ba956a-15c3-473c-8de4-b424a2c988da"])(*,*)(*"1:eJxTTMoPSmNkYGAoZgESHvk5KRCeEJBwK8rPK3HNS3GtSE0uLUlMykkNVgEKJxokJVqamiXqGpomG+uamBsn61qkpJroJpkYmSQaJVtaWKQkAgCGbhXO"*)(*]VB*)

Try to zoom in 💡

Preview using 3D points (compatible with ✨ Mathematica ✨)

filtered["Preview"]

(*VB[*)(FrontEndRef["2cbcc7c3-ed38-49e7-bdb1-2bc78bd54719"])(*,*)(*"1:eJxTTMoPSmNkYGAoZgESHvk5KRCeEJBwK8rPK3HNS3GtSE0uLUlMykkNVgEKGyUnJSebJxvrpqYYW+iaWKaa6yalJBnqGiUlm1skpZiamBtaAgCW/hY5"*)(*]VB*)

Export our modified model:

Export["filtered.spz", filtered, "SPZ"]

Voilà 🧙🏼‍♂️
Now you can mix traditional 3D geometry with Gaussian splats directly in your notebook!

Here’s an example I made for one of the lab reports — combining a splat scene with 3D text and a green arrow:

Graphics3D[{
	Scale[mesh, 10], Green, Arrowheads[0.03], 
	Translate[{Arrow[Tube[{{0.2, 0, 0.02}, {0.4,0,0.1}}, 0.02]], Blend[{Green, Black}], 
	Text[Style["Magnetic Field", FontSize->18], {0.5, 0, 0.1}]}, {-0.3842`,-0.2584`,-0.0067`}]
}, "OrthographicCameraZoom"->4, ViewPoint->{-30,-0,30}]

Notes on performance

For the frequent evaluation of the same splat mesh, one can convert it to a frontend object (compressed representation of wolfram expression in JSON format):

fe = CreateFrontEndObject[mesh];

and then reevaluate with no loading lag:

Graphics3D[{..., fe}]

Here fe will be cached.

Notes on performance​

Notes on performance