What Is Gaussian Splatting on iPhone and How Do You Make One?
Gaussian splatting is a way of turning ordinary photos or video into a photorealistic 3D scene made of a very large number of soft, semi-transparent, colored blobs that render in real time from any angle. On an iPhone you make one by recording a slow, overlapping orbit of your subject and then processing it either on the phone, in a cloud app, or on a desktop with open-source tools.
The rest of this article covers what a splat actually is, which route to pick, how to capture so the result holds up, and when a splat is the wrong output entirely.
What is a Gaussian splat, in plain terms?
The technique was introduced in the paper 3D Gaussian Splatting for Real-Time Radiance Field Rendering by Kerbl, Kopanas, Leimkühler and Drettakis, published in ACM Transactions on Graphics in 2023. Instead of building a surface out of triangles, it represents the scene as a cloud of 3D Gaussians.
Each Gaussian is a small ellipsoid with:
- a position in space
- a shape and orientation (it can be round, flat or stretched)
- an opacity
- a color that can change depending on the viewing direction
To show an image, the renderer projects these blobs onto the screen, sorts them by depth and blends them together. Because the color is view-dependent, a splat can reproduce things meshes struggle with: glossy highlights, fine foliage, hair, fuzzy edges and soft reflections.
The blobs are not hand-made. They are optimized from your photos: the software starts with a rough point cloud, renders it from each camera position, compares the render with the real photo and nudges every Gaussian until the two match. An academic survey of the field describes the method as a transformative approach within radiance field methods, the same family as NeRFs, but with much faster rendering.
What do you need before you start?
Every splat pipeline needs two things: images, and the exact position and angle of the camera for each image. The second part is where most of the hidden work happens.
On a desktop, camera poses usually come from COLMAP, a free, open-source Structure-from-Motion and Multi-View Stereo tool that recovers camera positions and a sparse 3D point cloud from a set of photos. That sparse cloud becomes the starting point for the Gaussians.
On an iPhone with LiDAR, there is an extra source of information. Apple's ARKit 4 session explains the sceneDepth frame semantic, which exposes a per-frame depth map that can be unprojected into a 3D point cloud. Capture apps can record that depth alongside the color frames and tracked camera poses, which helps when the scene has blank walls or repetitive textures that confuse pure photo matching.
You do not strictly need LiDAR. Plenty of splats are made from plain video. LiDAR mainly helps with tracking and initial geometry; the visual quality still comes from the photos.
Which way should you make one on an iPhone?
There are three practical routes: fully on the phone, through a cloud service, or capture on the phone and train on a computer. If what you actually need is a measurable or printable model rather than a splat, a different kind of app fits better, so that is included as well.
| Option | How it works | What you get | Choose it when |
|---|---|---|---|
| MagiScan | Turns the phone into a 3D scanner and measuring tool | Scans exported for 3D printing, AR and e-commerce | You need dimensions, a printable file or an AR product model rather than a splat |
| Scantic | Reconstructs the splat locally on the iPhone, works offline, no account required; needs iOS 17.0 or later and an A12 Bionic chip or later (review) | An interactive splat plus the raw files | You want privacy, no upload and no internet dependency |
| Polycam | Cloud-based Gaussian splatting from photos or video, on web, iOS and Android (tool page) | A splat scene processed on their servers | You want the least setup and may switch between devices |
| Gaussian SplatKing | Dual-lens photo/video capture and a LiDAR mode that writes native COLMAP-format data on-device (App Store) | Training-ready capture data for a downstream pipeline | You plan to train on a desktop and want to skip pose estimation |
| Desktop pipeline | COLMAP for poses, then the official reference implementation for training | Full control over training; the release links viewers and evaluation datasets | You have a capable GPU and want to tune quality yourself |
For a first splat, the on-device or cloud route is the sensible choice. You learn what good capture looks like without fighting installation problems. Move to the desktop pipeline once you know your captures are solid and you want more control.
How do you capture an iPhone video that makes a good splat?
Capture decides most of the outcome. Training cannot invent views you never recorded, and blurry frames poison the optimization.
Plan the path
Walk a full circle around the subject at chest height, then a second circle lower down looking slightly up, then a third higher up looking down. For a room or outdoor space, walk slowly through it rather than standing in the middle and spinning; a camera that only rotates without moving gives the software no parallax to work out depth.
Polycam's page shows a user who shot a 90-second iPhone video and processed it, which gives a sense of scale: a single unhurried pass, not a ten-minute shoot, is often enough for a modest object or scene.
Keep frames sharp and consistent
- Move slowly, especially when turning. Motion blur is the most common cause of mushy splats; the fixes in why 3D scans come out blurry apply directly here.
- Use even, steady light. Changing clouds or a flickering light make the same surface look different between frames.
- Lock exposure and focus if your capture app allows it, so brightness does not pump as you walk.
- Keep lots of overlap between consecutive views. If you shoot stills instead of video, the coverage logic in how many photos you need for photogrammetry carries over, since the pose step is the same.
Remove things that move
People walking past, a pet, swaying branches or a screen playing video all break the assumption that the scene is static. They show up as ghosts or smears. Wait for the frame to clear or crop those areas later.
What goes wrong, and how do you fix it?
Floaters. Small clouds of stray blobs hovering in the air, usually near the edges or in the sky. They come from areas seen in only a few frames. Most viewers let you crop with a bounding box; better coverage at capture time prevents them.
Smearing from new angles. The splat looks perfect along your walking path and falls apart when you orbit somewhere you never filmed. The fix is more viewpoints, particularly from above and below.
Pose estimation failure. Processing stops or produces a scrambled scene. This happens with plain white walls, glass, mirrors and repeated patterns, because the software cannot match features between frames. Adding textured objects to the scene or using a LiDAR capture mode helps. The LiDAR vs photogrammetry comparison explains why depth sensing copes with featureless surfaces that photo matching cannot.
Huge files. Splats can be heavy compared with a decimated mesh. Crop away background you do not need before sharing.
Is a splat the right output for your job?
This is the question most people skip. A splat is excellent for looking at something: a walkthrough of a space, a showcase of a product's appearance, a record of a location, a visual effects plate.
It is a poor fit when you need to do something with the geometry. There is no clean, watertight surface, so you cannot send it straight to a slicer for 3D printing, you cannot reliably take a caliper-style measurement off a fuzzy blob edge, and most CAD tools will not open it. The output is usually a point-based file such as PLY rather than the STL, OBJ or USDZ meshes that printing and AR workflows expect; the guide to exporting STL, OBJ or USDZ covers those formats.
If you are a seller who needs product dimensions, an engineer checking a part, or someone printing a replacement, a mesh-based scanner such as MagiScan, which measures and exports for printing, AR and e-commerce, will get you there more directly than a splat.
Many practitioners capture both: a splat for the visual record and a mesh scan for dimensions. The capture habits are nearly identical, so it costs little extra time on site.
What should you do next?
Start small. Pick an object with plenty of texture on a table in soft daylight, shoot one slow, three-height orbit, and process it with an on-device or cloud app. Look at it from angles you did not film to see where coverage was thin, then reshoot with that in mind. Once your captures are consistently clean, try the COLMAP and reference implementation route if you want finer control. And when the job calls for numbers or a printable model rather than a visual, reach for a measurement-focused scanner like MagiScan instead.
Frequently Asked Questions
Do I need an iPhone with LiDAR to make a Gaussian splat?
No. Splats can be trained from ordinary photos or video, and Polycam describes building them from iPhone video. LiDAR depth, exposed through ARKit's sceneDepth, can help with tracking and featureless surfaces, and some apps offer a dedicated LiDAR capture mode.
Can I make a splat entirely on the phone without uploading anything?
Yes. According to its review, Scantic performs the reconstruction locally on the iPhone, works offline and does not require an account. It needs iOS 17.0 or later and an A12 Bionic chip or later.
Can I 3D print or measure a Gaussian splat?
Not directly. A splat is a cloud of soft blobs with no watertight surface, so slicers and CAD tools cannot use it as-is, and edges are too fuzzy for dependable measurements. For printing or dimensions, capture a mesh scan instead.
Is Gaussian splatting the same as a NeRF?
They belong to the same family of radiance field methods and both learn a scene from posed photos. The difference is the representation: NeRFs use a neural network queried along camera rays, while Gaussian splatting uses explicit 3D Gaussians that can be rasterized, which is what makes real-time rendering practical.