Gaussian splatting for site context in architectural visualization

What a splat actually is

3D Gaussian splatting, introduced in a 2023 SIGGRAPH paper from Inria and the Max Planck Institute, represents a scene as millions of small, soft, coloured ellipsoids optimised from photographs. Instead of building a mesh and projecting textures onto it, the method learns where these blobs should sit and how they should look from each angle, then rasterises them very quickly. The result can be explored in real time and looks remarkably like the photographs it was trained on.

For architectural visualization, the appeal is context. Modelling a Victorian terrace, a weathered market hall or a mature tree line convincingly takes days. A splat captures all of that wear, dirt and irregularity from a walk around the site with a phone or a mirrorless camera.

It is worth being clear about what splats are not. They are not a measured survey, they do not produce clean geometry, and they do not react to new lighting the way a mesh with materials does. The lighting is baked into the capture from the day you shot it.

Capturing a site for context

Good captures start with good photography. Shoot on an overcast day if you can, because hard shadows will be locked into the result. Keep exposure and white balance fixed. Walk slow arcs around each facade at two or three heights, with plenty of overlap between frames. Two to four hundred stills, or a steady 4K video sampled to frames, is typical for a short street frontage.

Processing tools now range from open-source trainers built on the original research code to commercial apps such as Polycam, Luma and Postshot, plus plugins that load splats directly into Unreal Engine, Unity and Blender. Training a street on a recent GPU takes minutes to an hour depending on resolution. Expect to clean the result: floating artefacts in the sky, ghosted pedestrians and smeared cars all need cropping or deleting.

Scale and alignment matter more than resolution. Bring in a few surveyed points, or at least known dimensions such as a door height or a kerb line, and align the splat to your project coordinates. A splat that is two percent too large will make the proposed building look small, and clients notice that faster than they notice blurry brickwork.

Combining splats with modelled architecture

The most convincing approach treats the splat as a backdrop and the proposal as hero geometry. Model the new building, the ground plane immediately around it and any surfaces that receive its shadows or reflections. Let the splat fill everything beyond that. This keeps the parts the client is buying sharp and controllable while the neighbourhood feels real.

Lighting needs to be matched to the capture, not the other way round. Because the splat cannot be relit, set your sun and sky in the engine to agree with the conditions on capture day. This is another reason overcast captures are easier: soft light is forgiving and blends with almost any engine sky.

Shadows and occlusion are the remaining seam. A new building should cast shadows onto existing pavements and neighbouring facades, but a splat does not receive shadows natively in most pipelines. The common fix is a simple proxy mesh of the street, invisible to camera but receiving shadows, composited over the splat. In real-time engines, some plugins now support this directly.

Where splats do not belong

Splats do not belong in verified views or any image presented as accurate for planning. They are not measured, they can distort fine geometry and they record a moment that may already have changed. Use a survey, photogrammetry mesh or calibrated photography there. They are also a poor choice for anything the camera gets close to, because detail falls apart into soft blobs at short range.

Used for what they are good at, splats close one of the oldest gaps in arch viz: the gap between a precise new building and a vague, generic context. They let a proposal sit in the actual street it will stand in, with its real textures and history, at a fraction of the modelling time.